Title: Retrieve data from a URL
Question: How do I fetch text from a URL?
Answer:
This is a follow up to article 1240 on "Checking if a URL is valid" which returns the data at the web page in a string. If it fails you should get a Status:xxx where xxx is the status or nothing at all if there isn't a web server at the url you try.
It uses the InternetReadfile function to read the data in 4kb chunks. The actual size of the buffer is irrelevant unless (as in this case) it is declared local to the function. This takes up stack space and a large buffer could potentially lead to a stack overflow if there were many functions nested in the call stack. Either move it somewhere non stack based or keep it small.
One thing to watch is the string conversion. If you are using buffer data, make sure you add a #0 on the end and use pchars to convert before ending up with strings.
Function FetchHTML(url:string):string;
var
databuffer : array[0..4095] of char;
ResStr : string;
hSession, hfile, hRequest: hInternet;
dwindex,dwcodelen,datalen,dwread,dwNumber:cardinal;
dwcode : array[1..20] of char;
res : pchar;
Str : pchar;
begin
ResStr := '';
if pos('http://',lowercase(url))=0 then
url := 'http://'+url;
hSession := InternetOpen('InetURL:/1.0',
INTERNET_OPEN_TYPE_PRECONFIG,nil, nil, 0);
if assigned(hsession) then
begin
hfile := InternetOpenUrl(
hsession,
pchar(url),
nil,
0,
INTERNET_FLAG_RELOAD,
0);
dwIndex := 0;
dwCodeLen := 10;
HttpQueryInfo(hfile, HTTP_QUERY_STATUS_CODE,
@dwcode, dwcodeLen, dwIndex);
res := pchar(@dwcode);
dwNumber := sizeof(databuffer)-1;
if (res ='200') or (res ='302') then
begin
while (InternetReadfile(hfile,@databuffer,dwNumber,DwRead)) do
begin
if dwRead =0 then
break;
databuffer[dwread]:=#0;
Str := pchar(@databuffer);
resStr := resStr + Str;
end;
end
else
ResStr := 'Status:'+res;
if assigned(hfile) then
InternetCloseHandle(hfile);
end;
InternetCloseHandle(hsession);
Result := resStr;
end;
This code was written with the help of 'Essential WinInet' by Aaron Skonnard and if you are interested in this subject I strongly suggest you buy the book. ISBN 0-201-37936-8. The code in the book is in C/C++ but isn't too difficult to convert to delphi.