wget is a webpage downloader available from GNU at http://www.gnu.org/software/wget/.
I found this since weblech (http://weblech.sourceforge.net)
mentioned it, and I finally realized it would be better to use, since
weblech is only in the pre-alpha stage.
I was recently able to download and build ver 1.10 as of July 1, 2005,
although it wasn't easy. Here's the compiled executable for DOS (in
Windows 32): wget1v10.zip -- but let me be the first to warn you to make
sure that your anti-virus scanner is up-to-date before downloading and
executing ".exe" files from strange websites!
You'll also need some libraries or DLLs from openSSL http://www.openssl.org/related/binaries.html
-- I specifically used those at http://www.slproweb.com/products/Win32OpenSSL.html,
which was the only offered link. I downloaded and installed
Win32OpenSSL-v0.9.7g.exe from there.
You can find the wget manual from the GNU link above -- it's pretty
versatile.
In order to build ver 1.10, I had to work around a MSVC 6.0 compiler
bug, as the expert on the mailing list advised me. See my posts of June
30 and July 1, 2005: "I had an unsuccessful
compile of ver 1.10 on Windows 2000 and also XP with MSVC++ 6.0"
(and "Getting past an https login screen") on the mailing list at http://www.mail-archive.com/wget@sunsite.dk/.
The work-around was to build http.c and retr.c without optimization --
I posted my makefile modification on the list in those threads.
Normal build steps include running configure.bat --msvc to set up for
compiling with MSVC ver 6.0 and making sure that VCVARS32.BAT (in
C:\Program Files\Microsoft Visual Studio\VC98\Bin) is run to set up
environment variables. You have to read the readme in the Windows
directory of the wget download and copy (from the installed OpenSSL
directory) the whole inc32\openssl directory and its contents to
"C:\Program Files\Microsoft Visual Studio\VC98\Include\openssl" (ie
copy the former openssl directory to the VC98\Include directory),
and from out32dll (in the OpenSSL directory) the two needed libraries
(libeay32.lib and ssleay32.lib) to "C:\Program Files\Microsoft Visual
Studio\VC98\lib".