C# code to download webpage

By codegambit52 on Oct 11, 2013

This code helps you in downloading any web page.

using System;
using System.Net;
using System.IO;

public class Test
{
    public static void Main (string[] args)
    {
        if (args == null || args.Length == 0)
        {
            throw new ApplicationException ("bestfishingrodsreview.com"); // Replace it with the URL that you want to download
        }
        WebClient client = new WebClient ();

// Add user agent header if the URL contains query.

        client.Headers.Add ("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");

        Stream data = client.OpenRead (args[0]);
        StreamReader reader = new StreamReader (data);
        string s = reader.ReadToEnd ();
        Console.WriteLine (s);
        data.Close ();
        reader.Close ();
    }
}

Like Page Follow Author

Comments

Sorasyn - Oct 11, 2013

Wouldn't it be best to throw an NullReferenceException or an ArgumentNullException instead of an ApplicationException with the only helpful data being a website URL?

 throw new ArgumentNullException("Command line arguments expected, but not found.");

2 Responses

codegambit52 - Oct 11, 2013

@SunnyD I think u r rite. Let me write an updated version of this snippet. Thanks for pointing it out.

Sorasyn - Oct 11, 2013

To be clear, does this just grab the page's resulting HTML code and print it onto the console in a giant glob of text and symbols? What is the purpose of doing so?

You could also rewrite the StreamReader code to use a "using" code block, which would forego the need to close both data, and reader, as they will both be closed automatically.

using (StreamReader sr = new StreamReader(client.OpenRead(args[0]))) {
   while (!sr.EndOfStream)
      Console.WriteLine(sr.ReadLine());
}