How to use a proxy with in HtmlAgilityPack

Question

I need to use a proxy with HtmlAgilityPack. I give a link to my app RefURL. After that I want the app get url from a proxy address. For instance "101.109.44.157:8080"

I searched and found out this:

WebClient wc = new WebClient();
wc.Proxy = new WebProxy(host,port);
var page = wc.DownloadString(url);

and used it like this.

RefURL = new Uri(refLink.Text);

WebClient wc = new WebClient();
wc.Proxy = new WebProxy("101.109.44.157:8080");
var page = wc.DownloadString(RefURL);

RefURL.ToString();
HtmlWeb web = new HtmlWeb();
HtmlAgilityPack.HtmlDocument doc = web.Load(RefURL.ToString());

but it does not work!

Answer 1

The proxy IP is not responding but also you're not passing web proxy in this code line:

HtmlAgilityPack.HtmlDocument doc = web.Load(RefURL.ToString());

Should be:

HtmlAgilityPack.HtmlDocument doc = web.Load(RefURL.ToString(),"GET", webProxy);

First step is finding "fresh proxy IP" list, for example:

• https://free-proxy-list.net/uk-proxy.html
• https://hidemy.name/en/proxy-list/
• http://free-proxy.cz
• http://nntime.com

Most of these addresses would work for few hours. Check out how to set proxy IP in a browser. If the proxy is anonymous, this page should be unable to detect your location and IP.

Once you have a proxy IP and port that works, you can create webProxy object or simply pass IP and port.

string RefURL = "https://www.whatismyip.com/";
string myProxyIP = "119.81.197.124"; //check this is still available
int myPort = 3128;
string userId = string.Empty; //leave it blank
string password = string.Empty;
try
{
    HtmlWeb web = new HtmlWeb();
    var doc = web.Load(RefURL.ToString(), myProxyIP, myPort, userId, password);
    Console.WriteLine(doc.DocumentNode.InnerHtml);
}
catch (Exception ex)
{
    Console.WriteLine(ex.Message);
}