Multiple HTTP request in C#

Posted on

Problem :

I need to send about 200 HTTP requests in parallel to different servers and get response.
I use HttpWebRequest class in C#.
But I don’t see good time enhancement when requests are handled in parallel.
For example if one request needs 3sec to get response, 2 request in parallel – 6sec, 3 requests – 8 secs, 4 requests – 11sec …
It is not good, I hope that there should be best time, about 10 sec for 200 requests.
It looks like only 2-3 requests performs in parallel, but timeout starts immediately after WebRequest object creation.
I tried set DefaultConnectionLimit and MaxServicePoints values, but id didn’t help. As I understand these parameters for number of requests to one site in parallel. I need requests to different sites.

Code example that I use to test:

ArrayList a = new ArrayList(150);

for (i = 50; i < 250; i++ )
{
   a.Add("http://207.242.7." + i.ToString() + "/");
}

for (i = 0; i < a.Count; i++)
{
    Thread t = new Thread(new ParameterizedThreadStart(performRequest));
    t.Start(a[i]);
}


static void performRequest(object ip)
{
      HttpWebRequest req = (HttpWebRequest)WebRequest.Create((stirng)ip);

      HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
}

–°an anyone ever encountered such a problem?
Thank you for any suggestions.

Solution :

Instead of starting up your own threads try using the asynchronous methods of HttpWebRequest such as HttpWebRequest.BeginGetResponse and HttpWebRequest.BeginGetRequestStream.

This might help – http://social.msdn.microsoft.com/forums/en-US/netfxnetcom/thread/1f863f20-09f9-49a5-8eee-17a89b591007

Suggests there is a limit on the number of TCP connections allowed, but that you can increase the limit

Use asynchronous web requests in stead.

Edit: http://msdn.microsoft.com/en-us/library/86wf6409(VS.71).aspx

you can try this :

try
        {
            List<Uri> uris = new List<Uri>();
            uris.Add(new Uri("http://www.google.fr"));
            uris.Add(new Uri("http://www.bing.com"));

            Parallel.ForEach(uris, u =>
           {
               WebRequest webR = HttpWebRequest.Create(u);
               HttpWebResponse webResponse = webR.GetResponse() as HttpWebResponse;
           });
        }
        catch (AggregateException exc)
        {
            exc.InnerExceptions.ToList().ForEach(e =>
                {
                    Console.WriteLine(e.Message);
                });
        }

This is the code of android application of sending the request. How can we use the upper code like:

params.add(new BasicNameValuePair("key", "value");
HttpPost request = new HttpPost();
List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("key", "value");// How can we give the value in this format format
post.setEntity(new UrlEncodedFormEntity(params));

httpClient.execute(request);

Leave a Reply

Your email address will not be published. Required fields are marked *