RoBlog - Refactoring reality

Monday, November 15, 2004

Frustrating http connection behaviour in .Net (C#)

Current Status: Red

I seem to be having a bit of a frustrating time with http connections in .Net at the moment.
Basically I am working on a project in which many subsystems use http to communicate with eachother. Once in a while, my .Net client will fail with a "System.Net.WebException: The underlying connection was closed: Unable to connect to the remote server." exception.
The remote server can be an IIS server or a Jetty server running on Linux - I see the same problem with both.

To try and reproduce this problem I wrote a very simple client app that requests a single static page from a web server. It loops round making the request repeatedly, up to 100000 times.
Here is the static Main method from my console application:

static void Main(string[] args)
int i = 0;
for (i = 0; i < 10000; i++)

HttpWebRequest req = (HttpWebRequest)
using (HttpWebResponse resp =
byte[] buffer = new byte[1024*1024];
int read = 0;
Stream stream = resp.GetResponseStream();
read = stream.Read(buffer, 0, 1024*1024);
while (read !=0);
req = null;

if (i % 1000 == 0)
catch (Exception ex)
Console.WriteLine("Exception at iteration " + i);
Console.WriteLine( ex.ToString());


For this example I created a very small html file to put on my server - only 50 bytes.
I see some extremely peculiar behaviour when I run this application; generally the first time it runs, it will process many thousands of iterations before falling over with the aforementioned exception. Once it has failed, if I attempt to run it again straight away the application fails in a matter of a few 10's or 100's of iterations. Why?
It makes no difference to the overall behaviour if I swap my HttpWebRequest stuff for a System.Net.WebClient.
It makes no difference if I insert a GC.Collect() in the loop.
What is going on here and what is 'remembering' the fact that the client has previously failed?

I have to admit to being stumped for the time being.
Does anyone else see this behaviour?


  • At 10:21 PM, Blogger Rob Levine said…

    One quick addendum:
    This behaviour only seems to occur if the test file is very small. Once the file is a few K or more, the problem disappears.


<< Home