I’ve been working on a small piece of C# software this week that posts data to an HTTP server (which handles credit card processing), parses the results and then returns the results to a C# client. Pretty easy to do, right? First you create a HttpWebRequest object:
String url = "http://server/path";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
and then you post the data:
byte[] requestBytes = System.Text.Encoding.ASCII.GetBytes (some_data);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = requestBytes.Length;
Stream requestStream = req.GetRequestStream();
requestStream.Write(requestBytes,0,requestBytes.Length);
requestStream.Close();
Finally, you retrieve the HTML returned from the server:
// note: exception handling removed for easier reading
StreamReader sr = null;
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
sr = new StreamReader(res.GetResponseStream(), System.Text.Encoding.ASCII);
String line = streamReader.ReadToEnd();
streamReader.Close();
The reason that I was working on it was that the application was returning random exceptions of the form:
Error reading response stream: System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
Googling for this error message didn’t leave me with much. There were a smattering of posts on various web forums about the error, but not a whole lot of solutions. Long story short, I fired up TcpTrace and modified the KeepAlive property (setting it to false) of the HttpWebRequest object on a whim and voila! The application worked again. Best I can tell the HTTP server I’m working against doesn’t handle HTTP posts using Connection: Keep-Alive properly. For whatever reason it decides that the third request in a Keep-Alive connection should be closed.
Broadly, the reason I bring this up is because I think it’s important for all web developers to have an in-depth understanding of what’s going on under the hood of HTTP. Knowing the advantages and disadvantages of things like the HTTP Keep-Alive header becomes invaluable whenever you have to drop down to manually sending and receiving HTTP.
More pointedly, it was interesting to find out a couple tidbits about how .NET handles HTTP connections. First, by default .NET is configured (via machine.config) to use whatever proxy settings you have for Internet Explorer. You can turn this off by modifying the:
configuration/system.net/defaultProxy/proxy
element. Second, also by default, machine.config only allows .NET applications to make 2 persistent connections to external resources. You can modify/view this as well:
configuration/system.net/connectionManagement
Finally, the HttpWebRequest and it’s parent WebRequest again, by default, are set to use Keep-Alive connections.