I’ve been working on a small piece of C# software this week that posts data to an HTTP server (which handles credit card processing), parses the results and then returns the results to a C# client. Pretty easy to do, right? First you create a HttpWebRequest object:
String url = "http://server/path";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
and then you post the data:
byte[] requestBytes = System.Text.Encoding.ASCII.GetBytes (some_data);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = requestBytes.Length;
Stream requestStream = req.GetRequestStream();
requestStream.Write(requestBytes,0,requestBytes.Length);
requestStream.Close();
Finally, you retrieve the HTML returned from the server:
// note: exception handling removed for easier reading
StreamReader sr = null;
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
sr = new StreamReader(res.GetResponseStream(), System.Text.Encoding.ASCII);
String line = streamReader.ReadToEnd();
streamReader.Close();
The reason that I was working on it was that the application was returning random exceptions of the form:
Error reading response stream: System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
Googling for this error message didn’t leave me with much. There were a smattering of posts on various web forums about the error, but not a whole lot of solutions. Long story short, I fired up TcpTrace and modified the KeepAlive property (setting it to false) of the HttpWebRequest object on a whim and voila! The application worked again. Best I can tell the HTTP server I’m working against doesn’t handle HTTP posts using Connection: Keep-Alive properly. For whatever reason it decides that the third request in a Keep-Alive connection should be closed.
Broadly, the reason I bring this up is because I think it’s important for all web developers to have an in-depth understanding of what’s going on under the hood of HTTP. Knowing the advantages and disadvantages of things like the HTTP Keep-Alive header becomes invaluable whenever you have to drop down to manually sending and receiving HTTP.
More pointedly, it was interesting to find out a couple tidbits about how .NET handles HTTP connections. First, by default .NET is configured (via machine.config) to use whatever proxy settings you have for Internet Explorer. You can turn this off by modifying the:
configuration/system.net/defaultProxy/proxy
element. Second, also by default, machine.config only allows .NET applications to make 2 persistent connections to external resources. You can modify/view this as well:
configuration/system.net/connectionManagement
Finally, the HttpWebRequest and it’s parent WebRequest again, by default, are set to use Keep-Alive connections.
I am calling web services from .Net Forms client and experiencing random exceptions “The underlying connection was closed: An unexpected error occurred on a receive.”
The client code is just
Dim ws As New host.Service1()
Dim blob As Byte() = ws.GetImage(sName)
Do you know how can I modify KeepAlive value?
I am trying to do the same thing you have done in your code sample. What do you do to parse the return stream? Also, I am trying to get this accomplished in VB.NET so any pointers would be awesome.
thanks
please try this codes, i hope your problem solved.
codes for solving webservice issue..
private string RequestGet(String TheURL, String TheProxy)
{
Uri uri = new Uri(TheURL);
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(uri);
string page;
try
{
request.KeepAlive = false;
request.ProtocolVersion=HttpVersion.Version10;
request.Method = “GET”;
request.ContentType = “application/x-www-form-urlencoded”;
request.Proxy = System.Net.WebProxy.GetDefaultProxy();
//allow auto redirects from redirect headers
request.AllowAutoRedirect=true;
//maximum of 10 auto redirects
request.MaximumAutomaticRedirections=10;
//30 second timeout for request
request.Timeout=(int) new TimeSpan(0,0,60).TotalMilliseconds;
//give the crawler a name.
//request.UserAgent = “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)”;
request.UserAgent=”Mozilla/3.0 (compatible; My Browser/1.0)”;
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader readStream = new StreamReader (responseStream, Encoding.UTF8);
page = readStream.ReadToEnd();
}
catch (Exception ee)
{
page = “Fail message : “+ee.Message;
}
return page;
}
and the way to call
private void GetButton_Click(object sender, System.EventArgs e)
{
textBox1.Text = RequestGet(“http://localhost/post-get/get.asp?data=GET123456789″,””);
}
system using : (add system using in the first line of c# codes.)
using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using System.Data;
using System.Web;
using System.Net;
using System.IO;
using System.Text;
if you have difficulty to implement the codes, you may download directly from http://www.thepsn.co.nr choose source then get-post issue.
regards,
-xins
See you in Matrix.
I just want to say: thanks. I have problems with my app´s and this article help so much
to control the “Underlying connection” error´s.
I have web-site that is protected by Integrated Windows Authentication. I need keep-alive to be true so that I can keep the HTTPWebReq in a connection group so that every request will not need to authenticate.
hi Bob,
Turning on keep alives is pretty easy:
// construct the http web request
String url = “http://server/path”;
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
// turn on keep alive
req.KeepAlive = true;
// request the page…
More information about it on MSDN.
AJ
My problem is this happens intermittently on a webservice.
I read somewhere that the POST should not have any extra characters appended and the Browsers always put a CRLF after a pOST request.
Can a Webservice be configured as a GET request?
Excellent article. The information about connectionManagement was not found on any websites and won’t have known why my HttpWebRequest client component could only make 2 requests at one time. I’m glad I read this article.
Thanks.
You are my hero that worked great. Do you have any suggestions where a person could find a good article on the inner workings of HTTP?
Is the only way to set the HTTPWebRequest.KeepAlive Property on a server level? I am dealing with a vendor that needs to make this change but refuses too. The behavior that I am seeing is exactly what is described in the artical. If it could be set on the client or server side (machine.config/web.config)….
Thanks!
“I fired up TcpTrace and modified the KeepAlive property (setting it to false) of the HttpWebRequest object on a whim and voila! The application worked again”
Can anyone tell me when data is POSTED from client using POST method , how to receive data in the server