A while back I posted about a problem I encountered while trying to consume web services (and other goodness). The problem was related to the way that MS uses Win32 API's under the covers, specifically the WinInet API. I called it the Noah's Ark effect because of how I could spot my requests bunching up and going out two-by-two on the TCP/IP stack.

Well, it turns out that there's also a way within the .NET Framework to overcome the issue imposed by Microsoft's adoption of RFC 2616 (which  mandates only two open HTTP 1.1. endpoints to any given server). The quickest/easiest way to overcome it is to handle it in your web/app.config file, by playing around with a few settings in the system.net config handler section:

<system.net>
	<connectionManagement>
		<add address="*" maxconnection="2" />
		<add address="services.angrypets.com" maxconnection="12" />
	</connectionManagement>
</system.net>     

The cool thing is that, as you can see above, you can specify the number of concurrent connections you want to allow PER ADDRESS.

(Watch out though, the MSDN Documentation is wrong... well, actually the EXAMPLE is wrong, the documentation is right -- use address, not name as the key for your endpoints.)

The other cool thing, based on how the framework works, is that this config info isn't, of course, doing anything other than being passed into a class somewhere -- telling it what to do. The class in question is the System.Net.ServicePointManager class -- which exposes all sorts of cool static methods to allow you lots of control over your endpoints. I frankly think much of this functionality is overkill in all but the most advanced of cases -- but it's cool that the framework has everything available if needed. Furthermore, as opposed to trying to use this class, and it's associated ServicePoint class (which also has spiffy static members), to control connectivity, there's another option: use the functionality exposed by these classes for instrumentation.

I'm currently working on a project where scalability to a webservice will be a major concern. Because the webservice is wrapping logic/calls to a financial system (for credit card applications) some connections to the webservice may take up to 30 seconds to complete (and 'smaller' connections will take 5-10 seconds). That much lag when called from servers (acting as the webservice clients) that handle oodles of hits/second means that some threads are going to be tied up. Each tied up thread is a major concern during peak load, a precious resource to be sure. Obviously setting the web.config to allow more than two connections out will be a life saver. But, using System.Net.ServicePoint.CurrentConnections and .ConnectionLimit will allow the webservers/applications to keep an eye on how many connections/threads are tied up with requests. If the amount of requests becomes too great, internal logic based on the provided instrumentation can bypass the logic around contacting the webservice -- not an ideal situation, but it will beat cratering the webserver with outbound I/O requests. This same info will also be helpful if exposed externally (for example to content switches (like F5's BigIPs) that can poll specialized pages that report on how many 'worker' threads are tied up getting credit card application details -- allowing the switches to route client requests to less loaded servers if they can tell that an incomming request will trespace into the realm of credit card request/application processing. In other words, the possibilities are there... just from playing around with the ServicePoint/ServicePointManager classes provided by the Framework.