Big mistake in using HttpClient

Big mistake in using HttpClient

If you also program in C #, you must have used the HttpClient class many times.
The HttpClient class is a base class for sending requests and receiving responses from a source known by the URI, or in simpler language we use this class for sending requests and receiving responses by the URI.

Like all of you, according to the documentation provided by Microsoft, I have been using this class for years:

using(var client = new HttpClient())
{
    //Codes
}

In one of the projects I found the software to be unusually unstable! With a very small change, the software efficiency was much better and the problem was completely solved!
The phrase using is one of the beauties of C # and using it we will no longer worry about wasting resources, and when the using block is complete, the dispose method will be called and the resources will be released automatically (Dispose).
Using is very common and we use it in almost everything, from database connections to Stream Writers.
In fact, any object that uses resources temporarily and needs to release resources after it is done must use the IDisposable interface.
According to Microsoft documents, when we use an IDisposable object, we have to prototype it inside the using block.
Naturally, when we use the HttpClient class, we need to close the connection after use and release the resources, but HttpClient is a little different, instead of prototyping HttpClient for each request, we need to share a copy of it throughout the software.
Let's follow the example with an example:
Below is a short program to use the HttpClient class:

using System;
using System.Net.Http;

namespace ConsoleApplication
{
    class Program
    {
        public static void Main(string[] args) 
        {
            Console.WriteLine("Starting connections");
            for(int i = 0; i<10; i++)
            {
                using(var client = new HttpClient())
                {
                    var result = client.GetAsync("https://microsoft.com").Result;
                    Console.WriteLine(result.StatusCode);
                }
            }
            Console.WriteLine("Connections done");
        }

        
    }
}

As you can see, we have written a very simple code snippet that sends a Get request to the Microsoft site ten times using the block using the HttpClient class.
In the image below, you can see that the program has run well and ten submissions have been successfully answered.

So far so good, but let's use the Netstat tool to check open sockets before and after running the program:
Before running the program, you can see the open sockets in my system in the image:

 

A noteworthy point is the list of open sockets after running the program (at least 5 seconds after the end of the program):

Yes, you can see in the picture that even after the program is finished, 10 sockets are still open and not closed!
If you note that the status of these connections is in TIME_WAIT mode, which means that this connection is closed on one side (our side) but is still open on the other side and is waiting to receive new packages (because it may The network is delayed and the packages reach the server later).
In the image below you can see a diagram of the TCP / IP status available at www4.cs.fau.de.

Windows holds the connection in TIME_WAIT mode for 240 seconds (set at [HKEY_LOCAL_MACHINE \ SYSTEM \ CurrentControlSet \ Services \ Tcpip \ Parameters \ TcpTimedWaitDelay]) If you encounter this problem the following error should be known:

Unable to connect to the remote server
System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted.

A search for this error on Google will tell you to reduce the connection timeout !!!! This is a big mistake! This is because reducing the timeout of connections will cause you other problems in HttpClient and other parts.
We need to know exactly what the problem is and solve the root of the problem, not just cover the problem by changing the server variables!

-solution

Once we build and use an instance of HttpClient, we can prevent the sockets from being lost again.
We make changes in the above code and write as follows:

using System;
using System.Net.Http;

namespace ConsoleApplication
{
    class Program
    {
        private static HttpClient Client = new HttpClient();
        public static void Main(string[] args) 
        {
            Console.WriteLine("Starting connections");
            for(int i = 0; i<10; i++)
            {
                var result = Client.GetAsync("https://microsoft.com").Result;
                Console.WriteLine(result.StatusCode);
            }
            Console.WriteLine("Connections done");
        }

        
    }
}

As you can see, the using block is not used, and instead we created an instance of HttpClient and then use the same instance each time the loop is repeated.
Now if we run the program we will see the work run the same way as before and we will get the same result (a little faster in execution due to the use of an HttpClient instance).
Now if we run the program again and use the Netstat tool we will see that a socket is in use.

Note that all of this material was an example that we wrote just for testing. In the project I encountered this problem, about 4,500 sockets were used on average, which at the peak consumption of this number reached 5,000 to 5,500, which caused severe instability of the program. Been.
With these small changes in the use of HttpClient the result was amazing, the sockets used from about 4000 were reduced to 400, in most cases about 100 sockets are in use!

Conclusion:

Using the using block is very useful and efficient, but in connection with using the HttpClient class in the using block, this is very wrong and destructive.
If you have any need to load using HttpClient, keep in mind:
1- Introduce the HttpClient instance statically.
2- Do not use the using block!

If this article is useful to you, share it so that your friends can use it as well.

Comments (1)

نظرسنجی
میزان رضایت شما از امکانات و مقالات سایت
مجوزها
E Namad به پرداخت