Socket connect opened many times in same machine with same request

[wyaow]

My issue is that when my web browser navigates to these url:
http://localhost:44382/api/Todo/3
http://localhost:44382/api/Todo

With command "netstat -nao | findstr 44382", It shows only one socket connection port opened during the whole time. The same even with more different HTTP Request.
While Use RestClient(Code below), It seems that every HTTP Request Creates a different socket connection port althought it will be released within a few minute.

for (int i = 1; i <= 10; i++)
{
	var client = new RestClient("http://localhost:44382/api/Todo/3");
	var request = new RestRequest();
	equest.Method = Method.GET;
	request.RequestFormat = DataFormat.Json;
	
       var asyncHandle = client.GetAsync<TodoClass>(request, (resp, callback) =>
	{
		Console.WriteLine(resp.Content);	
		callback.Abort();
	});
}

Is there a best way to avoid create different socket port by use one same socket?
look forward to get your reply !

[alexeyzimarev]

Well, you are creating a new instance of the client for each call, why are you surprised?

[wyaow]

While the same result when creating client only once.

var client = new RestClient("http://localhost:44382/api/Todo");
var request = new RestRequest();
request.Method = Method.GET;
request.RequestFormat = DataFormat.Json;

for (int i = 1; i <= 5; i++)
{
    var response = client.Get(request);
    var responseJson = response.Content;
}

[DynamicOperationsFacilitator]

Same result for me, using .net core 2.2 on Windows 10.

Start local nginx instance:

docker run -it -p 8888:80 nginx

Start loop to download index.html

using System;
using RestSharp;
using System.Threading;

namespace RestSharpRecycle
{
  class Program
  {
    static void Main(string[] args)
    {
      var client = new RestClient();
      while(true)
      {
        Thread.Sleep(10);
        var resp = client.Get(new RestRequest("http://127.0.0.1:8888/index.html", Method.GET));
        if(!resp.IsSuccessful) { throw new Exception(); }
        if(resp.Content.Contains("just-to-make-sure-we-access-Content")) { throw new Exception(); }
      }
    }
  }
}

Result is 360+ connections to the nginx instance in TIME_WAIT

netstat -ano  | select-string 8888 | select-string TIME_WAIT | measure-object

Count    : 366

The scenario implemented using System.Net.Http.HttpClient yields significantly less connections in TIME_WAIT.

using System;
using System.Threading;
using System.Net.Http;

namespace RestSharpRecycle
{
  class Program
  {
    static void Main(string[] args)
    {
      var client = new HttpClient();
      while(true)
      {
        Thread.Sleep(10);
        var res = client.GetAsync("http://127.0.0.1:8888/index.html").GetAwaiter().GetResult();
        res.EnsureSuccessStatusCode();
        var content = res.Content.ReadAsStringAsync().GetAwaiter().GetResult();
      }
    }
  }
}

This is the entire output from netstat when running that code.

netstat -ano  | select-string 8888

  TCP    0.0.0.0:8888           0.0.0.0:0              LISTENING       7872
  TCP    127.0.0.1:8888         127.0.0.1:54306        TIME_WAIT       0
  TCP    127.0.0.1:8888         127.0.0.1:54311        TIME_WAIT       0
  TCP    127.0.0.1:8888         127.0.0.1:54318        ESTABLISHED     7872
  TCP    127.0.0.1:54318        127.0.0.1:8888         ESTABLISHED     29832
  TCP    [::1]:8888             [::]:0                 LISTENING       7872

[fgiele]

Check what is written on:
https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/

This issue might be related to what is mentioned in that blog. I ran into this issue as well where I have to massively query a single server, seriously considering moving to a native HttpClient implementation to see if that resolves the issue.

[dyardyGIT]

We are having similar where restsharp works for a while, then it starts failing.

Right now we are creating new restsharp client on each request.

Ideas on the solution?

"Error:RestSharp.RestResponseSystem.Net.WebException: An error occurred while sending the request. Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Http.HttpRequestException: An error occurred while sen "

[alexeyzimarev]

RestClient was made thread-safe quite a while ago and I would suggest not instantiating it every time.

[alexeyzimarev]

@wyaow this Todo API, is this some .NET WebAPI? I'd like to test it.

[dyardyGIT]

made thread-safe quite a while ago and I would suggest not instantiating it eve

At the current moment, i am using singleton pattern via asp.net core / dependency injection / addSingleton and so re-using the client and still ending up with connection issues.

[dyardyGIT]

Check out this blog post which describes the multiple issues that we are running up against https://www.nimaara.com/beware-of-the-net-httpclient/

[alexeyzimarev]

It was already mentioned in the old issue. It doesn't help really. HttpClient and WebRequest are different things.

[dyardyGIT]

issue. It doesn't help really. HttpClient

Does this mean their solution does not work also (if you have tried it?) At this point, i am looking for work arounds/other solutions to this issue.

[lixiandai]

our team has exactly same issue and one of our team member confirmed using httpClient and httpRequest is correctly resuing the open connections there so we have to write our own client libaray instead of using the RestSharp

[devenr23]

RestSharp relies on HttpWebRequest. In .NET Core, HttpWebRequest was implemented by creating a new HttpClient instance each time it is used. So, connections are not reused. It looks like a fix for it that handles some scenarios has been merged. I reproduce the socket problems using RestSharp on .NET Core 2.2 and 3.0. So, RestSharp probably needs to be updated to use HttpClient. Microsoft is recommending that HttpWebRequest be moved away from.

Recommendation to stop using HttpWebRequest
Reported issue with HttpWebRequest in Core
Merged fix for it

[rob-hendrickx-c4t]

Any progress on this? We currently suffer from socket exhaustion when code that uses RestSharp is under high load.

[monsieurberry]

Hi @rob-hendrickx-c4t

The deadly silence on resolving this issue meant I had to switch over to using HttpClient instead of RestSharp. See my post here on StackOverflow for the difference between the two...

https://stackoverflow.com/a/59013521/2527616

It wasn't too difficult a swap - but one I'd rather not have had to do... If this ever gets fixed I may return to using RestSharp, but its just too risky at the moment.

[alexeyzimarev]

@monsieurberry you realise that "deadly silence" is caused by the community not being involved in investigating this issue?

Nevertheless, I think the issue is caused by the latest HttpWebRequest implementation in .NET, which instantiates HttpClient for each request. I am not sure why Microsoft is doing that. They advise not to do it.

I am working on the new client that will use HttpClient but it takes time. Mainly, because I am not getting any help. Deadly silence, right?

[cesaralc]

Hi, Any update on this issue. I just ran into this as well?

[danilobbezerra]

I am one more with this problem, does anyone have any solution?

[alexeyzimarev]

There's no other solution than moving to HttpClient. It will be done in the next major version.
#1406

[alexeyzimarev]

I actually figured out that the issue is not in new instances of HttpClient, but HttpMessageHandler. Well, not me, I just read the docs and compare it with the HttpWebRequest code.

Interesting enough, the HttpWebRequest in the latest runtime caches the HttpClient instance but it only works if all the instances of HttpWebRequest use the same parameters.

So, I am quite surprised that the foreach loop that makes identical requests produces such a result. The issue is old though and I am looking at .NET 5 preview code, where it's done as I described.

[happysammy]

Hi, i just ran into the error message " Only one usage of each socket address (protocol/network address/port) is normally permitted" , so should i need to switch to using HttpClient to solve this problem?

[yilezhu]

Hello, friend, do you have any plans to fix this problem? I have seen this problem for a long time. Just after I tested it again, I found that the problem still exists!

[cesaralc]

I resolved this issue on my side a few months back and forgot to update the message. When you addjson file set reloadonchange = false this problem goes away. I hope this also helps someone else :).

[wyaow]

你看看他源码是不是new的时候没创建.  后期用的时候才真正创建链接。

…

---Original--- From: "依乐祝"<[email protected]&gt; Date: Sun, Jun 28, 2020 11:16 AM To: "restsharp/RestSharp"<[email protected]&gt;; Cc: "Mention"<[email protected]&gt;;"Ezreal Smith"<[email protected]&gt;; Subject: Re: [restsharp/RestSharp] Socket connect opened many times in same machine with same request (#1322) Hello, friend, do you have any plans to fix this problem？ — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[repo-ranger]

⚠️ This issue has been marked wontfix and will be closed in 3 days

[tonyqus]

@alexeyzimarev Do you need help on this issue?

[IanKemp]

@alexeyzimarev is this fixed in v107?

[alexeyzimarev]

It should be if you use the client instance correctly. If it's a singleton - it will use its own HttpClient. If you supply the HttpClient instance and use IHttpClientFactory for that - you can create RestClient instance for each request.

[21mac]

It should be if you use the client instance correctly. If it's a singleton - it will use its own HttpClient. If you supply the HttpClient instance and use IHttpClientFactory for that - you can create RestClient instance for each request.

unfortunately, it still does not work.
It does not matter if I use the SAME restclient or if I create a new instance for each request,
I still get the error:
"only one usage of each socket address (protocol/network address/port) is normally permitted"

The other side (an asp.net core rest webapi) does not receive the request, so it has to be a problem on the restsharp side.

I think it has to be a problem of port exhaustion and it happens sometimes after a few hundreds requests, sometimes multiple thousand requests are possible.

Why was this thread closed but the problem still occurs?
Does anyone have a solution for this?

Thanks

[rob-hendrickx-c4t]

@21mac I know it's not really a solution, but in our project we decided to migrate away from RestSharp in favor of Refit (https://github.com/reactiveui/refit), which is built on top of the HttpClient of .NET and can be combined with HttpClientFactory.

[alexeyzimarev]

@21mac I am not sure why using a single instance of RestClient would cause socket exhaustion. A single RestClient instance would use a single message handler instance, which suppose to behave well and close sockets for completed requests. It's interesting to see that people from Influx believe that HttpClient would magically resolve those issues. I might be not fully aware of what happens inside the default implementation of IHttpClientFactory, but last time I checked a couple of years ago, it was all about keeping a pool of message handlers, providing those on-demand, and disposing those that are no longer in use. I am not sure if it will provide more message handlers when there are too many calls executed at the same time.

To answer your question why this issue was closed, I think you experience a different one. Using a single instance of HttpClient within a single instance of RestClient should have solved this issue as it is described in many places like this one: https://www.c-sharpcorner.com/blogs/use-httpclient-in-the-right-way.

When you make lots of concurrent calls, you might experience the dynamic port range exhaustion. The issue can be fixed by amending the OS settings that control the number of available dynamic ports and/or decrease the TIME_WAIT. The first search result in Google gives me this: http://help.socketlabs.com/docs/how-to-fix-error-only-one-usage-of-each-socket-address-protocolnetwork-addressport-is-normally-permitted

@rob-hendrickx-c4t Nothing stopped you from using something like this:

public sealed class MyScopedCaller(IHttpClientFactory factory) : IDisposable {
     IRestClient _restClient = new RestClient(factory.CreateClient(), disposeHttpClient: true);

    public async Task<MyResult> MakeCall() {
         // do something 
    }

    public void Dispose() => _restClient.Dispose();
}

[alexeyzimarev]

Btw, there's an open issue #1791

[rob-hendrickx-c4t]

@alexeyzimarev The fact we faced this in 2019 (I commented on this issue back then)
, long before v107 of RestSharp that added HttpClient support was released, did

[21mac]

Thanks guys.
Yeah that's kinda weired, because it does not matter if I use version 106, 107 or 110, the problem still occurs.
At first, I created a new instance vor every HTTP requests but after finding this issue and another (still open) issue, I tried with a global accessible singleton that uses the SAME restClient.

No matter what I try, after some minutes, no request reaches the web api (iis / asp.net logging shows no received requests).

unfortunately, I can't change the dynamic port settings on the OS, since I develop a plugin for "NopCommerce", which will be given to our customers and it's not possible to access and reconfigure their servers as a system requirement for our plugin.

Why do I stick with restSharp? NopCommerce already uses and deploys RestSharp and recommends using it, for whatever reason they might bring up.

How can I test if it's REALLY this issue?
as I initially wrote: I think that it might be this issue because the error message "only one usage of each socket address (protocol/network address/port) is normally permitted" and the fact, that the request does not leave the sending server.
So the receiving server can't be responsible.

[alexeyzimarev]

@21mac can you try using the IHttpClientFactory as I described?

[alexeyzimarev]

Also, maybe SocketsHttpHandler would work better but it is only available for .NET 5+

[NCLnclNCL]

Also, maybe SocketsHttpHandler would work better but it is only available for .NET 5+

Limit max connection, right ??

[SubjectiveReality]

@21mac have you set useClientFactory: true as a parameter when creating your RestClient? That's all I needed to do (defaults to dispose, otherwise) False alarm, I still see massive port use.