Set a default limit to the connections undici can open

[WilfredAlmeida]

At Triton One, we have observed our customers face socket/connection issues. More information here

These issues have been there for more than 2 years.

tl;dr of the issues is that NodeJS tries to open infinite connections with the server and hits a limit and existing connections break and new connection attempts time out. In our tests, NodeJS opened over 16,000 connections. Opening new connection is unnecessary since existing ones can be resued. This is not the case with the Bun runtime. It opens less than 400 connections and doesn't error out at all.

With the current versions of web3.js (latest 1.91.8 as of this writing) the issues are frequent. We've also tested with the new web3.js 2.0.0-preview.3 and the error rate is less, but it still does exist.

The fix for these issues, as mentioned in the doc shared above, is to limit the number of connections NodeJS can open. This can be configured with the http.Agent configurations.

We propose that Web3JS should have a default configurable connection limit like the following:

// npm install undici

import { setGlobalDispatcher, Agent } from "undici";

setGlobalDispatcher(
  new Agent({
    connections: 1,
  })
);

From the http.Agent docs

maxSockets Maximum number of sockets to allow per host. If the same host opens multiple concurrent connections, each request will use new socket until the maxSockets value is reached. If the host attempts to open more connections than maxSockets, the additional requests will enter into a pending request queue, and will enter active connection state when an existing connection terminates. This makes sure there are at most maxSockets active connections at any point in time, from a given host. Default: Infinity.

Setting a default limit will allow connection reuse instead of a new connection being set for every request.

[NotoriousPyro]

Very nice you fixed all my issues I was having when subscribing to a load of accounts on program startup...

I use this

export const connection = new Connection(
    config.rpc.httpEndpoint,
    {
        httpAgent: new Agent({
            keepAlive: config.rpc.keepAlive,
            keepAliveMsecs: 60000,
            maxSockets: 256,
        }),
        ...config.rpc,
        fetch: RetryFetcher,
    },
);

[steveluscher]

I wrote benchmarks to find this number!

git clone https://github.com/solana-labs/solana-web3.js
cd solana-web3.js/packages/fetch-impl/
pnpm i
pnpm benchmark https://some.rpc-server-with-no-rate-limit.com

> @solana/[email protected] benchmark /home/sol/src/solana-web3.js-git/packages/fetch-impl
> ./src/__benchmarks__/run.ts "https://some.rpc-server-with-no-rate-limit.com"

(node:2243087) [UNDICI-H2] Warning: H2 support is experimental, expect them to change at any time.
(Use `node --trace-warnings ...` to show where the warning was created)
┌─────────┬───────────────────────────────────┬─────────┬────────────────────┬───────────┬─────────┐
│ (index) │ Task Name                         │ ops/sec │ Average Time (ns)  │ Margin    │ Samples │
├─────────┼───────────────────────────────────┼─────────┼────────────────────┼───────────┼─────────┤
│ 0       │ 'no dispatcher'                   │ '0'     │     1_902_934_963  │ '±37.13%' │ 10      │
│ 1       │ 'unlimited connections http/2'    │ '0'     │     4_605_433_659  │ '±1.58%'  │ 10      │
│ 2       │ 'unlimited connections'           │ '0'     │     4_043_684_846  │ '±4.88%'  │ 10      │
│ 3       │ '16 connections'                  │ '0'     │     6_892_149_604  │ '±41.75%' │ 10      │
│ 4       │ '16 connections, http/2'          │ '2'     │       439_921_781  │ '±21.03%' │ 10      │
│ 5       │ '16 connections, pipeline 2 wide' │ '0'     │     2_944_779_337  │ '±17.62%' │ 10      │
│ 6       │ '32 connections'                  │ '0'     │     4_135_789_213  │ '±51.36%' │ 10      │
│ 7       │ '32 connections, http/2'          │ '1'     │       681_074_594  │ '±39.77%' │ 10      │
│ 8       │ '32 connections, pipeline 2 wide' │ '0'     │     3_762_177_439  │ '±38.15%' │ 10      │
│ 9       │ '64 connections'                  │ '0'     │     2_845_502_630  │ '±27.39%' │ 10      │
│ 10      │ '64 connections, http/2'          │ '1'     │       784_407_918  │ '±32.93%' │ 10      │
│ 11      │ '64 connections, pipeline 2 wide' │ '0'     │     2_871_394_050  │ '±17.78%' │ 10      │
└─────────┴───────────────────────────────────┴─────────┴────────────────────┴───────────┴─────────┘

So at least in my testing:

1. I doubt that 1 is the right number, but unlimited doesn't seem great either.
2. No dispatcher is better than any non-HTTP2 manual config I could come up with.
3. Pipelining has nice effects when the connection count is low, but comes with the usual drawbacks of pipelining.
4. HTTP/2 slaps.

@WilfredAlmeida, can you download that benchmark suite, add in some tests (connections down to 1, above 64, and add in some wider pipelines) and run them against Triton servers? I'd do it but I'm pretty sure I've already been blocklisted on Mainnet.

[steveluscher]

Finally got a clean test run for 1 and 128 connections.

┌─────────┬────────────────────────────────────┬─────────┬────────────────────┬───────────┬─────────┐
│ (index) │ Task Name                          │ ops/sec │ Average Time (ns)  │ Margin    │ Samples │
├─────────┼────────────────────────────────────┼─────────┼────────────────────┼───────────┼─────────┤
│ 0       │ '1 connections'                    │ '0'     │   294_520_572_463  │ '±0.05%'  │ 10      │
│ 1       │ '1 connections, pipeline 2 wide'   │ '0'     │   294_785_419_969  │ '±0.16%'  │ 10      │
│ 2       │ '1 connections, pipeline 8 wide'   │ '0'     │   171_703_417_929  │ '±38.19%' │ 10      │
│ 3       │ '128 connections'                  │ '0'     │     3_147_103_325  │ '±2.11%'  │ 10      │
│ 4       │ '128 connections, pipeline 2 wide' │ '0'     │     3_088_783_513  │ '±1.10%'  │ 10      │
│ 5       │ '128 connections, pipeline 8 wide' │ '0'     │     3_093_144_608  │ '±0.99%'  │ 10      │
└─────────┴────────────────────────────────────┴─────────┴────────────────────┴───────────┴─────────┘

1. Setting the default to 1 connection: ngmi.
2. I'm generally uncomfortable setting any default at all. No dispatcher config at all performs well, and different RPC are all likely to need different tuning parameters. I think it will be more serviceable for RPC providers themselves to publish their own undici dispatcher configs and let people plug them in to web3.js. cc/ @0xIchigo @johnpmitsch

[WilfredAlmeida]

Sounds good to me

It'd be better if we could have it mentioned in the web3.js docs something like "If you're facing socket issues, based on your RPC provider, check this out"

We already have the doc published here