Rate Limiting with HTTP

[jakerumbles]

I have followed the jsonrpsee guide for adding rate limiting middleware to my server. When I make an API call to it from postman they are all succeeding despite allowing only 1 call every 15 seconds. I can see from my logs that it is not storing the RateLimit state between calls. But when I try the example server they are using a websocket cliet (WsClientBuilder). This successfully returns the rate limit error. I assume this is because jsonrpsee is maintaining the RateLimit state under the hood because the WS connection is still alive between individual calls vs HTTP requests are individual connections.

So my question is how is one supposed to rate limit a specific IP for JSON-RPC requests over HTTP?

[niklasad1]

Hey,

It really depends how you instantiate the rate middleware and you could decide whether to use enable it globally on the server or per connection.

	let global_cnt = Arc::new(AtomicUsize::new(0));

	let rpc_middleware = RpcServiceBuilder::new()
		// This state is created per connection.
		.layer_fn(|service| CallsPerConn { service, count: Default::default() })
		// This state is shared by all connections.
		.layer_fn(move |service| GlobalCalls { service, count: global_cnt.clone() });

       // start server....

So you could enable two rate limiting middlewares:

1. one global that runs for each new connection (both HTTP and WS)
2. per connection that is useful for WebSocket connections

However, if you want to rate-limit/blacklist certain peer/ip addr then you need to something more complicated than you can use low-level API and utilize jsonrpsee as a service.

[jakerumbles]

Ahh ok I think you hit on my key issue in your 2nd point "per connection that is useful for WebSocket connections". We are using JSON RPC over HTTP. I tried the examples/rpc_middleware_rate_limiting.rs example, but with HTTP connections that strategy effectively does nothing because it just creates a brand new RateLimit object every time an api endpoint gets hit, so it never reuses the object state to check if it should rate limit, so all requests are processed always.

I found something called Tower Governor that I am trying now (https://github.com/benwis/tower-governor), but it doesn't seem that it implements a trait so if I have the "add GovernorLayer" uncommented then the server start() fn is no longer callable.
use tower_governor::{governor::GovernorConfigBuilder, GovernorLayer};

    pub async fn start(&self) -> Result<RpcServerHandle> {
        // Allow bursts with up to five requests per IP address
        // and replenishes one element every two seconds
        // We Box it because Axum 0.6 requires all Layers to be Clone
        // and thus we need a static reference to it
        let governor_conf = Arc::new(
            GovernorConfigBuilder::default()
                .per_second(2)
                .burst_size(5)
                .finish()
                .unwrap(),
        );

        let governor_limiter = governor_conf.limiter().clone();
        let interval = Duration::from_secs(60);
        // a separate background task to clean up
        std::thread::spawn(move || loop {
            std::thread::sleep(interval);
            tracing::info!("rate limiting storage size: {}", governor_limiter.len());
            governor_limiter.retain_recent();
        });

        // Add rate limiter middleware
        // let rpc_middleware = RpcServiceBuilder::new().layer_fn(
        //     |service: jsonrpsee::server::middleware::rpc::RpcService| {
        //         tracing::info!("MIDDLEWARE: RATE LIMITING");
        //         RateLimit::new(
        //             service,
        //             Rate {
        //                 num: 1,
        //                 period: Duration::from_secs(15),
        //             },
        //         )
        //     },
        // );

        let http_middleware = ServiceBuilder::new().rate_limit(3, Duration::from_secs(60));
        // .layer(GovernorLayer {
        //     config: governor_conf,
        // });

        // let (stop_handle, handle) = stop_channel();

        // let new_server = ServerBuilder::new()
        //     .set_http_middleware(http_middleware)
        //     // .to_service_builder()
        //     // .build(self.rpc_module.clone(), stop_handle);
        //     .build(self.settings.http_listen_addr())
        //     .await?;

        let server = Server::builder()
            // .set_rpc_middleware(middleware)
            .set_http_middleware(http_middleware)
            .build(self.settings.http_listen_addr())
            .await?;

        let addr = server.local_addr()?;
        tracing::info!("RPC server started at: {}", addr);

        let handle = server.start(self.rpc_module.clone());

        Ok(RpcServerHandle {
            listen_addr: addr,
            handle,
        })
    }

I was able to get the built in Towers rate_limit layer to work, but I believe this just limits calls globally. I need to have rate limiting per IP over HTTP essentially. Trying to mix and match towers and jsonrpsee here has it's challenges. Hoping there is a solution. Tower Governor seems to be the perfect fit for my use case, just have to figure out this trait issue.

[niklasad1]

I was able to get the built in Towers rate_limit layer to work, but I believe this just limits calls globally. I need to have rate limiting per IP over HTTP essentially. Trying to mix and match towers and jsonrpsee here has it's challenges. Hoping there is a solution. Tower Governor seems to be the perfect fit for my use case, just have to figure out this trait issue.

Sorry, the jsonrpsee rpc middleware is not "tower compatible" it's a separate trait and currently we have no way to use tower layers https://docs.rs/jsonrpsee-server/latest/src/jsonrpsee_server/middleware/rpc/mod.rs.html#40-l#50.

If you really just want to do http rate limiting you shouldn't need the RpcMiddleware at all and just the http middleware, which version of jsonrpsee and tower-governor are you using? Just double check that both are using the same version of tower and http....

[jakerumbles]

Yeah I saw that so I already switched over to using http middleware yesterday.

Versions:

tower_governor = "0.4.2"
tower = "0.4.13"

[jakerumbles]

I left another post on the tower-governor repo: benwis/tower-governor#33

[niklasad1]

Might be jsonrpsee's trait bounds that causes that, most likely and not tower-governor itself