Package org.microproxy.extras

Class ConcurrencyLimiter

java.lang.Object
org.microproxy.extras.ConcurrencyLimiter
All Implemented Interfaces:
HttpFiltersSource
public final class ConcurrencyLimiter extends Object implements HttpFiltersSource
Limits how many exchanges run at once per key: per client address by default, or per user, per target host, or anything else a key function derives from the request and its client connection. Requests over the limit wait in a bounded queue, or are answered with 429 Too Many Requests. 

 ConcurrencyLimiter limiter = ConcurrencyLimiter.builder()
         .key(ConcurrencyLimiter.byUser())        // default: the client's IP address
         .permits(8)                               // per key
         .permits(user -> user.equals("batch") ? 32 : null)   // null: the default
         .queue(16, Duration.ofSeconds(2))         // up to 16 wait up to 2 s each
         .build();
 MicroProxy.bootstrap().plusFiltersSource(limiter).start();

How a permit is held:

  • It is taken in HttpFilters.clientToProxyRequest(org.microproxy.http.HttpObject) for the request head, which runs after proxy authentication, so byUser() sees the authenticated user and requests refused with 407 are never counted. Place the limiter first among the filters (HttpFiltersChain.of(limiter, ...)) so that later filters do no work for refused requests; an earlier filter that answers a request itself means the limiter never sees it.
  • It is released exactly once, in HttpFilters.exchangeEnded(boolean): after the response has been written, or when the exchange was abandoned (client disconnected, server failed, a filter answered or aborted, the proxy stopped).
  • A CONNECT that becomes a byte tunnel, and a request upgraded with 101 (WebSocket), hold their permit until the tunnel closes. With countTunnels(false) CONNECTs are not counted at all and an upgraded request gives its permit back once the 101 has been sent. An intercepted (MITM) CONNECT holds its permit only until interception starts; the requests inside the session are counted one by one.
  • As a safety net, a permit held for longer than ConcurrencyLimiter.Builder.permitTimeout (10 minutes by default) is reclaimed and logged at WARNING. Tunnels are exempt once established: they end when either side closes or goes idle.

A key's bookkeeping is dropped as soon as none of its permits are in use and nobody waits, so the number of keys tracked is bounded by the number of exchanges in progress. Per-key rejection counts in snapshot() therefore cover the key's current busy period; the totals cover the limiter's lifetime.

The limiter only reads request heads (SelectiveFilters), so bodies keep the proxy's fast path. Waiting blocks only the client connection's virtual thread.

  • Method Details

    • builder

      public static ConcurrencyLimiter.Builder builder()
      Starts a builder with the default settings.
      Returns:
      a new builder with default settings

    • byClientIp

      public static BiFunction<HttpRequest,FlowContext,String> byClientIp()
      Keys requests by the client's IP address (the default).
      Returns:
      a function extracting the client IP key

    • byUser

      public static BiFunction<HttpRequest,FlowContext,String> byUser()
      Keys requests by the authenticated user (flow.getClientDetails().getUserName()), or by the client's IP address for clients that did not authenticate.
      Returns:
      a function extracting the authenticated-user or client-IP key

    • byTargetHost

      public static BiFunction<HttpRequest,FlowContext,String> byTargetHost()
      Keys requests by the host they go to, lower-cased and without the port: the CONNECT target, the absolute URI's host, or the Host header (inside intercepted sessions).
      Returns:
      a function extracting the lower-case destination host key

    • permits

      public int permits()
      Returns the permits per key unless ConcurrencyLimiter.Builder.permits(Function) says otherwise.
      Returns:
      the permits per key unless ConcurrencyLimiter.Builder.permits(Function) says otherwise

    • shadow

      public boolean shadow()
      Returns whether requests over the limit are only counted and reported, never refused.
      Returns:
      whether requests over the limit are only counted and reported, never refused

    • countTunnels

      public boolean countTunnels()
      Returns whether CONNECT tunnels hold a permit (see the class documentation).
      Returns:
      whether CONNECT tunnels hold a permit (see the class documentation)

    • filterRequest

      public HttpFilters filterRequest(HttpRequest originalRequest, FlowContext flowContext)
      Description copied from interface: HttpFiltersSource
      Returns the filters for a request, or null to proxy it unfiltered.
      Specified by:
      filterRequest in interface HttpFiltersSource
      Parameters:
      originalRequest - a copy of the request as received from the client
      flowContext - the client connection the request arrived on
      Returns:
      the per-request filters, or null to leave the request unfiltered

    • snapshot

      public ConcurrencyLimiter.Snapshot snapshot()
      The current state, per key and in total. Also reclaims expired permits.
      Returns:
      the current state, per key and in total

    • registerStats

      public void registerStats(StatsRegistry registry, String prefix)
      Registers the limiter's state with registry as <prefix>.in_use and <prefix>.waiting (gauges), and <prefix>.acquired, <prefix>.rq_blocked (requests over the limit, refused or let through in shadow mode) and <prefix>.reclaimed (counters), read when the metrics are shown. A proxy with metrics registers the limiters among its filters itself, as concurrency_limit.
      Parameters:
      registry - the registry to add the metrics to
      prefix - the names' prefix, such as concurrency_limit