Expand description
Per-service concurrency limiting for gRPC servers.
Unlike [tower::limit::GlobalConcurrencyLimitLayer] applied around a whole
server, ServiceConcurrencyLimit bounds the in-flight requests of a
single gRPC service, so services sharing one listener cannot crowd each
other out of admission slots.
Tower’s per-service [tower::limit::ConcurrencyLimitLayer] cannot be used
here: its ConcurrencyLimit wrapper does not implement tonic’s
[NamedService], which Routes::add_service requires for routing, and
shedding through tower’s LoadShed surfaces as a BoxError, incompatible
with the router’s Error = Infallible bound — over-limit requests must be
answered in-band with a gRPC RESOURCE_EXHAUSTED response instead.
Structs§
- Service
Concurrency Limit - Bounds the number of concurrent in-flight requests to the wrapped gRPC service, independently of any other service registered on the same server.