Multi-tenancy = isolation + fairness: behind a shared FIFO queue, one tenant can take an arbitrary share

pattern
established
config

Scope. The fair-queueing result is about packets at a gateway with known sizes; carrying it to requests, jobs or queries needs a cost estimate that is only approximate, and fairness is defined per flow key, not per real-world tenant. Where tenants pay for different service levels, equal shares are the wrong target and weights or priority levels replace them.

Versions. flowcontrol.apiserver.k8s.io/v1 is the stable API since Kubernetes 1.29; before v1beta3 the shares field was named assuredConcurrencyShares.

Primary sources: Alan Demers, Srinivasan Keshav, Scott Shenker — Analysis and Simulation of a Fair Queueing Algorithm (Internetworking: Research and Experience, 1990; SIGCOMM 1989), AWS Well-Architected SaaS Lens — Noisy Neighbor

References: AWS Whitepaper — SaaS Tenant Isolation Strategies: Isolation: Security or noisy neighbor?, Kubernetes Documentation — API Priority and Fairness, Prometheus Documentation — Configuration: scrape_config sample_limit

Other claims on this card, typed separately:

- model
- established
Under first-come-first-served queueing a single source sending fast enough can capture an arbitrarily high fraction of the outgoing capacity; fair queueing gives fair allocation, lower delay for sources using less than their share, and protection from ill-behaved sources.
Primary sources: Alan Demers, Srinivasan Keshav, Scott Shenker — Analysis and Simulation of a Fair Queueing Algorithm (Internetworking: Research and Experience, 1990; SIGCOMM 1989)

A shared platform owes its tenants two different things. Isolation: one tenant cannot see or change another’s data. Fairness: one tenant cannot degrade another’s service. They need different mechanisms, and the second is the one usually missing: under first-come-first-served, a single source that sends fast enough takes an arbitrarily large share of the capacity. Fair queueing, with a separate queue per source served in turn, removes exactly that property.

Layers, not one knob. A quota caps a tenant’s total consumption, which bounds the worst case but says nothing about who goes first under contention. Per-tenant admission control rejects excess early with a 429 instead of letting it occupy a shared queue (23). Fair queueing inside the service divides what is actually available among the tenants that are actually busy, so idle capacity is not wasted. Shuffle sharding bounds how many others a heavy tenant can collide with, and cells turn that bound into a wall (27). Priority is a separate axis: fairness among equals does not decide whether the platform’s own leader election goes before a tenant’s batch job.

config

Kubernetes · flowcontrol v1

# The API server is itself a multi-tenant service: a priority level caps a class
# of work, and fair queuing inside it keeps one namespace from starving the rest.
apiVersion: flowcontrol.apiserver.k8s.io/v1
kind: PriorityLevelConfiguration
metadata:
  name: tenant-workloads
spec:
  type: Limited
  limited:
    nominalConcurrencyShares: 20    # a share of the server's seats, not an absolute count
    lendablePercent: 50             # idle seats may serve other levels
    limitResponse:
      type: Queue                   # Reject would answer 429 immediately
      queuing:
        queues: 128
        handSize: 8                 # shuffle sharding: each flow is dealt 8 of the 128 queues
        queueLengthLimit: 50        # one flow can queue at most handSize * queueLengthLimit requests
---
apiVersion: flowcontrol.apiserver.k8s.io/v1
kind: FlowSchema
metadata:
  name: tenant-workloads
spec:
  priorityLevelConfiguration:
    name: tenant-workloads
  matchingPrecedence: 5000          # lowest value is tested first; the first match wins
  distinguisherMethod:
    type: ByNamespace               # flow = target namespace = tenant
  rules:
  - subjects:
    - kind: Group
      group:
        name: system:serviceaccounts
    resourceRules:                  # namespaced requests only; cluster-scoped ones fall through
    - verbs: ["*"]
      apiGroups: ["*"]
      resources: ["*"]
      namespaces: ["*"]

Fairness is only as good as attribution. A flow is whatever key the scheduler sees; a tenant that splits into many keys (users, namespaces, API keys) gets many shares. The tenant key must therefore come from authenticated identity (102) and travel with the request from the edge inward — a component that cannot attribute work cannot be fair about it. The second hard part is cost: fair queueing was designed for packets whose size is known on arrival, while a request’s cost is known only afterwards, which is why the Kubernetes API server estimates how many concurrency seats a list call will occupy from the number of objects it may return.

Tenants also load the platform’s own planes. Tenant-controlled dimensions — metric labels, topics, partitions (92), per-tenant objects in the control plane — grow cardinality and cost, not request rate, and can take down the observability or control plane while the data plane looks healthy. Limits there are often cliffs: in Prometheus, a scrape over its sample_limit fails as a whole. Decision rule: for every shared resource — pool, queue, partition, cache, metrics backend, control-plane API — name the tenant key, the limit, what happens at the limit, and whose SLO absorbs the burst; track cost per tenant as an SLI (78).