limit-conn

Limits concurrent connections per key.

Category TrafficAdd it to Route, Service, Consumer, Consumer group, Global rule, Plugin configEngine plugin limit-connPriority 1003

What it does

limit-conn caps how many requests from one key are open at once on a route, sent and not yet answered. Under One ceiling for everything here, Conn is that ceiling, Burst is how many more wait instead of being refused, and the key tells callers apart, for example remote_addr.

A waiting request is held for Default connection delay and then let through. A request beyond the burst is answered by the gateway itself with Rejected code, 503 unless you set another, as How G8 handles a request describes.

When to use it

• A backend that serves a fixed number of requests at a time, where more in flight from one client would hold every worker.
• Slow paths, a report export for example, that one client could fill by opening many requests at once.
• One ceiling for each tenant and another for each client address, under A ceiling for each key, with keys like tenant-$http_x_tenant and $remote_addr.
• A ceiling that several gateway nodes count together, through one Redis server or a Redis cluster.

Not for a quota counted per time window, for example 100 requests a minute — see limit-count (Limits the number of requests in a time window, per key) or a steady rate per second with a burst that waits its turn — see limit-req (Limits the request rate with a leaky bucket).

Set it up

Allow 20 open connections for each client address and let 5 more wait

Under Basics choose One ceiling for everything here, set Conn to 20, set Burst to 5, set Key to remote_addr and set Default connection delay to 0.5 seconds. Under Behavior set Rejected code to 429.

limit-conn · as the console saves it

{
  "conn": 20,
  "burst": 5,
  "default_conn_delay": 0.5,
  "key": "remote_addr",
  "rejected_code": 429
}

More scenarios

Share the connection count across gateway nodes through a Redis cluster

Under Basics choose One ceiling for everything here, set Conn to 100, set Burst to 20, set Key to remote_addr and set Default connection delay to 0.5 seconds. Under Behavior set Policy to redis-cluster, set Redis cluster name to gateway-limits, fill Redis cluster nodes as in the JSON below and set Redis password to <secret>.

limit-conn · as the console saves it

{
  "conn": 100,
  "burst": 20,
  "default_conn_delay": 0.5,
  "key": "remote_addr",
  "policy": "redis-cluster",
  "redis_cluster_name": "gateway-limits",
  "redis_cluster_nodes": [
    "redis-1.internal:6379",
    "redis-2.internal:6379",
    "redis-3.internal:6379"
  ],
  "redis_password": "<secret>"
}

Share the connection count across gateway nodes through one Redis server

Under Basics choose One ceiling for everything here, set Conn to 100, set Burst to 20, set Key to remote_addr and set Default connection delay to 0.5 seconds. Under Behavior set Policy to redis, set Redis host to redis.internal, set Redis password to <secret> and leave Redis port at 6379.

limit-conn · as the console saves it

{
  "conn": 100,
  "burst": 20,
  "default_conn_delay": 0.5,
  "key": "remote_addr",
  "policy": "redis",
  "redis_host": "redis.internal",
  "redis_port": 6379,
  "redis_password": "<secret>"
}

Give each tenant its own connection ceiling and each client address another

The tenant header has to be set by a trusted layer in front and stripped from client requests; a client that sends no tenant header is counted under the per-address ceiling alone.

Under Basics choose A ceiling for each key, fill Rules as in the JSON below and set Default connection delay to 0.5 seconds.

limit-conn · as the console saves it

{
  "default_conn_delay": 0.5,
  "rules": [
    {
      "conn": 50,
      "burst": 10,
      "key": "tenant-$http_x_tenant"
    },
    {
      "conn": 20,
      "burst": 0,
      "key": "$remote_addr"
    }
  ]
}

Things to know

• Rejected code — Set to 429, with a ceiling of 20 and 5 more waiting, 40 requests sent at once to /delay/1 are answered partly 200 and partly 429.
• Rejected code — Unset, a request beyond the burst is answered 503, the answer of a service that is unavailable.
• Conn — A reference with text around it, for example abc${http_x_conn}, is refused when you save.
• Conn — A ceiling of 0 is refused when you save.
• Key — A variable the request does not carry, or a name with a dollar sign under Variable, is saved without a word, and every request is then counted by client address. So is a combination in which no $name resolves.
• Rules — Rules cannot be set beside Conn, Burst and Key; the save is refused.
• Rules — An empty list of rules is refused when you save.
• Key — A rule's key holds at least one $variable, $remote_addr and not remote_addr. A key that names none is refused when you save.
• Allow degradation — When the limit cannot be counted, because the store is unreachable or a value resolves to nothing, off answers 500 and on lets every request through unlimited.
• AdviceLet a trusted layer in front of the gateway set a tenant header like X-Tenant and strip it from client requests, so that a client cannot choose the ceiling it is counted under.
• AdviceBehind a load balancer, add real-ip to the same route, so that each client is held to the ceiling by its own address and not by the load balancer's.

Settings

The fields as the console shows them, section by section. A field under “When … is …” is there only while that value is set.

Basics

Choose one:

• One ceiling for everything hereA ceiling, a burst and a key held over every connection this object serves.
• A ceiling for each keyA list of rules, each holding a ceiling and a burst over the key it names.

One ceiling for everything here

Connconninteger or stringRequired

How many requests one key may have in flight at once, such as 20. A $variable or ${variable} is read on every request, and a request without it fails with 500. $env://NAME is read once when the configuration loads.

integer over 0 · a whole number above zero, a $variable, or a $secret:// or $env:// reference

pattern

^([0-9]*[1-9][0-9]*|[$](env|secret)://.+|[$][{][^}]*[}]|[$][A-Za-z_][A-Za-z0-9_.]*)$

Burstburstinteger or stringRequired

How many connections beyond Conn wait instead of being refused; 0 refuses every one beyond Conn.

integer at least 0 ·

pattern

^([0-9]+|[$](env|secret)://.+|[$][{][^}]*[}]|[$][A-Za-z_][A-Za-z0-9_.]*)$

KeykeystringRequired

The variable that tells callers apart, such as remote_addr; with Key type var combination, a template such as $remote_addr $server_port. Not read when Rules carries entries.

example remote_addr

A ceiling for each key

Rulesrulesarray of objectsRequired

One ceiling per entry, each with its own Conn, Burst and key template, such as $http_x_api_key. Rules cannot be stored together with Conn, Burst and Key: the engine refuses the pair. Key type is ignored for rules.

at least 1 item

Each item:

Burstrules[].burstinteger or stringRequired

How many connections beyond Conn this entry lets wait instead of refusing. Zero is a value.

integer at least 0 ·

pattern

^([0-9]+|[$](env|secret)://.+|[$][{][^}]*[}]|[$][A-Za-z_][A-Za-z0-9_.]*)$

Connrules[].conninteger or stringRequired

How many connections this entry allows at once. May be written as a $variable, ${variable} or $env://NAME, like Conn.

integer over 0 · a whole number above zero, a $variable, or a $secret:// or $env:// reference

pattern

^([0-9]*[1-9][0-9]*|[$](env|secret)://.+|[$][{][^}]*[}]|[$][A-Za-z_][A-Za-z0-9_.]*)$

Keyrules[].keystringRequired

A template holding at least one $variable, expanded on every request. An entry whose key expands no variable is dropped and limits nothing.

a $variable, with or without braces

pattern

[$]([{][^}]+[}]|[A-Za-z_][A-Za-z0-9_.]*)

Key typekey_typestringdefault var

Whether Key is one variable name or a template of several. Not read when Rules carries entries.

one of var, var_combination

Default connection delaydefault_conn_delaynumberRequired

How long a waiting connection is held before it is let through.

over 0 · in seconds

Behavior

Policypolicystringdefault local

one of local, redis, redis-cluster

When Policy is redis

Key TTLkey_ttlintegerdefault 3600

How long a connection count stays on the Redis server after its last use.

in seconds

Redis databaseredis_databaseintegerdefault 0

at least 0

Redis hostredis_hoststringRequired

at least 2 characters

Redis keepalive poolredis_keepalive_poolintegerdefault 100

at least 1

Redis keepalive timeoutredis_keepalive_timeoutintegerdefault 10000

at least 1000 · in milliseconds

Redis usernameredis_usernamestring

at least 1 character

Redis passwordredis_passwordstring

Stored on the engine and shown only to roles that manage this object.

Redis portredis_portintegerdefault 6379

at least 1

Redis SSLredis_sslbooleandefault false

Redis SSL verifyredis_ssl_verifybooleandefault false

Server name for TLSredis_server_namestring

The server name sent in the TLS handshake and, with verification on, checked against the certificate; unset, the host is sent, and an address never is.

at least 1 character

Redis timeoutredis_timeoutintegerdefault 1000

at least 1 · in milliseconds

When Policy is redis-cluster· 9 fields

Key TTLkey_ttlintegerdefault 3600

How long a connection count stays on the Redis server after its last use.

in seconds

Redis cluster nameredis_cluster_namestringRequired

Redis cluster nodesredis_cluster_nodesarray of stringsRequired

at least 1 item; each 2 to 100 characters

Redis cluster SSLredis_cluster_sslbooleandefault false

Redis cluster SSL verifyredis_cluster_ssl_verifybooleandefault false

Redis keepalive poolredis_keepalive_poolintegerdefault 100

at least 1

Redis keepalive timeoutredis_keepalive_timeoutintegerdefault 10000

at least 1000 · in milliseconds

Redis passwordredis_passwordstring

Stored on the engine and shown only to roles that manage this object.

Redis timeoutredis_timeoutintegerdefault 1000

at least 1 · in milliseconds

Rejected coderejected_codeintegerdefault 503

200 to 599

Rejected messagerejected_msgstring

Text the client reads in the JSON body of a refused request, such as Too many connections. Left empty, 503 and 429 are answered with the server's HTML error page and any other code with an empty body.

at least 1 character

Allow degradationallow_degradationbooleandefault false

Let requests through when the limit cannot be counted, rather than answering 500.

Advanced

Only use default delayonly_use_default_delaybooleandefault false

Hold every waiting connection for the default delay, rather than for a delay measured from how long the upstream took.

Every plugin also has Enabled, Priority, Filter, Error response and Pre-function: see Plugin execution settings.