Upstream
Upstreams are the backends that routes and services send traffic to.
upstreamWhat an upstream is
An upstream is a set of backend instances, its nodes, kept once under a name. Every route or service that chooses it under Target sends its requests to these nodes, spread over them by Type.
With an Active health check, the gateway probes every node and takes one that fails its probes out of rotation until it passes them again. Without one, every node stays in rotation whatever its state, and a request a stopped node refuses is tried on another node.
How the health check decides
With Active health check on, the gateway sends a request to Path on every node:
- A node in rotation is probed every Healthy › Interval seconds, a node taken out every Unhealthy › Interval seconds.
- A probe answered with one of Healthy › HTTP statuses counts as a success. Unless you change them, those are
200and302. - A probe answered with one of Unhealthy › HTTP statuses counts as a failure.
404,500and503are among them. - The node is taken out of rotation after a run of failed probes as long as HTTP failures. A run of probes that cannot connect, as to a stopped instance, counts under TCP failures, and a run left unanswered under Timeouts.
- A run of successes as long as Successes brings it back into rotation.
While another node is in rotation, a node taken out receives no requests.
When to use it
- Several routes send to the same backend instances, and you want to list them, and change them, in one place.
- An instance is restarted now and then, and requests should stop going to it until it answers again.
- A backend expects its own host name rather than the one clients call.
- Requests of one session have to keep reaching the same instance, or the instances are found through DNS rather than listed.
Guides
Pages that walk through a job with a upstream, step by step.
- Share a backend between routesKeep the backend nodes once as an upstream with a health check, and point routes at it.
Set it up
Take a backend node out of rotation while it fails its probe, and back once it answers
Each backend node must answer GET /health with 200 while it is well. Both intervals are set on purpose: the probes of healthy nodes run every Healthy interval, those of a node taken out every Unhealthy interval.
Under General set Name to orders. Under Targets choose Nodes and fill Nodes as in the JSON below. Under Balancing leave Type at roundrobin. Under Health checks fill Health checks as in the JSON below.
{
"name": "orders",
"type": "roundrobin",
"nodes": {
"10.0.1.10:8080": 1,
"10.0.1.11:8080": 1
},
"checks": {
"active": {
"http_path": "/health",
"healthy": {
"interval": 1,
"successes": 2
},
"unhealthy": {
"interval": 1,
"http_failures": 2
}
}
}
}
What you will see
- While both backend nodes pass their probes, repeated requests are shared by the two, each answered
200. - Each backend node receives
GET /healthfrom the gateway: the probe that decides whether it stays in rotation. - Once the second backend node has failed its probe twice in a row, every request is answered
200by the first: the second is out of rotation. - Once the second backend node answers
GET /healthwith200again, it is back in rotation and the two share the requests again.
More scenarios
Present the host name a backend expects
With verification on, the certificate of the node must be valid for orders.example.com, the name the request now carries.
Under General set Name to orders-public. Under Targets choose Nodes and fill Nodes as in the JSON below. Under Balancing leave Type at roundrobin, set Scheme to https, set Host passing to rewrite and set Upstream host to orders.example.com. Under TLS fill TLS as in the JSON below.
{
"name": "orders-public",
"type": "roundrobin",
"scheme": "https",
"nodes": {
"10.0.1.20:443": 1
},
"pass_host": "rewrite",
"upstream_host": "orders.example.com",
"tls": {
"verify": true
}
}
What you will see
- The backend receives the request with the host name
orders.example.comand answers200. - A route that sets a Host of its own with proxy-rewrite does not change it: the backend still receives
orders.example.com.
Keep each session on the same instance
Requests that carry the same session_id cookie reach the same instance; one without the cookie is placed by its client address instead, so that client, and every client behind the same address, always reaches the same instance.
Under General set Name to checkout. Under Targets choose Nodes and fill Nodes as in the JSON below. Under Balancing set Type to chash, set Hash on to cookie and set Key to session_id.
{
"name": "checkout",
"type": "chash",
"hash_on": "cookie",
"key": "session_id",
"nodes": {
"10.0.1.10:8080": 1,
"10.0.1.11:8080": 1
}
}
Ramp an instance added later up before it takes its full share
An instance added to the list later starts at 10 percent of its weight of 100 and reaches the whole of it over 60 seconds; the instances present when the upstream is created take their full share from the start.
Under General set Name to orders-warmed. Under Targets choose Nodes and fill Nodes as in the JSON below. Under Balancing leave Type at roundrobin and fill Slow start as in the JSON below.
{
"name": "orders-warmed",
"type": "roundrobin",
"nodes": {
"10.0.1.10:8080": 100,
"10.0.1.11:8080": 100
},
"warm_up_conf": {
"slow_start_time_seconds": 60,
"min_weight_percent": 10,
"interval": 5
}
}
Balance the orders service over two instances
Under General set Name to orders and set Description to Both instances take an equal share. Under Targets choose Nodes and fill Nodes as in the JSON below. Under Balancing leave Type at roundrobin.
{
"name": "orders",
"desc": "Both instances take an equal share",
"type": "roundrobin",
"nodes": {
"10.0.1.10:8080": 1,
"10.0.1.11:8080": 1
}
}
What you will see
- Repeated requests are all answered
200, some by the first instance and some by the second: the two share the traffic.
Find the instances of the orders service through DNS
DNS discovery must be enabled in the gateway's static configuration first; without it every request is refused with 503.
Under General set Name to orders-discovered. Under Targets choose Service discovery, set Discovery type to dns and set Service name to orders.internal.example.com:8080. Under Balancing leave Type at roundrobin.
{
"name": "orders-discovered",
"type": "roundrobin",
"discovery_type": "dns",
"service_name": "orders.internal.example.com:8080"
}
Things to know
- Slow start — Read only on a roundrobin upstream whose nodes share one priority; nodes of different priorities are refused at save.
- Refresh interval — Not above the ramp-up time: the engine refuses a larger interval at save.
- Trusted CA certificates — Not read beside a ws or wss scheme: the engine refuses the list at save when either is set.
- The engine assigns the
idwhen the object is created; it is not the name, and it is what other objects refer to. - Path — Point it at an address every instance answers with
200while it is well. A probe answered with404, as many APIs answer at the default/, counts as a failure. An instance that fails as many probes in a row as HTTP failures says is taken out of rotation as if it had stopped. - AdviceGive the service a health address of its own, for example
/health, that answers fast and answers200only while the instance can really serve requests. - Retries — Left empty, a request an instance refuses is tried on the other node, a GET and a POST alike: while one of two instances is stopped, clients are still answered by the other.
- Host passing — At
rewriteevery instance receives the name under Upstream host, also from a route that sets another Host with proxy-rewrite. - Host passing — At
pass, the default, a route that sets a Host with proxy-rewrite sends that Host to the instances.
Settings
The fields as the console shows them, section by section. A field under “When … is …” is there only while that value is set.
General
namestringRequireddescstringlabelsobjectKey and value pairs of your own, for grouping and filtering. They change nothing about traffic. An object whose keys are any name, each with a string as its value.
Targets
Where the traffic goes. Weight 0 keeps a node in the list without sending it traffic; a lower priority value is used only when every higher one is down.
Choose one:
Nodes
nodesobject or array of objectsRequiredAlso stored as a map from host:port to the weight.
nodes[].hoststringRequiredpattern
^\*$|^\*?[0-9a-zA-Z-._\[\]:]+$nodes[].weightintegerRequirednodes[].metadataobjectFree JSON kept beside the node. The engine compares nodes by it when a list is refreshed.
nodes[].portintegernodes[].priorityintegerdefault 0Service discovery
discovery_argsobjectdiscovery_args.group_namestringdiscovery_args.namespace_idstringdiscovery_typestringRequiredThe discovery driver must be enabled in the engine's configuration (dns, consul, nacos, eureka, kubernetes…).
service_namestringRequiredBalancing
typestringdefault roundrobinHow requests are spread over the nodes; a hash type groups them by the key below.
When Type is chash
hash_onstringdefault varsWhat Key names. vars: a variable such as remote_addr. header or cookie: its name. vars_combinations: variables joined, like $host$uri. consumer: Key is unused; the consumer name is hashed.
keystringRequired when Hash on is not consumerWhat the consistent hash groups requests by, such as remote_addr, a header or a cookie. A request that carries no value for Key is placed by its client address instead, with no error: such a client, and every client behind the same address, always reaches the same node.
When Type is roundrobin· 6 fields
warm_up_confobjectRamps a node that just appeared up to its full weight instead of handing it its whole share at once.
warm_up_conf.min_weight_percentintegerRequiredThe share of its weight, in percent, a new node begins with.
warm_up_conf.slow_start_time_secondsintegerRequiredHow long a new node takes to reach its full weight.
warm_up_conf.aggressionnumberdefault 1Linear at the default; a higher value front-loads the increase, a lower one holds it back.
warm_up_conf.intervalintegerdefault 1How often the effective weight is recomputed during the ramp.
warm_up_conf.startup_grace_period_secondsintegerdefault 0For this long after the data plane starts, a node seen for the first time counts as mature and skips the ramp.
schemestringdefault httpHow the engine speaks to the nodes: plain or TLS HTTP, gRPC, WebSocket, a raw TCP, TLS or UDP stream, or Kafka through the built-in pub/sub bridge.
pass_hoststringdefault passThe Host header the node receives: the request's own, the node's address, or a value written here.
When Host passing is rewrite
upstream_hoststringRequiredHost header sent to the nodes when host passing is rewrite.
pattern
^\*$|^\*?[0-9a-zA-Z-._\[\]:]+$retriesintegerAttempts on another node after a failure. Left empty, each other node is tried once.
retry_timeoutnumberHow long the retries of one request may take in total. 0 means no limit.
Health checks
checksobjectchecks.activeobjectRequiredThe engine probes every node on its own schedule. Without it, every node stays in rotation regardless of its state.
checks.active.concurrencyintegerdefault 10How many nodes are probed at the same time.
checks.active.healthyobjectchecks.active.healthy.http_statusesarray of integersdefault [200,302]A probe answered with one of these statuses counts as a success.
checks.active.healthy.intervalintegerdefault 1How often a node counted as healthy is probed.
checks.active.healthy.successesintegerdefault 2Successful probes in a row before an unhealthy node is taken back into rotation.
checks.active.hoststringHost header of the probe. Empty uses the node.
pattern
^\*$|^\*?[0-9a-zA-Z-._\[\]:]+$checks.active.http_methodstringdefault GETThe method the probe request is sent with.
checks.active.http_pathstringdefault /The path the probe requests on every node.
checks.active.http_req_bodystringdefault checks.active.https_verify_certificatebooleandefault trueApplies to HTTPS probes: the probe checks the node's certificate against the trusted CAs.
checks.active.portintegerEmpty uses the node port.
checks.active.req_headersarray of stringschecks.active.timeoutnumberdefault 1How long a probe waits for the node to answer before it counts as a timeout.
checks.active.typestringdefault httpWhat a probe is: an HTTP or HTTPS request to the path below, or only a TCP connection.
checks.active.unhealthyobjectchecks.active.unhealthy.http_failuresintegerdefault 5Probes in a row answered with an unhealthy status before the node is taken out of rotation.
checks.active.unhealthy.http_statusesarray of integersdefault [429,404,500,501,502,503,504,505]A probe answered with one of these statuses counts as an HTTP failure.
checks.active.unhealthy.intervalintegerdefault 1How often a node counted as unhealthy is probed.
checks.active.unhealthy.tcp_failuresintegerdefault 2Probes in a row that cannot connect before the node is taken out of rotation.
checks.active.unhealthy.timeoutsintegerdefault 3Probes in a row left unanswered in time before the node is taken out of rotation.
checks.passiveobjectMarks nodes from real traffic. The engine allows it only together with the active check.
checks.passive.healthyobjectchecks.passive.healthy.http_statusesarray of integersdefault [200,201,202,203,204,205,206,207,208,226,300,301,302,303,304,305,306,307,308]An answer to live traffic with one of these statuses counts as a success.
checks.passive.healthy.successesintegerdefault 5Answers in a row with a healthy status before an unhealthy node is taken back into rotation.
checks.passive.typestringdefault httpWhat live traffic is judged by: the HTTP answers the node gives, or only whether the TCP connection succeeds.
checks.passive.unhealthyobjectchecks.passive.unhealthy.http_failuresintegerdefault 5Answers in a row to live traffic with an unhealthy status before the node is taken out of rotation.
checks.passive.unhealthy.http_statusesarray of integersdefault [429,500,503]An answer to live traffic with one of these statuses counts as an HTTP failure.
checks.passive.unhealthy.tcp_failuresintegerdefault 2Requests in a row that cannot connect before the node is taken out of rotation.
checks.passive.unhealthy.timeoutsintegerdefault 7Requests in a row left unanswered in time before the node is taken out of rotation.
Timeouts
timeoutobjectSet together: once any of connect, send and read is given, all three are required. Left out, the engine's own timeouts apply.
timeout.connectnumberRequiredtimeout.readnumberRequiredtimeout.sendnumberRequiredTLS
tlsobjectHow the engine identifies itself to the nodes.
tls.ca_certsarray of stringsPEM certificates the node certificates are checked against when verification is on.
Choose one:
Client certificate from Certificates
tls.client_cert_idstring or integerA certificate of type client, kept under Certificates. Not stored beside Client certificate and Private key together.
pattern
^[a-zA-Z0-9-_.]+$Inline certificate and key
tls.client_certstringNeeds Private key beside it.
tls.client_keystringThe engine stores the key sealed and never returns it in readable form. Needs Client certificate beside it.
tls.verifybooleanChecks the node's certificate on https, grpcs, kafka and wss nodes; left out, an https, grpcs or wss node's certificate is not checked and any one is accepted. A stream route never checks it, and an upstream written inline under traffic-split checks it only over wss.
Keepalive
keepalive_poolobjectConnections the engine keeps open to the nodes.
keepalive_pool.idle_timeoutnumberdefault 60How long an idle connection waits in the pool before it is closed.
keepalive_pool.requestsintegerdefault 1000How many requests one pooled connection carries before it is closed and replaced.
keepalive_pool.sizeintegerdefault 320How many idle connections to the nodes each worker keeps open for reuse.