Skip to main content

How an installation is laid out

An installation has two halves that are installed separately and fail separately: the gateway cluster your clients call, and G8, the console and server that manage it. Two stores sit beside the cluster, a log store for the record of each request and a Prometheus for the counts. This page says what each piece holds, which shapes the installation guides build, what keeps serving when something fails, and what a backup has to cover.

The pieces of an installation: the gateway cluster your clients call, G8 on its database, and the two stores beside the clusterTwo halves, installed separately, and the stores beside the clusterClientsthe apps thatcall your APIsThe gateway clusterGateway nodesone or three, behinda load balancerConfigurationstoreone or threereadsBackendsyour services,upstream nodesBeside the clusterLog storeone record perrequestPrometheuscounts andresponse timesthe gateway writes the records; Prometheus scrapes the nodesG8Console and serverone image, one ormore replicasPostgreSQLusers, roles,environments, auditDirectoryActive Directory,sign-in and groupswrites objects throughthe Admin APIreads records and countsPeoplein a browser,or a scriptOne installation manages several clusters: each one is an environment,and the console writes to the chosen environment's cluster and no other.The gateway keeps serving when G8 is down: it reads its configuration store, not G8.
The gateway cluster is installed first and runs on its own. G8 is pointed at its Admin API and at the two stores beside it.

The pieces and what each one holds​

PieceWhat runs thereWhat it holds
Gateway nodesThe gateway your clients call, one or three nodes behind a load balancer.Nothing of its own; each node reads the configuration store and keeps serving from what it read.
Configuration storeOne or three members beside the nodes.The routes, services, upstreams, consumers, plugins and certificates: everything the console writes to an environment.
G8 serverOne image that serves the console and the API, one or more replicas.Nothing between restarts; every setting is read at start and everything else is in the database.
PostgreSQLA database you provide.Users, roles and their assignments, service accounts, the environments with their sealed admin keys, the directory connection, the audit log, the alert rules and history.
Log storeElasticsearch or OpenSearch, one per environment or shared.One record per request, written by the gateway into a day's index named after the environment.
PrometheusOne per environment or shared.The counts and response times the gateway publishes, scraped from the nodes under the environment's label.
Active DirectoryYours.The people who sign in with a domain account and the groups that give them roles.

The two halves talk in one direction each. G8 writes to a cluster through its Admin API, and reads the two stores; the gateway never calls G8. A G8 server that is down leaves every gateway serving exactly what it served before, and the console comes back on the same database as if nothing had happened. How the console writes to the gateway follows one write; Where request records go and Where the numbers come from follow the two stores.

One installation manages several clusters. Each is an environment, chosen in the switcher at the top of the console, and every write goes to that environment's cluster and no other; Environments explains what stays separate.

The topologies the guides install​

GuideNodesStore membersFor
Dev on Docker11Trying things out on one machine. Not highly available.
Test on a virtual machine11A test environment without containers. Not highly available.
Production on Docker33Production on virtual machines, with a load balancer of yours in front of the three nodes.
OKD33Production on OKD or OpenShift; the router is the load balancer.
k3s33Production on k3s or another Kubernetes.

Each guide installs the cluster, verifies it, and sets up the log store and Prometheus beside it. G8 itself is installed afterwards, on OKD or OpenShift with its chart, as Install G8 on OKD or OpenShift shows, and pointed at the cluster's Admin API.

What keeps serving when something fails​

  • The gateway cluster. With three store members, the store keeps accepting writes while two of them agree, so one member can be lost; the production guides measure that. With three nodes behind a load balancer, one node can be lost. The OKD guide says plainly that three replicas on a single-node cluster verify the shape of the manifests and not fault tolerance, because the node takes all of them with it.
  • G8. The chart's replicaCount can be above 1, and the replicas share nothing but the database. Each one probes every environment on its own, and the sign-in and token rate limits are counted per replica, so an address gets up to that many times the limit. An upgrade or a key rotation stops every replica before starting the new ones, because the Deployment uses the strategy Recreate; for that moment the console answers nothing while every gateway keeps serving.
  • PostgreSQL. It is yours, and its availability is yours to arrange. The chart's own PostgreSQL, switched on with db.internal.enabled, runs one replica without backup and its notes call it a lab store.
  • The two stores. A log store or a Prometheus that is down costs you records or counts for that time and nothing else; the gateway and the console keep working, and the system alerts say which store is unreachable. Alerts lists them.

What a backup covers​

A G8 backup is the PostgreSQL database and the key that seals the secrets in it, G8_SECRETS_KEY; without the key the server refuses to start on a restored database. It brings back who may sign in, which environments exist and how to reach them, and who changed what. It brings back no route: the gateway's own configuration lives in the configuration store, which each cluster guide backs up on its own. Request records and counts are not backed up by G8 either; they live in the two stores for as long as those keep them. Back up and restore is the procedure.

G8 dev