Skip to main content

Request records

The gateway can write a record of every request it handles to a log store, Elasticsearch or OpenSearch. Traffic Explorer, under Traffic Explorer in the menu, reads those records. Each row is one request: what came in, what went out and how long it took.

Who may open it​

Opening the page needs the view permission of Traffic Explorer in the environment (traffic.view), and the Environment administrator holds it. The Editor and the Viewer do not: the page is absent from their menu, and opening it by its address shows a page that names their role. A custom role can carry it; see Custom roles.

Two more permissions open the rest. The view permission of Request bodies (traffic_bodies.view) lets a role open a request or response body. The manage permission of Traffic Explorer (traffic.manage) lets it turn logging on, change what the gateway records and masks, change the address the gateway writes to, and set how long records are kept.

What has to be connected​

  1. The environment's log store. A Platform administrator enters its address and account under Observability in the environment's settings, as Add an environment shows. Without it, the page carries this title: The log store is not connected for Production.
  2. The store prepared for records: the ingest pipeline and the index template the installation guides apply. The Settings tab shows whether the store holds them under Ingest pipeline and Index template.
  3. Logging turned on. Until then, the page carries this title: Traffic logging is not turned on. An Environment administrator turns logging on from the same page with Turn on traffic logging. G8 then writes the global rule g8-request-records to the gateway, with the elasticsearch-logger plugin on it, which sends every request's record to the store.

The console has no switch that turns logging off. Logging stops when the elasticsearch-logger plugin on g8-request-records is switched off or removed, or when the rule is deleted, on the global rules page. That needs the manage permission of Global rules. Once the rule is gone, the page shows the title for logging that is not turned on.

Each environment's records sit in the store under names built from the environment's slug, one index per UTC day. For the slug dev the days are named g8-dev-access-*, and no environment reads another's.

Reading the records​

Traffic Explorer of an environment: the time range, the search box, the status filter and the table of records with time, method, path, status, latency, service, route and upstream node.

The Explorer tab lists requests with their time, method, path, status, latency, service, route, client and the backend node that answered. Search by path, client address, request ID, route, service or consumer, and narrow by any of them with the filter chips. The time range is up to 31 days; the presets go from the last 15 minutes to the last 30 days.

Open a row for the record. Its tabs show the Overview, the Request and the Response as the gateway saw them, and the Related records. Copy as cURL gives you the request to send again. Request and response bodies are shown only to a role with the Request bodies permission. The panel says what happens when one is opened: Viewing a body is written to the audit log.

The Live tab lists requests as they end.

When the store does not answer within ten seconds the page shows what it got and says so. Narrow the window or add a filter.

What a record holds​

The gateway writes one record per request it handles. Opening a row shows it in four tabs.

TabWhat it shows
OverviewThe Timing: the total time, split into the gateway's own part and the backend's. Then the Time, the Status, the Client IP and the Host.
OverviewThe Service, the Route and the Consumer, each linked to its object. A request that matched no route says no route matched.
OverviewThe Upstream node that answered, with the number of attempts. The Sizes of the request and the response, and the Gateway node with its version.
OverviewThe request's x-request-id header, and Stored in, the day that holds the record.
RequestThe request's headers and query string, and its body when the gateway recorded one.
ResponseThe response's headers, and its body when the gateway recorded one.
RelatedThe records that carry the same x-request-id header, from five minutes before this one to five minutes after.

The gateway masks a record before the record leaves it, so a masked value never reaches the log store. It replaces with ******** the values of three things that the Masking card of the Settings tab names.

  • Request headers. From the start these are authorization, proxy-authorization, cookie, apikey and x-api-key.
  • Query arguments. From the start these are token, access_token and apikey.
  • Fields of JSON bodies, named by a path, in the request body and the response body alike. None is named from the start, and a body is recorded only when the Capture card on the same tab says so.

Response headers are always stored as they arrived. A change to the lists applies to the records written from then on. When the gateway cannot mask a record, it stores the record without its headers, its query string and its bodies. In the record, a masked value is marked Masked by the gateway before it was stored.

A body is read only when you select Show body, and only by a role with the Request bodies permission. The server writes the view to the audit log before it reads the body. When that row cannot be written, the panel says: The audit log could not record this view, so the body was not read.

The filters​

Every filter narrows the list, the count and the export together, and each one is kept in the page's address, so a filtered view can be shared as a link.

FilterWhat it matches
Time rangeA preset from Last 15 minutes to Last 30 days, or a range of your own of up to 31 days.
StatusA status class, 2xx, 3xx, 4xx or 5xx, or one exact code.
Route, Service, Upstream, ConsumerThe requests of that object. Type part of a name into the search box to pick one.
Client IPOne client address, or a block of addresses in CIDR form.
Upstream nodeThe backend node that answered.
PathRequests whose path contains the text.
Request IDOne request ID.

The search box reads what you type and offers the filters it could mean. An address becomes a client filter, and other text a path, a request ID, or a name to pick. In a record, select a value such as the client address to show only requests with it.

Where the records live​

The records sit in the environment's own log store, the one in its settings, and G8 reads them nowhere else. Each day is named after the environment's slug and the UTC date: g8-dev-access-2026.10.01 holds the slug dev's records of 1 October. A request made at half past one in the morning of 2 October in a zone three hours ahead of UTC still belongs to 1 October.

The names are derived from the slug at every read and never stored. A search names every day of its window one by one. A list of days keeps only the names that are a real day of this environment's slug. So one environment never reads another's records, even when the two slugs begin alike.

A record's link names its day and its document. Opened in another environment, the same link finds nothing, and the panel says: It was deleted with its day, or the link names a record of another environment.

Export​

Export CSV downloads the records the current filter matches as a CSV file. An export stops at a hundred thousand records; narrow the window when the page says the match is larger.

How long records are kept​

Records are kept for a number of days you set per environment, from 1 to 30, under Retention on the Settings tab. The default is 30. Changing it needs the manage permission of Traffic Explorer, and each change leaves a Retention of request records changed row (traffic.retention_changed) in the audit log, with the number before and after.

Days are counted in UTC. With a retention of N days, the store keeps today and the N + 1 days before it. With the default of 30, it keeps today and the 31 days before it, thirty-two days in all. The dialog that sets the number puts it another way: Records older than 30 days are deleted at the next daily cleanup.

Once a day, the server deletes every older day by its own name and writes the deletion to the audit log. G8 can run as several servers on one database, and then one of them runs the deletion and the others skip that turn. The tab shows the Last cleanup and the Oldest index still in the store. When a run fails, the Request records cleanup failed system alert can tell you; see Alerts. Where request records go describes the deletion step by step.

Removing an environment stops the deletion of its old records. Clear them in the store yourself, or lower the retention before you remove the environment.

G8 dev