tcp-logger

Sends request logs to a TCP server.

Category ObservabilityAdd it to Route, Service, Consumer, Consumer group, Global rule, Plugin configEngine plugin tcp-loggerPriority 405

What it does

tcp-logger sends the records of the requests that reach the route or other object it is added to, to a server that listens on TCP at Host and Port. The records are collected on the gateway node and sent in batches, so one send can carry several records.

With TLS on, the connection is encrypted, but the collector's certificate is never checked. TLS options carries the server name the gateway sends in the handshake, not a list of options.

When to use it

• A log collector of the operations team listens on a plain TCP port, with no HTTP endpoint in front of it.
• The records of the orders API cross a network they should not cross in plain text, and the collector accepts TLS on its port.
• Every route should send the same few fields, the client, method, path, status and time, set once in the plugin metadata.

Not for records sent over UDP, without a connection — see udp-logger (Sends request logs to a UDP server), records sent to a syslog server — see syslog (Sends request logs to a syslog server) or records sent to an HTTP collector — see http-logger (Sends request logs to an HTTP endpoint in batches).

Set it up

Send request records to a TCP collector over TLS

The connection is encrypted, but the collector's certificate is not checked; the TLS options only name the server the gateway asks for.

Under Basics set Host to logs.internal, set Port to 5140, turn on TLS and set TLS options to logs.internal.

tcp-logger · as the console saves it

{
  "host": "logs.internal",
  "port": 5140,
  "tls": true,
  "tls_options": "logs.internal"
}

More scenarios

Shape the record every route sends to the TCP collectorenvironment-wide settings

Fill Log format as in the JSON below.

tcp-logger environment-wide settings · as the console saves it

{
  "log_format": {
    "client": "$remote_addr",
    "method": "$request_method",
    "path": "$uri",
    "status": "$status",
    "at": "$time_iso8601"
  }
}

Things to know

• Port — A port of 0 or above 65535 is refused when you save.
• TLS — The collector's certificate is never checked, and TLS options carries the server name sent in the handshake rather than a list of options.
• A Log format in the plugin metadata shapes the record of every route whose tcp-logger names no Log format of its own. In the example each record carries path with the path the client asked for.
• Log format extra — Added to the default record only: while a log format is set, on this object or in the plugin's metadata, the engine writes that format alone and reads nothing here.
• Timeout — A whole number of milliseconds: a fraction, 1.5 for example, is refused when you save.
• Include request body when — Not checked at save: an expression the engine cannot read is stored, and the body is then never logged. Only the gateway's error log says so.
• Include response body when — Not checked at save: an expression the engine cannot read is stored, and the body is then never logged. Only the gateway's error log says so.
• Batch size — A batch goes out when it holds Batch size records, when its first record has waited Buffer duration, or when nothing new arrived for Inactive timeout.
• In the plugin metadata, Pending entry limit caps the backlog. While more entries than this wait to be sent, new ones are dropped rather than queued.
• AdviceTurn on TLS whenever the records cross a network you do not control.
• AdviceTurn on the request and response bodies only on routes whose collector is meant to hold what clients and the backend send.

Settings

The fields as the console shows them, section by section. A field under “When … is …” is there only while that value is set.

Basics

HosthoststringRequired

PortportintegerRequired

1 to 65535

TLStlsbooleandefault false

When TLS is true

TLS optionstls_optionsstring

Behavior

Timeouttimeoutintegerdefault 1000

at least 1 · in milliseconds

Log formatlog_formatobject

example {"host":"$host","client":"$remote_addr","status":"$status","at":"$time_iso8601"}

Log format extralog_format_extraobject

example {"zone":"eu-west","gateway":"$hostname"}

Include request bodyinclude_req_bodybooleandefault false

Include request body wheninclude_req_body_exprarray

at least 1 item · conditions on request variables: see Conditions on request variables

Include response bodyinclude_resp_bodybooleandefault false

Include response body wheninclude_resp_body_exprarray

at least 1 item · conditions on request variables: see Conditions on request variables

Request body limitmax_req_body_bytesintegerdefault 524288

at least 1

Response body limitmax_resp_body_bytesintegerdefault 524288

at least 1

Advanced

Batch sizebatch_max_sizeintegerdefault 1000

at least 1

Buffer durationbuffer_durationintegerdefault 60

at least 1 · in seconds

Inactive timeoutinactive_timeoutintegerdefault 5

at least 1 · in seconds

Retriesmax_retry_countintegerdefault 0

at least 0

Retry delayretry_delayintegerdefault 1

at least 0 · in seconds

Namenamestringdefault tcp logger

Environment-wide settings

Set once per environment, under Plugins › tcp-logger. Every tcp-logger in the environment uses these.

Log formatlog_formatobject

example {"host":"$host","client":"$remote_addr","status":"$status","at":"$time_iso8601"}

Log format extralog_format_extraobject

example {"zone":"eu-west","gateway":"$hostname"}

Pending entry limitmax_pending_entriesintegerdefault 8192

How many log entries may wait to be sent. While the backlog is above this number, new entries are dropped rather than queued.

at least 1

Every plugin also has Enabled, Priority, Filter, Error response and Pre-function: see Plugin execution settings.