syslog

Sends request logs to a syslog server.

Category ObservabilityAdd it to Route, Service, Consumer, Consumer group, Global rule, Plugin configEngine plugin syslogPriority 401

What it does

syslog sends the records of the requests that reach the route or other object it is added to, to a syslog server at Host and Port, over tcp or udp as Socket type says. The lines are held back on the gateway node until Flush limit bytes have built up, however long that takes.

The lines always travel in plain text, whatever TLS says. For records that must travel encrypted, choose a logger that sends over an encrypted connection, for example http-logger with an HTTPS address.

When to use it

• The operations team gathers every log of the company on one syslog server, and the request records should join them there.
• A security team keeps the trail of who called the orders API on its own syslog collector.
• Every route should send the same few fields, the client, method, path, status and time, set once in the plugin metadata.
• The syslog server listens on TCP, and connections to it should stay open between requests.

Not for records that must travel encrypted — see http-logger (Sends request logs to an HTTP endpoint in batches), records sent to a Kafka topic — see kafka-logger (Sends request logs to a Kafka topic) or records kept in Elasticsearch — see elasticsearch-logger (Sends request logs to an Elasticsearch index).

Set it up

Send request records to a syslog server over TCP

A flush limit of 1 sends each record as soon as it is written instead of holding lines back until 4 KB have built up.

Under Basics set Host to syslog.internal, set Port to 514, leave Socket type at tcp and set Pool size to 10. Under Advanced set Flush limit to 1.

syslog · as the console saves it

{
  "host": "syslog.internal",
  "port": 514,
  "sock_type": "tcp",
  "flush_limit": 1,
  "pool_size": 10
}

More scenarios

Shape the record every route sends to syslogenvironment-wide settings

Fill Log format as in the JSON below.

syslog environment-wide settings · as the console saves it

{
  "log_format": {
    "client": "$remote_addr",
    "method": "$request_method",
    "path": "$uri",
    "status": "$status",
    "at": "$time_iso8601"
  }
}

Things to know

• Flush limit — With Flush limit at 1, each record goes to the server once it is written, without waiting for later requests.
• Flush limit — With the default, one request's line stays unsent until later requests have filled the buffer.
• Flush limit — The drop limit has to stand above the flush limit: at or under it the socket logger never starts and no line is ever sent.
• TLS — Turning this on changes nothing: the socket is opened under a name this value never reaches, so the lines always travel in plain text.
• Port — A port of 0 or above 65535 is refused when you save.
• Socket type — Only tcp and udp are accepted; http is refused when you save.
• Pool size — A pool below five connections, 4 for example, is refused when you save.
• A log format in the plugin metadata reaches the records of a route whose syslog names none of its own: each record carries path with the path the client asked for.
• Log format extra — Added to the default record only: while a log format is set, on this object or in the plugin's metadata, the engine writes that format alone and reads nothing here.
• Batch size — A batch goes out when it holds Batch size records, when its first record has waited Buffer duration, or when nothing new arrived for Inactive timeout.
• Pending entry limit — While more entries than this wait to be sent, new ones are dropped rather than queued.
• Include request body when — Not checked at save: an expression the engine cannot read is stored, and the body is then never logged. Only the gateway's error log says so.
• Include response body when — Not checked at save: an expression the engine cannot read is stored, and the body is then never logged. Only the gateway's error log says so.
• AdviceOn a route with little traffic, set a low Flush limit, as the example does, and keep Drop limit well above it.
• Drop limit — The drop limit has to stand above the flush limit: at or under it the socket logger never starts and no line is ever sent.

Settings

The fields as the console shows them, section by section. A field under “When … is …” is there only while that value is set.

Basics

HosthoststringRequired

PortportintegerRequired

1 to 65535

Socket typesock_typestringdefault tcp

one of tcp, udp

When Socket type is tcp

Pool sizepool_sizeintegerdefault 5

How many connections to the collector are kept open for reuse. Kept connections never time out.

at least 5

Behavior

TLStlsbooleandefault false

Timeouttimeoutintegerdefault 3000

How long a send may take before it is given up.

at least 1 · in milliseconds

Log formatlog_formatobject

The fields written in each entry, each a name and the request variable it takes its value from, such as client = $remote_addr.

example {"host":"$host","client":"$remote_addr","status":"$status","at":"$time_iso8601"}

Log format extralog_format_extraobject

example {"zone":"eu-west","gateway":"$hostname"}

Include request bodyinclude_req_bodybooleandefault false

Include request body wheninclude_req_body_exprarray

at least 1 item · conditions on request variables: see Conditions on request variables

Include response bodyinclude_resp_bodybooleandefault false

Include response body wheninclude_resp_body_exprarray

at least 1 item · conditions on request variables: see Conditions on request variables

Request body limitmax_req_body_bytesintegerdefault 524288

at least 1

Response body limitmax_resp_body_bytesintegerdefault 524288

at least 1

Advanced

Batch sizebatch_max_sizeintegerdefault 1000

How many entries are sent together in one batch.

at least 1

Buffer durationbuffer_durationintegerdefault 60

How long entries may wait for a batch to fill before it is sent anyway.

at least 1 · in seconds

Inactive timeoutinactive_timeoutintegerdefault 5

How long the buffer may sit idle before it is flushed.

at least 1 · in seconds

Retriesmax_retry_countintegerdefault 0

How many times a failed send is retried.

at least 0

Retry delayretry_delayintegerdefault 1

at least 0 · in seconds

Flush limitflush_limitintegerdefault 4096

Bytes of log lines held back before a send. Lines wait until this much has built up, however long that takes. With the default, one request's line stayed unsent until later requests had filled the buffer. Keep it below Drop limit.

at least 1

Drop limitdrop_limitintegerdefault 1048576

The buffered size, in bytes, at which further entries are dropped.

Namenamestringdefault sys logger

A name for this logger, so its connections are kept apart from other loggers' in the pool.

Environment-wide settings

Set once per environment, under Plugins › syslog. Every syslog in the environment uses these.

Log formatlog_formatobject

example {"host":"$host","client":"$remote_addr","status":"$status","at":"$time_iso8601"}

Log format extralog_format_extraobject

example {"zone":"eu-west","gateway":"$hostname"}

Pending entry limitmax_pending_entriesintegerdefault 8192

How many log entries may wait to be sent. While the backlog is above this number, new entries are dropped rather than queued.

at least 1

Every plugin also has Enabled, Priority, Filter, Error response and Pre-function: see Plugin execution settings.