tencent-cloud-cls

Sends request logs to Tencent Cloud Log Service.

Category ObservabilityAdd it to Route, Service, Consumer, Consumer group, Global rule, Plugin configEngine plugin tencent-cloud-clsPriority 397

What it does

tencent-cloud-cls sends the records of the requests that reach the route or other object it is added to, to Tencent Cloud Log Service, into the CLS topic on the CLS host. The records are collected on the gateway node and sent in batches, each one a POST to /structuredlog that names the topic, signed with the Secret ID and Secret key.

CLS host is the host alone, ap-guangzhou.cls.tencentcs.com for example: the Scheme, https unless you change it, is put in front of it. A host written with its scheme is refused when you save.

When to use it

• The operations team reads its logs in Tencent Cloud Log Service, and the records of the orders API should reach a topic there, api-gateway-access for example.
• Every record should carry the same few labels, the cluster or the environment for example, under Global tag.
• A busy route should send only part of its records, under Sample ratio.
• Every route should write the same few fields, the client address, the method, the path, the status and the time, set once in the plugin metadata.

Not for records kept in Google Cloud Logging — see google-cloud-logging (Sends request logs to Google Cloud Logging), records sent to a collector of your own behind an HTTP endpoint — see http-logger (Sends request logs to an HTTP endpoint in batches) or records sent to a Kafka topic — see kafka-logger (Sends request logs to a Kafka topic).

Set it up

Send request records to Tencent Cloud Log Service over HTTPS

CLS_SECRET_ID is read from the environment of every gateway node when the configuration loads.

Under Basics set CLS host to ap-guangzhou.cls.tencentcs.com, set CLS topic to api-gateway-access, set Secret ID to $env://CLS_SECRET_ID and set Secret key to <secret>. Under Behavior leave Scheme at https and turn on SSL verify.

tencent-cloud-cls · as the console saves it

{
  "cls_host": "ap-guangzhou.cls.tencentcs.com",
  "cls_topic": "api-gateway-access",
  "secret_id": "$env://CLS_SECRET_ID",
  "secret_key": "<secret>",
  "scheme": "https",
  "ssl_verify": true
}

More scenarios

Shape the record every route sends to Tencent Cloud Log Serviceenvironment-wide settings

Fill Log format as in the JSON below.

tencent-cloud-cls environment-wide settings · as the console saves it

{
  "log_format": {
    "client": "$remote_addr",
    "method": "$request_method",
    "path": "$uri",
    "status": "$status",
    "at": "$time_iso8601"
  }
}

Things to know

• CLS topic — Each batch arrives as a POST to /structuredlog?topic_id=api-gateway-access for the topic api-gateway-access, so the value is the topic's ID.
• CLS host — The host alone, without a scheme: the scheme below is put in front of it and /structuredlog after it.
• CLS host — A host written with its scheme, https://ap-guangzhou.cls.tencentcs.com for example, is refused when you save.
• CLS host — A bare IPv6 address, 2001:db8::1 for example, is refused when you save.
• Scheme — Only http and https are accepted; ftp is refused when you save.
• Global tag — Each pair is written into the log entry after it is built, so a name the log format already uses is overwritten.
• Global tag — Pairs of a name and a value: a plain list of values is refused when you save.
• Sample ratio — A ratio above one, 1.5 for example, is refused when you save.
• A log format in the plugin metadata reaches the records of a route whose tencent-cloud-cls names none of its own: a field set to $time_iso8601 arrives as a time that ends in +00:00.
• Log format extra — Added to the default record only: while a log format is set, on this object or in the plugin's metadata, the engine writes that format alone and reads nothing here.
• Include request body when — The console’s form draws one level of conditions; a value that nests deeper is kept as it is and edited in Raw JSON.
• Include response body when — The console’s form draws one level of conditions; a value that nests deeper is kept as it is and edited in Raw JSON.
• Batch size — A batch goes out when it holds Batch size records, when its first record has waited Buffer duration, or when nothing new arrived for Inactive timeout.
• In the plugin metadata, while more entries than the Pending entry limit wait to be sent, new ones are dropped rather than queued.
• AdviceKeep the Secret ID and Secret key out of the configuration with a reference, as the example does with $env://CLS_SECRET_ID.
• AdviceTurn on the request and response bodies only on routes whose topic is meant to hold what clients and the backend send.

Settings

The fields as the console shows them, section by section. A field under “When … is …” is there only while that value is set.

Basics

CLS hostcls_hoststringRequired

pattern

^([$]secret://.+|[$][Ee][Nn][Vv]://.+|([^/:\s@\[\]]+|\[[0-9A-Fa-f.]*:[0-9A-Fa-f:.]*\])(:([1-9][0-9]{0,3}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5]))?)$

CLS topiccls_topicstringRequired

Secret IDsecret_idstringRequired

Secret keysecret_keystringRequired

Stored on the engine and shown only to roles that manage this object.

Behavior

Schemeschemestringdefault https

one of http, https

When Scheme is https

SSL verifyssl_verifybooleandefault true

Global tagglobal_tagobject

Sample ratiosample_rationumberdefault 1

0.00001 to 1

Log formatlog_formatobject

example {"host":"$host","client":"$remote_addr","status":"$status","at":"$time_iso8601"}

Log format extralog_format_extraobject

example {"zone":"eu-west","gateway":"$hostname"}

Include request bodyinclude_req_bodybooleandefault false

Include request body wheninclude_req_body_exprarray

at least 1 item · conditions on request variables: see Conditions on request variables

Include response bodyinclude_resp_bodybooleandefault false

Include response body wheninclude_resp_body_exprarray

at least 1 item · conditions on request variables: see Conditions on request variables

Request body limitmax_req_body_bytesintegerdefault 524288

at least 1

Response body limitmax_resp_body_bytesintegerdefault 524288

at least 1

Advanced

Batch sizebatch_max_sizeintegerdefault 1000

at least 1

Buffer durationbuffer_durationintegerdefault 60

at least 1 · in seconds

Inactive timeoutinactive_timeoutintegerdefault 5

at least 1 · in seconds

Retriesmax_retry_countintegerdefault 0

at least 0

Retry delayretry_delayintegerdefault 1

at least 0 · in seconds

Namenamestringdefault tencent-cloud-cls

Environment-wide settings

Set once per environment, under Plugins › tencent-cloud-cls. Every tencent-cloud-cls in the environment uses these.

Log formatlog_formatobject

example {"host":"$host","client":"$remote_addr","status":"$status","at":"$time_iso8601"}

Log format extralog_format_extraobject

example {"zone":"eu-west","gateway":"$hostname"}

Pending entry limitmax_pending_entriesintegerdefault 8192

How many log entries may wait to be sent. While the backlog is above this number, new entries are dropped rather than queued.

at least 1

Every plugin also has Enabled, Priority, Filter, Error response and Pre-function: see Plugin execution settings.