ai-aws-content-moderation

Moderates request content with AWS Comprehend before the LLM.

Category AIAdd it to Route, Service, Consumer, Consumer group, Global rule, Plugin configEngine plugin ai-aws-content-moderationPriority 1031

What it does

ai-aws-content-moderation sends the text of each prompt to Amazon Comprehend for a toxicity score before the model receives it, and with Check response on, the model's answer too. Content whose overall score passes Moderation threshold, or whose score in one category passes its value under Moderation categories, is refused with Deny code.

It scores only what an AI proxy on the same route handles, ai-proxy or ai-proxy-multi. The refusal is answered 200 unless you change Deny code, and client libraries read that as a normal completion, not as an error.

When to use it

• A community assistant whose prompts must not carry insults or harassment, scored before any model reads them.
• A company on AWS that already holds credentials for Comprehend in its own region, eu-west-1 for example.
• One category held stricter than the rest, for example PROFANITY under Moderation categories with a value below the overall threshold.
• Temporary credentials of an assumed role, given with their Session token.

Not for refusing prompts by patterns you write yourself — see ai-prompt-guard (Allows or denies prompts that match configured patterns), prompts rated by risk level with Alibaba Cloud content moderation — see ai-aliyun-content-moderation (Moderates request content with Alibaba Cloud before the LLM) or asking your own authorization service whether a request may go on — see forward-auth (Delegates authentication to an external service).

Set it up

Refuse prompts Amazon Comprehend scores as toxic

Acts only on a route that also carries ai-proxy or ai-proxy-multi.

Under Basics fill Comprehend as in the JSON below. Under Behavior set Moderation threshold to 0.8 and set Deny code to 403.

ai-aws-content-moderation · as the console saves it

{
  "comprehend": {
    "access_key_id": "$env://AWS_ACCESS_KEY_ID",
    "secret_access_key": "<secret>",
    "region": "eu-west-1"
  },
  "moderation_threshold": 0.8,
  "deny_code": 403
}

Things to know

• Moderation categories — A category over its value denies the content whatever the overall bar says, so a value below the overall bar makes that category stricter than the rest.
• Moderation categories — A category outside the six the help names is refused when you save.
• Moderation categories — Each value is a score from 0 to 1, and a value above 1 is refused when you save.
• Region — An empty region is refused when you save.
• Check request — A request whose content cannot be scored, because the service fails or answers something unreadable, is refused with 500. A streamed answer is let through unscored.
• Check response — On a streamed answer the default final_packet mode only tags flagged text and sends it on, while realtime cuts the stream.
• Fail mode — Nothing is scanned unless an AI proxy on the same route picked a model first, and the default then lets the request through in silence.
• Request check mode — A selected system role is checked on every request whatever the mode, and selecting assistant makes the last block longer.
• Request check roles — A role listed twice is refused when you save.
• Request check length limit — A segment may hold at most 1024 bytes, and 2000 is refused when you save.
• AdviceSet Deny code to a 4xx, for example 403, when your clients should treat a refused prompt as an error.

Settings

The fields as the console shows them, section by section.

Basics

ComprehendcomprehendobjectRequired

Access key IDcomprehend.access_key_idstringRequired

Regioncomprehend.regionstringRequired

at least 1 character

Secret access keycomprehend.secret_access_keystringRequired

Stored on the engine and shown only to roles that manage this object.

Endpointcomprehend.endpointstring

an http or https address

pattern

^https?://

Session tokencomprehend.session_tokenstring

The session token of temporary credentials, such as an assumed role's, left empty for a long-lived key pair. Stored on the engine and shown only to roles that manage this object.

SSL verifycomprehend.ssl_verifybooleandefault true

Check requestcheck_requestbooleandefault true

Check responsecheck_responsebooleandefault false

Behavior

Moderation thresholdmoderation_thresholdnumberdefault 0.5

The bar for the overall toxicity score, read on every request even where a category below carries a threshold of its own.

0 to 1

Moderation categoriesmoderation_categoriesobject

Thresholds per category, keyed by PROFANITY, HATE_SPEECH, INSULT, HARASSMENT_OR_ABUSE, SEXUAL or VIOLENCE_OR_THREAT, each denying the content on its own when exceeded. A category over its value denies the content whatever the overall bar says. An object whose keys are a name matching ^(PROFANITY|HATE_SPEECH|INSULT|HARASSMENT_OR_ABUSE|SEXUAL|VIOLENCE_OR_THREAT)$, each with a number as its value.

Deny codedeny_codeintegerdefault 200

The status a denied request is answered with, where 200 reads as a normal completion to client libraries and a 4xx as an error.

200 to 599

Deny messagedeny_messagestring

Fail modefail_modestringdefault skip

What to do with a request the plugin cannot inspect: skip lets it through, warn also logs it, and error rejects it with 500. Plain traffic that did not pass through an AI proxy is such a request.

one of error, skip, warn

Request check moderequest_check_modestringdefault all

Which messages of the selected roles to moderate: last is the latest consecutive block, and all is every one of them. A selected system role is checked either way, and choosing assistant widens the last block.

one of all, last

Request check rolesrequest_check_rolesarray of stringsdefault ["user","tool","system","assistant"]

The message roles moderated on the request side, all four by default. User, tool and assistant follow the mode above. A selected system role is checked on every request, and a tool result is read only where the protocol gives it a role of its own.

at least 1 item · one of assistant, system, tool, user

Request check length limitrequest_check_length_limitintegerdefault 1000

The longest text segment sent for scoring, where longer content is cut into several segments of the same call.

4 to 1024

Response check length limitresponse_check_length_limitintegerdefault 1000

The longest response text segment sent for scoring, where longer content is cut into several segments of the same call.

4 to 1024

Timeouttimeoutintegerdefault 10000

How long the moderation service has to answer one call.

at least 1 · in milliseconds

Advanced

Stream check modestream_check_modestringdefault final_packet

Realtime checks a streamed answer in batches and replaces the rest on a hit, while final packet checks it once at the end. Final packet tags the last chunk with the risk level.

one of final_packet, realtime

Stream check intervalstream_check_intervalnumberdefault 3

How long a batch of streamed text may gather before it is checked, in realtime mode.

at least 0.1 · in seconds

Stream check cache sizestream_check_cache_sizeintegerdefault 128

How much streamed text a batch may gather before it is checked, in realtime mode.

at least 1

Keepalivekeepalivebooleandefault true

Keeps the connection to the moderation service open between calls for reuse.

Keepalive timeoutkeepalive_timeoutintegerdefault 60000

How long a pooled connection to the moderation service may sit idle before it is closed.

at least 1000 · in milliseconds

Every plugin also has Enabled, Priority, Filter, Error response and Pre-function: see Plugin execution settings.