Moderations

Create moderation

Classify text inputs for harmful content

POST/v1/moderations

Check one or more text inputs for harmful content. This endpoint's schema is compatible with the OpenAI Moderations API. It works with the OpenAI SDK.

The response includes standard OpenAI moderation categories and AILuminate safety classifications for more detailed signals.

See Errors in the Moderation guide for status codes and what to do.

Authorizations

Authorizationstringrequired

Your authentication credentials. For Basic authentication, please populate Basic $INWORLD_API_KEY.

A standard API key can call chat completions, the Responses and Messages APIs, moderation and Decisions. To create, update or delete routers, the key also needs write permissions for the Router API. You can create a key in one command with the Inworld CLI: inworld workspace add-key.

Body

application/json

inputoneOfrequired

A string, an array of strings, or an array of {"type": "text", "text": ...} parts to classify. Image inputs are not supported. Up to 16 inputs of up to 32 KB each. Only about 3 KB of a long input is classified, half from its beginning and half from its end; the middle is not checked, and the response doesn't say so.

modelstringdefault: "inworld/moderation-latest"

Accepted for OpenAI SDK compatibility; the Inworld moderation model is always used.

Response

200 - application/json

idstring

Unique identifier for the moderation request.

modelstring

The model used for classification.

resultsobject[]

Array of moderation results, one per input.

Show child attributes

flaggedboolean

true when ailuminate.safety is unsafe. A category can be true while flagged is false, when the content is controversial.

categoriesobject

OpenAI-compatible category flags, derived from the AILuminate classification. sexual/minors, harassment/threatening, hate, hate/threatening, self-harm/intent and violence/graphic are currently always false.

Show child attributes

sexualboolean

sexual/minorsboolean

harassmentboolean

harassment/threateningboolean

hateboolean

hate/threateningboolean

illicitboolean

illicit/violentboolean

self-harmboolean

self-harm/intentboolean

self-harm/instructionsboolean

violenceboolean

violence/graphicboolean

category_scoresobject

1 or 0 for each category, mirroring categories. The classifier returns labels, not calibrated probabilities, so thresholding these scores has no effect. Returned as integers.

Show child attributes

sexualnumber

sexual/minorsnumber

harassmentnumber

harassment/threateningnumber

hatenumber

hate/threateningnumber

illicitnumber

illicit/violentnumber

self-harmnumber

self-harm/intentnumber

self-harm/instructionsnumber

violencenumber

violence/graphicnumber

category_applied_input_typesobject

Input types evaluated for each category.

ailuminateobject

Additional safety classifications based on the AILuminate benchmark by MLCommons. Extends the standard OpenAI moderation categories with more granular signals.

Show child attributes

safetyenum<string>

Overall safety assessment.

Available options:safeunsafecontroversial

categoriesobject

Boolean flags for AILuminate-specific safety categories.

Show child attributes

violent_crimesboolean

child_sexual_exploitationboolean

suicide_self_harmboolean

indiscriminate_weaponsboolean

intellectual_propertyboolean

defamationboolean

non_violent_crimesboolean

hateboolean

specialized_adviceboolean

privacyboolean

sexual_contentboolean

extensionsobject

Additional classification signals.

Show child attributes

politically_sensitiveboolean

unethical_actsboolean

jailbreakboolean

refusalboolean

Whether the content represents a refusal.

Create moderation

curl --request POST \
  --url https://api.inworld.ai/v1/moderations \
  --header 'Authorization: <api_key>' \
  --header 'Content-Type: application/json' \
  --data '{
  "input": "Hello world!"
}'

Single input

{
  "id": "modr-5cf388daeb3a41b9803fe73f4345a5c3",
  "model": "inworld/moderation-latest",
  "results": [
    {
      "flagged": false,
      "categories": {
        "sexual": false,
        "sexual/minors": false,
        "harassment": false,
        "harassment/threatening": false,
        "hate": false,
        "hate/threatening": false,
        "illicit": false,
        "illicit/violent": false,
        "self-harm": false,
        "self-harm/intent": false,
        "self-harm/instructions": false,
        "violence": false,
        "violence/graphic": false
      },
      "category_scores": {
        "sexual": 0,
        "sexual/minors": 0,
        "harassment": 0,
        "harassment/threatening": 0,
        "hate": 0,
        "hate/threatening": 0,
        "illicit": 0,
        "illicit/violent": 0,
        "self-harm": 0,
        "self-harm/intent": 0,
        "self-harm/instructions": 0,
        "violence": 0,
        "violence/graphic": 0
      },
      "category_applied_input_types": {
        "sexual": [
          "text"
        ],
        "sexual/minors": [
          "text"
        ],
        "harassment": [
          "text"
        ],
        "harassment/threatening": [
          "text"
        ],
        "hate": [
          "text"
        ],
        "hate/threatening": [
          "text"
        ],
        "illicit": [
          "text"
        ],
        "illicit/violent": [
          "text"
        ],
        "self-harm": [
          "text"
        ],
        "self-harm/intent": [
          "text"
        ],
        "self-harm/instructions": [
          "text"
        ],
        "violence": [
          "text"
        ],
        "violence/graphic": [
          "text"
        ]
      },
      "ailuminate": {
        "safety": "safe",
        "categories": {
          "violent_crimes": false,
          "sex_related_crimes": false,
          "child_sexual_exploitation": false,
          "suicide_self_harm": false,
          "indiscriminate_weapons": false,
          "intellectual_property": false,
          "defamation": false,
          "non_violent_crimes": false,
          "hate": false,
          "specialized_advice": false,
          "privacy": false,
          "sexual_content": false
        },
        "extensions": {
          "politically_sensitive": false,
          "unethical_acts": false,
          "jailbreak": false
        },
        "refusal": false
      }
    }
  ]
}