Decision logic

How moderation decisions are determined

Every completed moderation job reports is_prohibited as a boolean. While the job is still processing, is_prohibited is null.

Decision rule#

On an image or video job, the AI Adult General check owns the decision:

  • is_prohibited: true: the check found a breach of the content policy, of one of your custom categories, or content depicting an apparent minor.
  • is_prohibited: false: the check found nothing prohibited.
Code
AI Adult General trips (built-in policy, custom category, or apparent minor) → is_prohibited: true
AI Adult General does not trip                                              → is_prohibited: false

The decision is binary, and there is no threshold for you to set. There is also no categories object on the response: the reason string explains a rejection instead. A completed text or audio job can also carry informational risk scores, but they never change the decision.

Example: a rejection#

reason is present only when is_prohibited is true. It names the policy or the custom category (by slug, in parentheses) that tripped.

JSON
{
  "is_prohibited": true,
  "reason": "The image contains references to coffee and coffee beans (coffee).",
  "nsfw": false,
  "job_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "type": "image",
  "status": "completed",
  "created_at": "2026-05-19T12:00:00.000Z",
  "completed_at": "2026-05-19T12:00:05.000Z"
}

The NSFW label is not a decision#

nsfw is informational. It reports whether an image or video contains nudity or sexual content, and it never rejects a job on its own. An image can return is_prohibited: false together with nsfw: true. Apply your own product rule to that label.

The field is present only when the NSFW check is enabled for your account. See per-account category toggles.

Risk scores#

A completed text or audio job, and each completed item of a text moderation batch, can carry two extra fields with a risk score per category: scores for the built-in policy categories, and custom_scores for your custom categories. Omnifence returns a risk score from 0 to 1, rounded to two decimals: 0 means the content clearly does not match the category, and 1 means it clearly matches.

Image and video jobs do not currently return risk scores.

JSON
{
  "is_prohibited": false,
  "scores": { "violence": 0.04, "weapons": 0.12 },
  "custom_scores": { "brand_logos": 0.71 },
  "job_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "type": "text",
  "status": "completed",
  "created_at": "2026-11-09T12:00:00.000Z",
  "completed_at": "2026-11-09T12:00:05.000Z"
}
  • Keys: a scores key is the ID of the built-in category, without the default_ prefix that enabled_categories uses. For example, the category that you configure as default_violence has the score key violence. See default categories. A custom_scores key is the slug of your custom category.
  • Scores are informational. is_prohibited is the decision. A score can be high on content that passed, and low on content that was rejected. A score is not a percentage chance and is not a calibrated probability, so do not show it to your users as one.
  • Both fields are optional. They are absent on an image or video job, and on a pending or failed job. They are also absent on a completed job that produced no score, for example a text job rejected by the blocked-keyword list, very long text, or a job where scoring was not available. A single category key can be missing too. Do not require any key in your integration.
  • Child-safety categories are never scored. Child-safety detection returns a verdict only, so no child-safety key appears in scores.
  • Disabled categories are not scored. A category you switched off in enabled_categories has no key.
  • Audio: the scores come from the check on the transcript and use the same keys as text.

The fields appear on the job status endpoint, in the completed webhook, and on each item of a text moderation batch.

Image and video specifics#

  • Every enabled custom category is evaluated as part of the AI Adult General check, so a custom match rejects the job the same way a built-in policy breach does.
  • Image and video jobs do not return risk scores.

Text and audio specifics#

  • These modalities carry no nsfw label, because no image or video is evaluated.
  • is_prohibited is true when the text (or transcript, for audio) is judged prohibited. reason explains why.
  • Every enabled custom category is evaluated as part of that same check, so a custom match rejects a text or audio job the way it rejects an image one. The reason names the category by slug.
  • An audio job's risk scores come from the check on its transcript, so they use the same keys as a text job's.

Content involving minors#

Content that appears to depict a minor is rejected by the always-on AI Adult General check, which has no account toggle. Repeated detections may be reviewed manually and can result in account termination at our discretion. See account termination.