Decision logic
How moderation decisions are determined
Every completed moderation job reports is_prohibited as a boolean. While the job is still
processing, is_prohibited is null.
Decision rule#
On an image or video job, the AI Adult General check owns the decision:
is_prohibited: true: the check found a breach of the content policy, of one of your custom categories, or content depicting an apparent minor.is_prohibited: false: the check found nothing prohibited.
AI Adult General trips (built-in policy, custom category, or apparent minor) → is_prohibited: true
AI Adult General does not trip → is_prohibited: false
The decision is binary, and there is no threshold for you to set. There is also no categories
object on the response: the reason string explains a rejection instead. A completed text or audio
job can also carry informational risk scores, but they never change the decision.
Example: a rejection#
reason is present only when is_prohibited is true. It names the policy or the custom category
(by slug, in parentheses) that tripped.
{
"is_prohibited": true,
"reason": "The image contains references to coffee and coffee beans (coffee).",
"nsfw": false,
"job_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"type": "image",
"status": "completed",
"created_at": "2026-05-19T12:00:00.000Z",
"completed_at": "2026-05-19T12:00:05.000Z"
}
The NSFW label is not a decision#
nsfw is informational. It reports whether an image or video contains nudity or sexual content, and
it never rejects a job on its own. An image can return is_prohibited: false together with
nsfw: true. Apply your own product rule to that label.
The field is present only when the NSFW check is enabled for your account. See per-account category toggles.
Risk scores#
A completed text or audio job, and each completed item of a text moderation batch, can carry two
extra fields with a risk score per category: scores for the built-in policy categories, and
custom_scores for your custom categories.
Omnifence returns a risk score from 0 to 1, rounded to two decimals: 0 means the content
clearly does not match the category, and 1 means it clearly matches.
Image and video jobs do not currently return risk scores.
{
"is_prohibited": false,
"scores": { "violence": 0.04, "weapons": 0.12 },
"custom_scores": { "brand_logos": 0.71 },
"job_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"type": "text",
"status": "completed",
"created_at": "2026-11-09T12:00:00.000Z",
"completed_at": "2026-11-09T12:00:05.000Z"
}
- Keys: a
scoreskey is the ID of the built-in category, without thedefault_prefix thatenabled_categoriesuses. For example, the category that you configure asdefault_violencehas the score keyviolence. See default categories. Acustom_scoreskey is the slug of your custom category. - Scores are informational.
is_prohibitedis the decision. A score can be high on content that passed, and low on content that was rejected. A score is not a percentage chance and is not a calibrated probability, so do not show it to your users as one. - Both fields are optional. They are absent on an image or video job, and on a pending or failed job. They are also absent on a completed job that produced no score, for example a text job rejected by the blocked-keyword list, very long text, or a job where scoring was not available. A single category key can be missing too. Do not require any key in your integration.
- Child-safety categories are never scored. Child-safety detection returns a verdict only, so no
child-safety key appears in
scores. - Disabled categories are not scored. A category you switched off in
enabled_categorieshas no key. - Audio: the scores come from the check on the transcript and use the same keys as text.
The fields appear on the job status endpoint, in the completed
webhook, and on each item of a
text moderation batch.
Image and video specifics#
- Every enabled custom category is evaluated as part of the AI Adult General check, so a custom match rejects the job the same way a built-in policy breach does.
- Image and video jobs do not return risk scores.
Text and audio specifics#
- These modalities carry no
nsfwlabel, because no image or video is evaluated. is_prohibitedistruewhen the text (or transcript, for audio) is judged prohibited.reasonexplains why.- Every enabled custom category is evaluated as part of that same check, so a custom match rejects a
text or audio job the way it rejects an image one. The
reasonnames the category by slug. - An audio job's risk scores come from the check on its transcript, so they use the same keys as a text job's.
Content involving minors#
Content that appears to depict a minor is rejected by the always-on AI Adult General check, which has no account toggle. Repeated detections may be reviewed manually and can result in account termination at our discretion. See account termination.