AI Moderation APIs

Create and manage content moderation rules to maintain a safe environment across your CometChat app.

The Moderation API provides endpoints for managing content moderation rules and keyword lists. You can define rules that automatically detect and block inappropriate content in messages, including text profanity, AI-based image/video moderation, and custom keyword filters.

Key Behaviors

• Default rules cover text profanity, AI-based image/video moderation, custom profanity, contact details, and email content.
• Text and custom profanity filters scan for words (including regex patterns) and drop the message if a violation is detected.
• AI-based image/video moderation scans attachments and discards the message if a violation is found.
• When a rule is violated, the message is blocked. Optionally, the sender can be kicked or banned (in groups) or blocked (in 1-on-1 conversations).

How Moderation Connects to Other Resources

• Messages — Moderation rules are evaluated against Messages in real time before delivery.
• Groups — Violations in group messages can trigger member kicks or bans via Group Members and Banned Users.
• Blocked Users — Violations in 1-on-1 messages can trigger a block on the sender.

Available Endpoints

Operation	Description
Rules	Create, read, update, and delete moderation rules
Keywords	Manage keyword lists or regex patterns for text moderation
Blocked Messages	Retrieve messages that violated moderation rules
Flagged Messages	Retrieve flagged messages for review
Reasons	Manage custom flag reasons
Rule Revisions	Fetch historical revisions of a rule

AI Image and Video Moderation Labels

Label	Description
Violence	Instruments or devices used to cause harm — firearms, sharp weapons, explosives, and ammunition
Alcohol	Drinking alcoholic beverages from bottles or glasses
Gambling	Participating in games of chance — cards, roulette, slot machines
Drugs & Tobacco	Tablets, capsules, or the act of smoking cigarettes, cigars, e-cigarettes, hookah, or joints
Rude gestures	Hand gesture with the middle finger extended
Explicit Nudity	Visible human genitalia
Non-Explicit Nudity	Exposed human back from neck to spine without full occlusion
Swimwear or underwear	Swimwear or undergarments — bikinis, swim trunks, swim briefs, etc.
Visually Disturbing	Extremely thin, undernourished human bodies with severe physical wasting
Hate symbols	Symbols, flags, or gestures associated with extremist or terrorist groups

API Constraints

Need higher limits? Contact CometChat Support to discuss enterprise options.

Rules

| Parameter | Limit | |-----------|-------| | Rule ID | Max 100 characters, no spaces or special characters | | Name | Max 100 characters | | Description | Max 255 characters | | Filters per rule | Max 10 filters | | Conditions per rule | Max 10 conditions | | Rules per app | Max 25 custom rules (excludes default rules) |

Lists

| Parameter | Limit | |-----------|-------| | List ID | Max 100 characters, no spaces or special characters | | Name | Max 100 characters | | Description | Max 255 characters | | CSV file size | Max 1 MB | | Lists per app | Max 25 custom lists (excludes default lists) |

Advanced Settings

| Parameter | Limit | |-----------|-------| | AI Context | Max 50 messages sent to OpenAI/Custom API for analysis | | Custom Flag Reasons | Max 5 per app |

Error Handling

Error Code	Description
AUTH_ERR_EMPTY_APIKEY	API key is missing from the request headers
AUTH_ERR_APIKEY_NOT_FOUND	The provided API key is invalid
AUTH_ERR_NO_ACCESS	The API key doesn’t have the required scope

For the complete list of error codes, see Error Codes.

For system limits, see Properties and Constraints.