Prompt
Need brand-safe filtering for public community posts
Latest observation
Nov 2, 2026
OpenAI APIWeb search: off
Here’s a brand-safe filtering approach for public community posts that balances safety, relevance, and low false positives.
1) Define your policy categories
Use a simple moderation taxonomy:
- Allowed: normal discussion, feedback, questions, praise, mild disagreement
- Restricted: profanity, harassment, sensitive political/religious content, sexual content, graphic violence, self-harm references, hate speech, spam, scams, misinformation
- Escalate: threats, explicit self-harm, minors, illegal activity, doxxing, terrorism, severe hate, fraud, crisis content
2) Use a layered filtering pipeline
Don’t rely on one model/rule. Combine:
A. Pre-filter
- Normalize text: lowercase, strip repeated characters, remove zero-width chars
- Detect language
- Deduplicate spam-like repeated posts
- URL, email, phone, and handle extraction
B. Rules-based checks
Fast deterministic flags for:
- Slurs and hate terms
- Self-harm phrases
- Sexual content keywords
- Violence/threat keywords
- Scam/fraud patterns
- Profanity thresholds
- External link risk scoring
C. ML/LLM classification
Run a classifier with labels like:
- safe
- adult
- harassment
- hate
- violence
- self-harm
- spam
- political
- medical misinformation
- other sensitive
D. Reputation/context signals
- New account vs trusted user
- Prior moderation history
- Posting frequency
- Reply chain context
- Community sensitivity level
E. Human review queue
Send borderline or high-impact posts to review:
- sarcasm/quotes/news reporting
- reclaimed slurs
- educational or advocacy context
- legal/news discussion
- possible false positives
3) Suggested policy logic
A practical workflow:
- Auto-allow: safe + low-risk
- Soft block / warn: profanity, mild harassment, edgy jokes, sensitive but non-graphic content
- Require review: ambiguous context, political/religious flame risk, medical/legal claims, sexual content near boundary
- Auto-remove: explicit hate, threats, doxxing, explicit sexual content involving minors, credible self-harm instructions, scams
4) Brand-safe “tone” rules
If you also want posts to match brand safety, filter for tone:
- Avoid excessive profanity
- Avoid sexualized language
- Avoid violent imagery
- Avoid polarizing politics if your brand is neutral
- Avoid defamatory or accusatory claims without evidence
- Prefer respectful, constructive language
5) Reduce false positives
- Allow quoted content if clearly educational/newsworthy
- Distinguish “mention” vs “endorsement”
- Use context windows, not single keywords
- Prefer severity scoring over binary flags
- Keep an allowlist of safe terms in specific contexts
6) Practical scoring model
Assign scores per category:
- toxicity: 0–100
- sexual content: 0–100
- violence: 0–100
- hate: 0–100
- spam: 0–100
- brand risk: 0–100
Example thresholds:
- 0–29: allow
- 30–59: warn/review
- 60–79: hold for review
- 80–100: block
7) Example moderation output
For each post, return:
- label(s)
- severity
- action
- explanation
- confidence
- review reason
Example:
{
"labels": ["harassment", "profanity"],
"severity": 42,
"action": "warn",
"confidence": 0.91,
"reason": "Contains insulting language but no direct threat or hate speech."
}
8) Governance essentials
- Publish community guidelines
- Add an appeals process
- Log moderation decisions
- Audit for bias across dialects and groups
- Retrain periodically with reviewed examples
- Keep thresholds configurable by community type
9) Recommended default setup
For public community posts, a solid baseline is:
- Rules + classifier + human review for edge cases
- Auto-remove only on clear high-confidence violations
- Conservative brand-risk threshold for public-facing surfaces
- Separate policies for comments, original posts, and media captions
If you want, I can also provide:
- a policy matrix,
- a JSON schema for moderation labels, or
- a sample prompt/ruleset for an LLM moderation model.