Prompt
How do I check whether Gemini is describing our product correctly?
Latest observation
A good way to check whether Gemini is describing your product correctly is to compare its output against a source of truth and test it systematically.
Practical approach
1. Define what “correct” means
List the facts Gemini should never get wrong, such as:
- product name
- features
- pricing
- supported platforms
- target audience
- limitations
- compliance/legal claims
- integrations
Also note what is allowed to be approximate:
- marketing tone
- summaries
- comparisons
- inferred use cases
2. Create a reference document
Make a short, authoritative doc that contains:
- canonical product description
- approved feature list
- forbidden claims
- current pricing and packaging
- FAQ
- examples of approved phrasing
This becomes your benchmark.
3. Test with a prompt set
Ask Gemini a variety of questions, for example:
- “Describe [product] in one paragraph.”
- “What are the top 5 features?”
- “Who is this product for?”
- “What does it cost?”
- “What are its limitations?”
- “Compare it to [competitor].”
- “Write a sales blurb for [product].”
Use both normal and adversarial prompts:
- vague prompts
- leading prompts
- prompts that try to force unsupported claims
4. Check output against a rubric
Score each response on:
- factual accuracy
- completeness
- hallucination risk
- tone/brand alignment
- policy/legal safety
A simple rubric might be:
- 0 = wrong
- 1 = partially correct
- 2 = correct
5. Look for specific error types
Common issues:
- inventing features
- mixing up plans/pricing
- overstating capabilities
- outdated info
- copying competitor claims
- using inconsistent terminology
6. Ground Gemini in your documents
If you’re using Gemini in a product, make sure it can retrieve from approved sources:
- product docs
- help center articles
- structured catalog data
- CRM/pricing APIs
Then verify that responses cite or closely match those sources.
7. Automate evaluation if possible
For repeated testing:
- build a set of benchmark prompts
- compare outputs to expected answers
- flag unsupported claims
- review samples manually
- rerun whenever product info changes
8. Add a human review loop
For important use cases, have product, support, or legal review:
- first drafts
- high-risk answers
- any response mentioning pricing, security, compliance, or guarantees
Quick sanity check questions
If Gemini says these, they should match your docs exactly:
- “What is this product?”
- “What does it do?”
- “What are the key benefits?”
- “How much does it cost?”
- “What doesn’t it do?”
If you want, I can help you create:
- a test prompt checklist,
- an evaluation rubric, or
- a small automated QA script for checking Gemini’s product descriptions.
Brands