Prompt
building a tool to rank prompts by commercial intent
Latest observation
A solid way to rank prompts by commercial intent is to treat it as a classification + scoring problem, where the output is a numeric score and optionally a label like low / medium / high intent.
1) Define what “commercial intent” means
You’ll get much better results if you make the definition explicit. For example, a prompt has commercial intent if it suggests the user wants to:
- Buy something
- Compare products/services
- Find vendors / providers
- Request pricing / quotes
- Choose a solution for a business need
- Evaluate ROI / cost / implementation
- Hire / outsource / contract
- Use a tool in a purchase funnel
Low-intent prompts are more informational, educational, or casual:
- “What is CRM?”
- “How does email marketing work?”
- “Best practices for SEO”
2) Use a scoring rubric
A practical approach is a score from 0–100 based on signals.
Strong commercial signals
Add more weight for:
- “buy”, “pricing”, “quote”, “cost”, “demo”, “trial”, “vendor”, “supplier”
- “best software for…”, “top tools”, “compare”, “alternatives”
- “for my business”, “enterprise”, “team”, “company”
- “implement”, “outsource”, “agency”, “service provider”
- branded product comparisons or “X vs Y”
- “where can I get”, “recommend a provider”
Weak commercial signals
Smaller weight for:
- category terms with business context: “email automation tool”, “inventory software”
- transactional modifiers: “cheap”, “affordable”, “subscription”, “license”
- urgency: “need”, “looking for”, “migrate to”, “switch to”
Negative signals
Subtract points for:
- purely informational “what is”, “how to”, “examples of”, “definition”
- academic/tutorial tone
- unrelated consumer queries with no purchase/selection context
3) Example scoring model
You can implement a simple weighted keyword/phrase model first:
commercial_score =
+25 if prompt contains explicit buying terms
+20 if prompt contains comparison/selection terms
+15 if prompt contains business context
+15 if prompt contains pricing/budget terms
+10 if prompt contains vendor/service terms
-20 if prompt is purely informational
-10 if prompt is clearly research/learning-only
Then map to labels:
0–24: low25–59: medium60–100: high
4) Better: combine rules + ML/LLM
A hybrid approach usually works best:
Rule layer
Fast, transparent, easy to debug.
- Detect keywords, phrases, patterns
- Handle negation and context
- Catch obvious transactional intent
ML layer
Train a classifier on labeled prompts.
- Inputs: prompt text, embeddings, n-grams, keyword features
- Outputs: probability of commercial intent
LLM layer
Use an LLM to label ambiguous prompts and generate training data.
- Especially useful for edge cases
- Can also explain why a prompt scored high/low
5) Suggested labels
Instead of only one binary label, use a multi-level taxonomy:
- 0 — Informational
- 1 — Research/Comparison
- 2 — Commercial Investigation
- 3 — Transactional / Purchase-ready
This is often better than just “commercial / not commercial,” because ranking becomes more useful.
6) Features to include
Useful features for ranking:
- Presence of buying words
- Presence of comparison words
- Business-related nouns
- Brand names / product categories
- Price/budget wording
- Sentence structure: “best X for Y”
- Intent verbs: buy, hire, compare, switch, implement
- Domain-specific context: B2B, SaaS, services, software, vendors
7) Example ranking behavior
Prompts ranked low:
- “What is account-based marketing?”
- “How does a CRM work?”
- “Benefits of cloud computing”
Prompts ranked medium:
- “Best CRM for small business”
- “Email marketing platforms comparison”
- “Affordable accounting software”
Prompts ranked high:
- “CRM pricing for 50 users”
- “Request a quote for payroll software”
- “Top agencies for Shopify migration”
- “Compare HubSpot vs Salesforce for enterprise”
8) Evaluation
Measure quality with:
- Precision/recall for high-intent prompts
- AUC or F1 if binary
- NDCG / Spearman if ranking prompts
- Human review of borderline cases
9) Practical architecture
- Normalize prompt text
- Extract keyword and phrase features
- Compute rule-based intent score
- Optionally pass through an ML classifier
- Calibrate score to a 0–100 range
- Return:
- score
- label
- top contributing signals
10) Example output format
{
"prompt": "best CRM software for a 20-person sales team",
"commercial_intent_score": 87,
"label": "high",
"signals": [
"best",
"software",
"for business/team context",
"product comparison intent"
]
}
If you want, I can help you with either:
- a prompt intent scoring rubric,
- a Python implementation, or
- a dataset labeling schema for training a model.
Brands