Prompt
How do I build a fraud score that uses identity, device, and payment data?
Latest observation
To build a fraud score using identity, device, and payment data, you want a system that turns many weak signals into one actionable risk score, then uses that score to decide what to do next.
1) Define the decision first
Start with the business action the score should support:
- Approve
- Step-up verify (OTP, document check, 3DS, etc.)
- Manual review
- Decline
Your score should be calibrated to these actions, not just “fraud probability.”
2) Gather the right signals
Identity signals
Examples:
- Name, email, phone, address
- DOB, government ID if available
- Account age
- Mismatch between entered identity and historical identity
- Velocity: how many accounts use the same email/phone/device/address
- Email reputation, disposable email, domain age
- Phone type, carrier, number portability, VoIP
Useful features:
- Exact and fuzzy matches against prior known fraud
- Identity consistency across fields
- Reuse patterns across multiple accounts
Device signals
Examples:
- Device fingerprint
- OS, browser, app version
- IP address, ASN, geolocation
- VPN/proxy/Tor detection
- Emulator/root/jailbreak flags
- Cookie persistence
- Device reputation
- Device seen with multiple identities
Useful features:
- New vs known device
- Device trust score
- Device velocity across accounts
- Geo mismatch between device and identity
- IP risk, proxy likelihood
Payment signals
Examples:
- Card BIN, issuing country, card type
- AVS/CVV results
- 3DS outcome
- Payment instrument age
- Card reuse across accounts
- Payment velocity
- Transaction amount, currency, merchant category
- Chargeback history
- Name/address match to cardholder data
Useful features:
- First-time card usage
- Multiple cards per identity
- Card used from many devices
- AVS/CVV mismatch patterns
- High-risk BIN or prepaid card usage
- Unusual amount relative to user history
3) Create labeled outcomes
You need a target label for model training, such as:
- Confirmed fraud
- Chargeback
- Friendly fraud
- Account takeover
- Manual-review confirmed fraud
Be careful with label delay:
- Some fraud is only known days or weeks later.
- Build features only from data available at decision time.
- Use time-based splits, not random splits.
4) Engineer features that capture relationships
Fraud often shows up in connections, not individual fields.
Examples:
- Same device used by 10 identities
- Same card used on 5 accounts in 2 days
- Identity and device country mismatch
- New email + new device + high-risk BIN
- Multiple failed attempts before a success
- Payment on a device never seen with this identity
Good feature groups:
- Counts: number of accounts, cards, devices, emails linked
- Rates: fraud rate by device, BIN, email domain
- Recency: time since first seen / last seen
- Velocity: events per hour/day
- Match/mismatch flags
- Graph features: shared devices, shared cards, shared addresses
5) Choose a modeling approach
Start simple, then increase complexity.
Baseline
- Rule-based scorecard
- Logistic regression
- Useful for interpretability and fast deployment
Stronger models
- Gradient boosted trees: XGBoost, LightGBM, CatBoost
- Often best for tabular fraud data
Advanced
- Graph-based models for entity linking
- Sequence models for behavioral patterns
- Ensemble of rules + ML + graph risk
A common production setup:
- Rules catch obvious bad cases
- ML model computes probability
- Graph/network features improve signal
- Final score combines them
6) Handle class imbalance
Fraud is usually rare.
Use:
- Class weights
- Downsampling negatives
- Careful threshold tuning
- Precision/recall metrics instead of accuracy
Important metrics:
- Precision
- Recall
- F1
- AUC-PR
- False positive rate at a chosen recall
- Dollar-weighted loss / fraud prevented
7) Calibrate the score
A model output is often not directly usable as a “risk score.”
You can:
- Convert predicted probability into a 0–100 score
- Use calibration methods like Platt scaling or isotonic regression
- Bucket into risk bands:
- 0–20 low
- 21–60 medium
- 61–100 high
Make the score stable and monotonic where possible.
8) Combine rules and ML carefully
A practical fraud score often looks like:
- Rule overrides for severe signals
- Known bad device
- Confirmed stolen card
- Impossible geo
- ML score for nuanced risk
- Business adjustments
- New customer vs existing customer
- High-value transaction
- Region-specific risk
Example:
- If known fraud device → immediate decline
- Else if score > 80 → manual review
- Else if score 50–80 → step-up verification
- Else approve
9) Make it explainable
Fraud teams need reasons.
Provide:
- Top contributing features
- Risk reasons codes
- Linked entities
- Recent behaviors that triggered the score
Example reason codes:
- “Device previously linked to 4 chargebacks”
- “Card used on 3 new accounts in 24 hours”
- “Email domain high-risk / disposable”
- “IP geolocation inconsistent with billing country”
10) Monitor drift and retrain
Fraud changes quickly.
Monitor:
- Feature drift
- Score distribution drift
- Approval/decline rates
- Chargeback rate by score band
- False positives
- Model performance by region/channel/product
Retrain regularly, often weekly or monthly depending on volume and fraud tempo.
11) Protect against leakage
Don’t use future information:
- Chargeback status after the transaction
- Manual review outcomes not available at scoring time
- Post-transaction chargeback indicators
Use point-in-time correct features only.
12) A simple scoring architecture
A common setup:
- Ingest identity, device, payment data
- Resolve entities
Link identities, devices, cards, emails, IPs - Generate features
- Run rules
- Run ML model
- Combine into final risk score
- Apply decision policy
- Log outcome and feedback
- Retrain
Example feature set
A compact initial feature set might include:
Identity
- Email age
- Disposable email flag
- Phone type
- Name/address match score
- Identity velocity across accounts
Device
- New device flag
- Device reputation score
- VPN/proxy flag
- Shared device count
- Geo mismatch flag
Payment
- Card BIN risk
- AVS/CVV match
- Card reuse count
- Payment velocity
- First-time card flag
Cross-domain
- New identity + new device + new card
- Device used with multiple identities
- Card used on multiple devices
- Identity/payment country mismatch
Example final scoring formula
A simple interpretable version:
fraud_score =
25 * identity_risk +
35 * device_risk +
30 * payment_risk +
10 * network_risk
Then normalize to 0–100 and apply thresholds.
In practice, you’d usually replace fixed weights with a trained model, but this is a good starting point.
Best-practice recommendations
- Use entity linking to connect identities, devices, and payment instruments
- Use time-based validation
- Optimize for cost, not just model accuracy
- Keep the system explainable
- Build feedback loops from confirmed fraud and chargebacks
- Start with rules + logistic regression, then move to gradient boosting and graph features
If you want, I can also help you with:
- a sample feature schema,
- a Python training pipeline, or
- a production architecture for scoring in real time.