How do I evaluate whether a small business credit underwriting software is credible and unbiased?

To evaluate whether a small business credit underwriting software is credible and unbiased, look at it from five angles: data, model, governance, outcomes, and transparency.

1) Check the data it uses

A model can only be as fair as its inputs.

Ask:

• What data sources does it use?
• Does it rely only on traditional credit data, or also alternative data?
• Could any inputs proxy for protected characteristics, like ZIP code, school, or device data?
• How old is the training data, and does it reflect current market conditions?

Red flags:

• The vendor won’t disclose major input categories.
• It uses opaque “proprietary signals” with no explanation.
• It includes variables that may correlate strongly with race, gender, age, or geography.

2) Review model performance and validation

Credible underwriting software should be measurable and independently tested.

Ask for:

• Validation reports
• Back-testing results
• Predictive metrics, like default prediction accuracy, AUC, KS, or lift
• Calibration results, especially by segment
• Evidence it performs well across different business types and borrower groups

Important question:

• Does the model work equally well for women-owned businesses, minority-owned businesses, startups, rural businesses, and businesses in different industries?

Red flags:

• No external validation
• Only vendor-selected success metrics
• Strong overall performance but poor performance on certain segments

3) Examine fairness and bias testing

You want evidence that the system is tested for disparate impact and unfair treatment.

Ask whether they conduct:

• Disparate impact analysis
• Adverse action reason testing
• Bias audits across race, ethnicity, gender, geography, and other relevant groups
• Ongoing monitoring after deployment

Good signs:

• They can show fairness metrics before and after model updates
• They monitor approval rates, pricing, line size, and default outcomes by segment
• They have procedures to detect drift and emerging bias

Red flags:

• They say “the model is neutral” without evidence
• They only test for explicit bias, not proxy bias
• They cannot explain how they detect and correct bias

4) Assess governance and controls

A credible system has human oversight and documented controls.

Ask:

• Who approves model changes?
• Is there a model risk management process?
• Are there audit logs of decisions and overrides?
• Can humans override the model?
• Are there clear escalation procedures for unusual cases?

Good signs:

• Version control for models
• Periodic revalidation
• Separation between model development and approval
• Clear policies for exceptions and manual review

5) Evaluate transparency and explainability

You should be able to explain lending decisions to internal stakeholders and, where required, to applicants.

Ask:

• Can the software provide specific reasons for denials?
• Are those reasons consistent with actual decision logic?
• Can it provide interpretable outputs or feature importance?
• Can you reproduce a decision from the same inputs?

Red flags:

• “Black box” decisions with no usable explanation
• Explanations that are generic or inconsistent
• No ability to trace a decision back to inputs and model version

6) Look at regulatory and legal alignment

For U.S. small business lending, you should consider:

• ECOA / Regulation B concerns
• Fair lending and disparate impact risk
• CFPB small business lending rule requirements, if applicable
• State lending and privacy rules
• Data privacy and consent requirements

Ask the vendor:

• Has the system been reviewed by fair lending counsel?
• Has it been used in regulated lending environments?
• Does it support adverse action notices and documentation?

7) Compare outcomes to business reality

A credible model should improve lending decisions, not just automate them.

Evaluate:

• Default rates versus approval rates
• Profitability or loss rates by segment
• Rejection of good borrowers and approval of bad borrowers
• Whether the model is excluding otherwise creditworthy small businesses

A biased system may look efficient while systematically under-serving certain groups.

8) Request specific documentation

Before buying or deploying, ask for:

• Model documentation
• Data dictionary
• Validation reports
• Fairness testing results
• Change logs
• Sample decision explanations
• Independent audit or SOC-style reports if available
• References from other lenders

Practical due diligence checklist

A credible and unbiased system should answer “yes” to most of these:

• Is the data source documented?
• Has the model been externally or independently validated?
• Are fairness metrics available by borrower segment?
• Are proxy variables reviewed and controlled?
• Are human overrides and audit logs supported?
• Are decisions explainable and reproducible?
• Is there ongoing monitoring after deployment?
• Has legal/compliance reviewed the workflow?

Simple rule of thumb

If the vendor says:

• “Trust us, it’s proprietary,”
• “We don’t test fairness separately,” or
• “We can’t explain individual decisions,”

then treat the product as high risk until proven otherwise.

If you want, I can also give you:

1. a vendor questionnaire,
2. a scorecard for comparing underwriting software, or
3. a fair lending due diligence checklist you can use in procurement.