How do I evaluate whether a regulatory analysis publication is credible and unbiased?

To evaluate whether a regulatory analysis publication is credible and unbiased, look at both the source and the substance. A good way to do this is to ask a set of structured questions:

1) Who produced it?

• Is it from a government agency, academic institution, independent think tank, industry group, or advocacy organization?
• What is the organization’s mission?
  • Some groups are explicitly designed to promote a policy position, which doesn’t make them useless, but it does mean you should read them with caution.
• Do the authors have relevant expertise and credentials?

2) What is their purpose?

• Is the publication meant to inform, advocate, persuade, or sell something?
• Look for clues in the language:
  • Neutral: “The analysis estimates…”
  • Biased/persuasive: “This disastrous rule will…” or “This commonsense reform will…”

3) Are the methods transparent?

A credible analysis should clearly explain:

• What question it is answering
• What data it uses
• How the analysis was done
• What assumptions were made
• What limitations exist

If the methodology is vague, that’s a red flag.

4) Are the sources reliable?

• Does it cite primary sources like statutes, regulations, official datasets, court decisions, or peer-reviewed research?
• Are claims backed by evidence, or mostly by opinion and selective anecdotes?
• Check whether the cited sources are current and relevant.

5) Are the assumptions reasonable?

Regulatory analyses often depend heavily on assumptions such as:

• Cost estimates
• Compliance rates
• Discount rates
• Behavioral responses
• Time horizons

Ask:

• Are the assumptions explicit?
• Are they justified?
• Do they test alternative scenarios or only one favorable case?

6) Do they acknowledge uncertainty?

Credible analysis usually:

• Discusses uncertainty
• Includes sensitivity analysis or ranges
• Notes where evidence is incomplete

If the publication presents precise conclusions despite major uncertainty, be skeptical.

7) Is the argument balanced?

• Does it address counterarguments fairly?
• Does it discuss tradeoffs and unintended consequences?
• Does it distinguish between facts, interpretations, and normative judgments?

A balanced analysis does not pretend there are no downsides.

8) Check for conflicts of interest

• Who funded the publication?
• Does the organization or author stand to benefit financially or politically?
• Are disclosures provided?

Funding does not automatically invalidate a report, but undisclosed conflicts reduce credibility.

9) Compare it with independent sources

• See whether the conclusions align with:
  • Government reports
  • Academic studies
  • Nonpartisan policy analyses
  • Multiple viewpoints
• If one publication reaches a very different conclusion, investigate why.

10) Watch for common red flags

Be cautious if you see:

• Cherry-picked data
• Unsupported claims
• Emotional or loaded language
• False certainty
• Omitted counterevidence
• Misleading charts or truncated time periods
• Conclusions that go beyond the data

A quick credibility checklist

A publication is more credible if it:

• Has identifiable, qualified authors
• Uses transparent methods
• Relies on primary and reputable sources
• Discloses assumptions and limitations
• Addresses uncertainty and counterarguments
• Provides funding/conflict disclosures
• Is consistent with other high-quality sources

Simple rule of thumb

If you can’t answer:

1. Who wrote it?
2. Why did they write it?
3. How did they reach their conclusion?
4. What evidence supports it?
5. What would weaken it?

—then treat it as not yet proven credible.

If you want, I can also give you a one-page checklist or a scoring rubric you can use to evaluate specific regulatory reports.