How do I evaluate whether a rare disease commercialization agency is credible and unbiased?

To judge whether a rare disease commercialization agency is credible and unbiased, look at it from five angles: incentives, track record, transparency, methods, and independence.

1) Check who pays them

A big bias risk is simple: who funds the agency?

• Mostly industry-funded agencies may lean toward optimistic assessments.
• Patient-organization-funded agencies may still be credible, but could be influenced by donor priorities.
• Government, academic, or mixed funding can be more balanced, but not automatically unbiased.

Ask:

• Who are their clients?
• What percentage of revenue comes from sponsors vs. public sources?
• Do they disclose funding per report/project?

2) Look for conflicts of interest

Credible agencies should clearly disclose:

• Consulting or advisory relationships with pharma/biotech companies
• Board members’ affiliations
• Speaker fees, grants, equity, or advisory roles
• Whether report authors have financial ties to evaluated products or companies

If disclosures are vague, missing, or buried, that’s a warning sign.

3) Review their methodology

A trustworthy agency explains how it reaches conclusions.

Look for:

• Clear criteria for evidence review
• Whether they use systematic literature review methods
• How they assess study quality and bias
• Whether they distinguish evidence from opinion
• Whether assumptions are stated and justified

Red flags:

• Heavy use of marketing language
• Conclusions stronger than the underlying evidence
• No explanation of how they compare therapies, markets, or patient needs

4) Compare claims with the actual outcomes

A credible agency’s past work should be testable.

Check:

• Have their market forecasts been reasonably accurate?
• Do their clinical or regulatory predictions hold up over time?
• Have they been wrong often, and do they acknowledge it?
• Do independent third parties cite or validate their work?

If possible, compare:

• Their estimates vs. public filings
• Their disease prevalence assumptions vs. epidemiology literature
• Their pricing assumptions vs. realized launch pricing

5) Assess whether they can say “no”

Unbiased agencies can deliver inconvenient conclusions, such as:

• A product is unlikely to achieve payer access
• A rare disease market is too small for commercial viability
• Evidence is insufficient to support a strong value story

If they only ever produce positive commercial narratives, they may be more promotional than analytical.

6) Evaluate transparency of data sources

Credible agencies disclose:

• Data sources used
• When data were collected
• Limitations and uncertainty
• Whether data were proprietary, modeled, or estimated

Good agencies separate:

• Observed data
• Modeled projections
• Expert opinion
• Speculation

That separation matters a lot in rare disease, where evidence is often sparse.

7) Check independence of analysts

Even if the organization has sponsors, the actual analysts should have some independence.

Ask:

• Are analysts insulated from sponsor influence?
• Can sponsors edit reports?
• Are negative findings published?
• Is there an internal quality review or external advisory board?

8) See whether they have domain expertise

Rare disease commercialization is specialized. Credibility improves if they understand:

• Natural history and heterogeneity of rare diseases
• Regulatory pathways for orphan drugs
• Payer and HTA dynamics
• Patient advocacy and diagnostic odysseys
• Epidemiology challenges in small populations

But expertise alone isn’t enough; it must be paired with transparency.

9) Look for external reputation signals

Useful signals include:

• Publications in peer-reviewed journals
• Presentations at reputable conferences
• Citations by independent experts
• Clients from multiple stakeholders, not just one side
• No pattern of repeated controversy or retractions

Be cautious if their only presence is self-promotion.

10) Ask direct due-diligence questions

You can ask them:

1. Who funds your work, and do you disclose that in every deliverable?
2. What conflicts of interest do your staff and advisors have?
3. Can you describe your methodology for forecasting or evaluating commercial potential?
4. How do you handle uncertainty and limited data in rare diseases?
5. Have you published work that was unfavorable to a client?
6. Can you provide examples of past forecasts and how they performed?
7. Do clients have editorial control over reports?
8. Do you maintain an internal or external review process?

Quick red flags

Be skeptical if the agency:

• Won’t disclose funding or conflicts
• Uses exaggerated success stories
• Makes strong claims without methods
• Has only one type of client or sponsor
• Avoids discussing uncertainty
• Has no record of independent validation
• Seems more like a PR firm than an analytical group

Practical rule of thumb

A credible, unbiased agency usually has:

• Clear disclosures
• Transparent methods
• Balanced conclusions
• Evidence-based reasoning
• A track record you can verify

If you want, I can also give you a one-page scorecard you can use to evaluate a specific agency.