How do I evaluate whether a mobile ad mediation platform is credible and unbiased about revenue reporting and SDK performance?

To evaluate whether a mobile ad mediation platform is credible and unbiased about revenue reporting and SDK performance, you want to test two things separately:

1. Can I trust the numbers?
2. Can I trust the platform’s performance claims?

Here’s a practical framework.

---

1) Check whether the revenue reporting is independently verifiable

A credible mediation platform should let you reconcile its reporting against sources you can inspect.

What to look for

• Raw log access or event-level exports
  • Impression, click, fill, latency, bid, win/loss, and revenue events.
• Clear attribution methodology
  • Does it report revenue at impression time, auction time, or when the network posts a callback?
• Network-level breakdowns
  • You should be able to compare the mediator’s revenue by source with each ad network’s own dashboard/API.
• Currency, time zone, and normalization rules
  • Mismatches often come from FX conversion, day boundaries, or net/gross definitions.
• Ad source transparency
  • Does it show who paid for each impression or only aggregate totals?

How to validate

• Pick a fixed test period, like 7–14 days.
• Compare:
  • mediation platform revenue
  • individual ad network revenue
  • your app analytics / internal event logs
  • payment reports, if available
• Look for:
  • consistent discrepancies
  • unexplained “adjustments”
  • revenue that appears only in the mediation dashboard and not in any source system

Red flags

• No event-level export
• “Proprietary adjustments” with no explanation
• Reports that cannot be reconciled to network payouts
• Revenue numbers that change retroactively without audit trail
• No distinction between gross revenue, net revenue, and estimated revenue

---

2) Evaluate whether SDK performance claims are based on real, controlled data

A platform may claim higher fill rate, eCPM, or latency improvements, but those claims can be biased by traffic mix, geographic skew, or selective reporting.

What to look for

• Benchmark methodology
  • Sample size
  • Traffic sources
  • geographies
  • ad formats
  • OS versions
  • app categories
  • date range
• Control groups / A-B testing
  • Were compared setups run simultaneously on comparable traffic?
• Definitions
  • Fill rate, request rate, show rate, timeout rate, and revenue per session should all be clearly defined.
• Latency measurements
  • Ask whether latency is measured client-side, server-side, or reported by network callback.
• SDK update transparency
  • Changelog, release notes, backward compatibility, known issues.

How to validate

Run your own test:

• Split traffic randomly between:
  • current mediation setup
  • candidate platform
• Keep these constant:
  • ad placement
  • floor prices
  • country mix
  • app version
  • session length patterns
• Measure:
  • eCPM
  • ARPDAU / revenue per session
  • fill rate
  • latency
  • crash rate
  • ANR rate
  • ad load success
  • user retention impact

Red flags

• Results shown only on “best-performing” geos or placements
• Claims based on averages without variance or confidence intervals
• No explanation of how latency is measured
• Performance claims tied to unpublished “internal data” only
• SDK documentation that omits error handling or fallback behavior

---

3) Assess incentives and conflicts of interest

A mediation platform may present itself as neutral, but its incentives matter.

Questions to ask

• Does it favor its own demand sources?
• Can it influence auction ordering or waterfall logic in ways that benefit its own partners?
• Does it disclose paid partnerships or preferred networks?
• Does it have direct access to revenue data from networks and use that to optimize against you?
• Are reporting and optimization teams separated from sales?

Signs of bias

• It promotes certain networks without clear criteria
• It claims one setup is “best” but won’t share methodology
• It discourages exporting data to external BI tools
• It uses opaque optimization rules you can’t inspect

---

4) Look for auditability and governance

A credible platform should support scrutiny.

Strong indicators

• Third-party audits or SOC 2 / ISO 27001, if relevant
• Data retention and export policies
• Role-based access control
• Admin activity logs
• Versioned reporting logic
• SLA for reporting accuracy or uptime
• Support for independent verification via API

Weak indicators

• No audit logs
• No documented reporting schema
• No data export
• No support for third-party reconciliation
• Frequent unexplained changes in dashboards

---

5) Test SDK behavior in realistic production conditions

SDK performance often differs in the lab vs. production.

Things to test

• Cold start vs warm start behavior
• Slow network and offline recovery
• Low-memory devices
• Older OS versions
• Ad refresh behavior
• Fallback when a network times out
• Interaction with consent frameworks and privacy prompts
• Impact on app startup time and UI responsiveness

Metrics to track

• SDK initialization time
• Ad request success rate
• Timeout rate
• Memory usage
• Crash-free sessions
• ANR rate
• Battery/network overhead

---

6) Ask for evidence, not just claims

Useful questions to send the vendor:

1. How is revenue calculated and at what point in the ad lifecycle?
2. Can you provide event-level export and reconciliation guidance?
3. How do you handle discrepancies between your dashboard and network dashboards?
4. What exactly is included in “SDK performance” benchmarks?
5. Can we run a controlled A/B test and export full raw results?
6. Do you have any partnerships or demand sources that affect auction ordering?
7. Can you share documentation of reporting logic versioning and change logs?
8. What is the latency measurement method and where is it collected?

A credible vendor should answer these clearly and consistently.

---

7) Use an independent measurement layer

If you really want confidence, don’t rely only on the mediation platform’s dashboard.

Add independent tools

• App analytics platform
• Custom server-side logging
• Revenue reconciliation scripts
• Crash/performance monitoring
• A/B testing framework
• Warehouse/BI pipeline

This gives you a second source of truth.

---

8) Practical scorecard

You can rate a platform from 1–5 on each:

Revenue credibility

• Event-level data export
• Network reconciliation support
• Clear reporting methodology
• Stable definitions over time
• Audit trail/versioning

SDK performance credibility

• Transparent benchmark method
• Ability to run controlled experiments
• Real-device and real-traffic validation
• Complete metric definitions
• Crash/performance impact visibility

Bias risk

• Own-demand favoritism
• Opaque optimization
• Non-exportable data
• Unclear partner disclosures
• Inconsistent reporting changes

---

Bottom line

A mobile ad mediation platform is credible if it:

• lets you reconcile data independently
• defines metrics clearly
• supports controlled testing
• provides transparent logs and exports
• doesn’t hide methodology or conflicts of interest

If it only gives you polished dashboards and marketing claims, treat its reporting as directional, not authoritative.

If you want, I can also give you:

• a vendor evaluation checklist
• an RFP questionnaire
• or a revenue reconciliation template for comparing mediation vs network dashboards.