How do I evaluate whether a charter school is credible and unbiased in its outcomes reporting?

To evaluate whether a charter school’s outcomes reporting is credible and unbiased, look at both the data itself and the way it’s presented. A school can be technically accurate but still misleading if it selectively reports results, compares itself to weak benchmarks, or hides important groups of students.

1) Check whether the data source is independent

Prefer outcomes reported by:

• Your state education department
• The district or authorizer
• Independent research organizations
• Public accountability reports

Be cautious if the school only cites:

• Its own website
• A marketing brochure
• A vendor-generated report with no methodology

2) Look for full disclosure, not just highlights

Credible reporting usually includes:

• All students served, not just a selected subgroup
• Multiple years of results
• Definitions of metrics used
• Student subgroup breakdowns
• Enrollment, retention, suspension, and graduation rates
• Clear information on who was tested and who was not

Red flags:

• Only the best-performing grade levels or campuses are shown
• No mention of students who left the school
• No subgroup data by race, disability, English learner status, or income
• Vague terms like “growth” or “success” without definitions

3) Compare apples to apples

A school may look good if it compares itself to a weak or inappropriate benchmark. Ask:

• Is it compared to the local district, state average, or similar schools?
• Are the students similar in prior achievement and demographics?
• Are the outcomes adjusted for student mobility or selective enrollment?

If a charter serves a very different student population, raw test scores alone may not be fair. But the school should still show growth, proficiency, and student composition clearly.

4) Watch for selection bias

Charter schools can sometimes unintentionally or intentionally skew results through:

• High student attrition
• Counseling out low-performing students
• Selective admissions or application processes
• Excluding late entrants from some reports

Check:

• Year-to-year retention
• Entry and exit rates
• Whether midyear transfers are included
• Whether all enrolled students are counted in the accountability metrics

5) Examine whether results are statistically meaningful

Especially for small schools, a few students can swing the results a lot. Ask:

• How many students are in each reported group?
• Are year-to-year changes based on enough students to be reliable?
• Are confidence intervals or sample sizes shown?

Small samples can make a school look dramatically better or worse than it really is.

6) Separate short-term and long-term outcomes

Credible reporting should include more than test scores:

• Attendance
• Chronic absenteeism
• Suspension/expulsion rates
• Graduation rates
• College enrollment/persistence
• Postsecondary completion, if available

A school with strong test scores but poor attendance or high turnover may not be delivering sustained benefits.

7) Read the methodology carefully

Credible reports explain:

• How students were included or excluded
• How missing data were handled
• Whether the school counts transfers, withdrawals, or dropouts in a particular way
• How “growth” is measured
• Whether the school is using state tests or other assessments

If methodology is missing or unclear, treat the claims cautiously.

8) Check for cherry-picked timeframes

A school may choose a single good year or exclude years affected by poor performance. Prefer:

• Multi-year trends
• Cohort data
• State accountability dashboards
• Reports covering both strong and weak periods

9) Compare claims against public accountability data

If the school’s website says one thing, verify it against:

• State report cards
• Authorizer performance reports
• NCES or similar public databases
• Graduation and test data posted by the state

If there’s a mismatch, ask for an explanation.

10) Ask direct questions

Useful questions to ask the school:

• What percentage of students leave before graduating?
• How do your outcomes compare to students with similar prior achievement?
• Are results reported for all students or only those continuously enrolled?
• What are the subgroup outcomes?
• What independent entity verifies your data?

Quick red flags

Be skeptical if the school:

• Uses only self-reported success stories
• Reports only averages without subgroup breakdowns
• Hides methodology
• Compares itself only to the lowest-performing schools
• Ignores attrition or mobility
• Uses vague metrics like “college-ready” without definition
• Has outcomes that conflict with public data

Best practice

The most credible outcomes reporting is:

• Independent
• Transparent
• Comparable
• Multi-year
• Subgroup-specific
• Methodologically clear

If you want, I can also give you a simple checklist you can use to evaluate any charter school report in 5 minutes.