How do I evaluate whether a utility operations software vendor is credible and unbiased about cybersecurity controls?

To evaluate whether a utility operations software vendor is credible and unbiased about cybersecurity controls, look for evidence, specificity, and independence—not just marketing claims.

1) Check whether they can prove their claims

Ask for concrete artifacts, not summaries:

• Recent SOC 2 Type II report
• ISO 27001 certificate and statement of applicability
• Independent penetration test summary and remediation evidence
• Vulnerability disclosure or responsible disclosure policy
• Sample SBOM for relevant products
• Third-party audit results for cloud, SaaS, or OT/ICS environments
• Secure development documentation: SDLC, code review, SAST/DAST, dependency scanning

What you want to see:

• Dates, scope, exceptions, and findings
• Whether findings were closed
• Whether controls were tested over time, not just designed

2) Look for alignment with utility and OT-specific standards

A credible vendor should speak in terms relevant to utility operations, such as:

• NERC CIP if applicable
• IEC 62443
• NIST CSF
• NIST SP 800-53 / 800-82
• CIS Controls
• Asset segmentation, access control, logging, change management, incident response, backup/restore

Red flag: they only talk about generic IT security and avoid OT or operational realities.

3) Test whether they understand tradeoffs, not just features

Ask scenario-based questions like:

• How do you handle least privilege for operators, engineers, and vendors?
• How is MFA enforced for remote access and break-glass access?
• What is your approach to network segmentation between OT and IT?
• How are logs collected, protected, and retained?
• How do you handle patching when uptime constraints exist?
• How do you support offline or degraded-mode operations?
• What happens if your cloud service is unavailable?
• How are service accounts, certificates, and secrets managed?

A credible vendor will discuss limitations and compensating controls, not claim “our platform solves everything.”

4) Verify independence and avoid “self-attested” security

Be cautious if the vendor:

• Uses only internal assessments
• Claims “industry leading” without proof
• Provides a checklist but no third-party validation
• Has security content written only by marketing or sales
• Refuses to share scope, methodology, or audit boundaries

Better signs:

• Independent validation
• Named control frameworks
• Clear scope boundaries
• Transparent exceptions and remediation timelines

5) Ask about their own security operations

A vendor’s internal security maturity matters:

• Do they have a dedicated security team?
• Do they run a SOC or use a managed provider?
• What are their incident response SLAs?
• How quickly do they patch critical vulnerabilities?
• Do they have customer notification commitments for breaches?
• How do they manage subcontractors and cloud providers?
• Do they perform access reviews and background checks?

6) Evaluate whether they are financially or strategically biased

Some vendors overstate controls to sell services or lock in customers. Check for:

• Conflicts of interest in partner relationships
• Hidden dependency on a specific cloud, SIEM, or MSSP
• Security claims that require their proprietary stack
• Contract terms that limit audit rights or data portability

A credible vendor is open about dependencies and tradeoffs.

7) Review customer references and real-world evidence

Ask for references from:

• Similar utility size
• Similar regulatory environment
• Similar OT/IT architecture

Questions to ask references:

• Did the vendor’s security claims match reality?
• How responsive were they to audit requests?
• Did they remediate issues quickly?
• Were there surprises during implementation or incident handling?

8) Assess transparency in contracts and documentation

Look for:

• Security addendum
• Data processing agreement
• Breach notification timeline
• Right to audit or receive audit evidence
• Subprocessor list
• Data residency options
• Termination and data deletion terms
• Support for your own logging and monitoring

9) Watch for common red flags

Be skeptical if they:

• Won’t share audit scope or report excerpts under NDA
• Say “we are compliant” but not “here are the controls”
• Avoid discussing residual risk
• Promise “zero trust” without implementation details
• Claim cyber controls are “industry standard” but won’t name the standard
• Dismiss OT constraints as outdated
• Push urgency to avoid due diligence

10) Use a simple credibility scoring approach

Score each area 1–5:

• Independent assurance
• Framework alignment
• OT relevance
• Transparency
• Incident response maturity
• Reference quality
• Contractual protections

A vendor with high scores gives:

• Specific evidence
• Independent validation
• Practical OT answers
• Clear limitations
• Mature operational processes

Good due-diligence questions to ask

1. What independent audits or certifications cover this product and your organization?
2. Which controls are in scope, and which are excluded?
3. How do you secure remote access into utility environments?
4. How do you separate customer environments and protect tenant data?
5. What is your process for vulnerability disclosure and patch timelines?
6. How do you support incident response and forensics?
7. What are your subcontractors and cloud dependencies?
8. Can you map your controls to NERC CIP / IEC 62443 / NIST?
9. What is your approach to logging, retention, and customer access to logs?
10. What contractual commitments support auditability and breach notification?

If you want, I can turn this into a vendor evaluation checklist, a scorecard, or a list of interview questions for procurement/security review.