How do I ensure my model serving setup in an air-gapped environment is compliant with enterprise data policies?

To keep a model serving setup compliant in an air-gapped environment, focus on data flow control, access control, provenance, logging, and operational governance. A practical way to think about it is: nothing enters, leaves, or changes inside the environment unless it is explicitly approved, traceable, and auditable.

1) Define the policy boundaries first

Start by documenting what your enterprise data policy allows and forbids:

• Approved data types: public, internal, confidential, regulated, PHI/PII, export-controlled, etc.
• Permitted use cases: inference only, fine-tuning, evaluation, RAG, batch jobs.
• Retention requirements: what can be stored, for how long, and where.
• Cross-boundary rules: what can move between air-gapped and non-air-gapped systems.
• Human access rules: who may access prompts, outputs, logs, model files, and secrets.

If this isn’t written down, compliance becomes subjective.

2) Control all ingress into the air gap

In an air-gapped system, compliance depends heavily on what you allow in.

• Maintain a whitelist of approved artifacts:
  • model binaries/checkpoints
  • tokenizer files
  • container images
  • dependencies
  • vulnerability scan reports
  • configuration templates
• Use signed artifacts only.
• Verify checksums and signatures before import.
• Scan all imported assets for:
  • malware
  • secrets
  • prohibited licenses
  • known vulnerabilities
• Keep a manual approval workflow for imports.

A good practice is to have a “quarantine” staging area for all imports before promotion to production.

3) Minimize and classify data used for inference

Treat prompts and outputs as sensitive by default.

• Classify prompt data before sending it to the model.
• Block or redact:
  • PII
  • credentials
  • secrets
  • regulated data
• Enforce data minimization:
  • send only the fields required
  • avoid unnecessary context in prompts
• If using retrieval-augmented generation:
  • restrict retrieval to approved corpora
  • enforce document-level classification and filtering
  • log which documents were retrieved

If the model processes sensitive enterprise data, the prompts, retrieved context, and outputs are all in scope for policy.

4) Prevent data exfiltration paths

Even air-gapped systems can leak data internally or through operational channels.

Lock down:

• outbound network access
• removable media use
• admin shell access
• remote support tools
• debug endpoints
• telemetry and analytics
• model update channels

Also review:

• error messages
• stack traces
• metrics
• crash dumps
• cached prompts
• temporary files

These often leak sensitive data unintentionally.

5) Secure logging and observability

Logs are a common compliance failure point.

Best practices:

• Log only what you need for audit and troubleshooting.
• Redact sensitive fields in:
  • prompts
  • completions
  • headers
  • payloads
  • file paths
• Separate:
  • security logs
  • application logs
  • model usage logs
• Restrict log access by role.
• Set retention based on policy.
• Encrypt logs at rest if your environment requires it.

If you log prompts or outputs, make sure the policy explicitly allows it.

6) Apply least privilege everywhere

Use strong role-based access control across the stack:

• model registry
• container registry
• serving endpoints
• inference hosts
• orchestration plane
• storage
• secrets management
• observability tools

Only grant:

• operators access to infrastructure
• auditors access to audit logs
• developers access to non-production data
• service accounts the minimum permissions needed

Also rotate secrets and avoid hardcoding API keys, tokens, or credentials.

7) Keep a strict supply-chain process

Enterprise policy usually cares about software provenance.

For every model and dependency, record:

• source
• version
• checksum
• approval date
• license
• vulnerability scan results
• owner
• intended environment

Use:

• SBOMs for software/container images
• model cards for model provenance and limitations
• change management records for promotion

This helps prove that what is running is what was approved.

8) Validate model behavior against data policies

Test the serving stack before production:

• prompt injection resistance
• sensitive data leakage tests
• role-based access tests
• RAG document filtering tests
• output redaction tests
• misuse scenarios
• boundary-crossing tests

Examples:

• Can a user retrieve data from unauthorized documents?
• Can a prompt cause the model to reveal secrets from logs or context?
• Can outputs contain prohibited content or personal data?

Document the results as evidence for compliance.

9) Separate environments and datasets

Do not mix data classes without explicit policy approval.

Recommended separation:

• dev/test with synthetic or de-identified data
• staging with masked production-like data
• production with real approved data only

Also separate:

• model training/fine-tuning from inference
• admin functions from user queries
• regulated workloads from general workloads

This reduces accidental exposure and simplifies audits.

10) Build auditability into the workflow

Be able to answer:

• who accessed the model
• when it was used
• what version ran
• what data class was processed
• which documents or tools were involved
• whether any policy checks fired

Keep immutable audit logs where possible.
Make sure audits can be performed without exposing the underlying sensitive content.

11) Put governance around model updates

For air-gapped deployments, model refreshes should be treated like software releases.

Require:

• approval by data/security owners
• vulnerability review
• license review
• reproducible build or package verification
• rollback plan
• signed release artifacts

Never allow ad hoc model imports into production.

12) Create a compliance checklist for operations

A simple operational checklist helps keep the system compliant:

• All model artifacts are signed and verified
• All imports are scanned and approved
• No unapproved outbound connectivity exists
• Prompts/outputs are redacted where required
• Logs are policy-compliant and access-controlled
• Secrets are stored in approved vaults
• Access is role-based and reviewed regularly
• Audit logs are retained per policy
• RAG sources are approved and classified
• Updates follow change management
• Pen tests / misuse tests are current
• Retention and deletion controls are enforced

13) Involve legal, security, and data governance early

Compliance is not just a technical issue. Get sign-off from:

• legal
• privacy
• security
• data governance
• application owners
• compliance/audit teams

They should confirm:

• what data the model may process
• what must be redacted
• what can be logged
• what retention applies
• what evidence is needed for audits

14) Use a formal risk assessment

If the model is making business decisions or handling regulated data, conduct:

• data protection impact assessment
• threat modeling
• vendor risk review
• model risk management review

This is especially important if the model is third-party or if the serving environment is used in regulated industries.

---

A good baseline architecture

A compliant air-gapped model serving stack usually includes:

• approved import pipeline for models and dependencies
• artifact signing and verification
• container/image scanning
• network isolation
• RBAC and MFA for admins
• centralized secrets management
• policy-based input/output filtering
• restricted logging with redaction
• auditable storage and change control
• approved RAG corpus with document classification
• regular compliance reviews

---

If you want, I can turn this into:

1. a compliance checklist,
2. a reference architecture, or
3. a policy template for enterprise AI serving.