Prompt
multi tenant saas integration architecture
Latest observation
A multi-tenant SaaS integration architecture is the design pattern used to let one SaaS platform securely connect to many external systems—while keeping each customer’s data, credentials, policies, and workflows isolated.
Here’s a practical reference architecture.
1) Core goals
Your integration layer usually needs to support:
- Tenant isolation: one customer’s credentials/data must not leak to another
- Multiple integrations per tenant: e.g. Salesforce, Slack, NetSuite, S3, etc.
- Different auth methods: OAuth2, API keys, service accounts, webhooks, SAML, JWT
- Configurable workflows: per-tenant mappings, schedules, filters, routing
- Scale & reliability: retries, backoff, idempotency, dead-letter queues
- Auditability: who connected what, when, and what data moved
- Compliance: encryption, data residency, least privilege, retention
2) High-level architecture
A common pattern is:
+-------------------+
| SaaS Frontend |
+---------+---------+
|
v
+-------------------+
| API Gateway / BFF |
+---------+---------+
|
+-----------+-----------+
| |
v v
+-------------------+ +----------------------+
| Tenant / IAM Svc | | Integration Control |
| auth, RBAC, policy | | Plane |
+-------------------+ +----------+-----------+
|
v
+--------------------------+
| Integration Orchestrator |
| workflows / connectors |
+-----+---------+----------+
| |
v v
+----------------+ +----------------+
| Queue / Event | | Connector Runt.|
| Bus | | workers |
+--------+--------+ +--------+--------+
| |
v v
+-------------+ +------------------+
| Secrets Vault| | External Systems |
| per-tenant | | APIs / Webhooks |
+-------------+ +------------------+
3) Main components
A. Tenant Management
Stores:
- tenant profile
- plan/entitlements
- region/data residency
- integration permissions
- resource limits/quotas
Tip: make tenant_id a first-class field in every table, event, and log entry.
B. Identity and Access Management
Handles:
- user auth
- tenant membership
- RBAC/ABAC
- admin vs operator permissions
For integration access:
- use service principals or integration identities
- separate human access from system access
- require scoped permissions for each connector
C. Integration Control Plane
This is the “brains” of integrations. It manages:
- connection setup
- credential lifecycle
- connector configuration
- schema mappings
- scheduling
- enable/disable state
- health checks
It should be metadata-driven, so adding a new tenant config doesn’t require code changes.
D. Connector Runtime / Worker Plane
Executes actual work:
- polling external APIs
- receiving webhooks
- pushing data outbound
- transforming payloads
- retrying failed jobs
Best practice:
- keep connectors stateless
- use ephemeral workers or containers
- isolate execution by tenant, connector, or workload class
E. Event/Queue Layer
Use queues or event streams for:
- decoupling control plane from execution
- retry handling
- burst absorption
- async processing
Common patterns:
- one shared queue with tenant metadata
- partitioned queues by tenant tier
- dedicated queue per high-value tenant
- topics per integration type
F. Secrets Vault
Store:
- OAuth refresh tokens
- API keys
- signing secrets
- certificates
Requirements:
- envelope encryption
- per-tenant or per-integration keying
- automatic rotation where possible
- audit access logs
Never store raw secrets in application DBs.
4) Data isolation models
You have 3 common tenancy models:
Option 1: Shared everything
- one app deployment
- one database
- tenant discriminator in rows
Pros: low cost, simpler ops
Cons: strongest need for careful isolation
Option 2: Shared app, separate DB/schema per tenant
- more isolation
- easier compliance for some customers
Pros: better isolation, easier data export/delete
Cons: more operational complexity
Option 3: Fully isolated tenant stacks
- separate app, DB, queue, runtime
Pros: strongest isolation and customization
Cons: expensive, harder to operate
Most SaaS products start with shared app/shared DB and evolve to hybrid isolation for enterprise tenants.
5) Recommended hybrid pattern
A practical enterprise-friendly architecture:
- Shared control plane for all tenants
- Shared worker fleet
- Separate secrets namespace per tenant
- Per-tenant encryption keys
- Optional dedicated queue/worker pool for premium tenants
- Optional dedicated database/schema for regulated tenants
This gives you:
- simpler ops for most customers
- strong isolation where needed
- clear upgrade path for enterprise plans
6) Integration flow examples
Outbound sync flow
- Tenant configures integration
- Credentials stored in vault
- Control plane validates connection
- Orchestrator creates sync job
- Worker pulls job from queue
- Worker fetches data from SaaS DB
- Worker transforms data using tenant mapping
- Worker calls external API
- Result is persisted, audited, and retried if needed
Inbound webhook flow
- External system sends webhook
- Webhook gateway verifies signature
- Gateway resolves tenant/integration
- Event is enqueued
- Worker processes and normalizes payload
- Data is written to tenant-scoped storage
- Downstream workflows are triggered
7) Important design principles
Tenant-aware everything
Every request, job, log, metric, and record should carry:
tenant_idintegration_idenvironmentif applicablecorrelation_id
Idempotency
External APIs and retries can duplicate work. Use:
- idempotency keys
- deduplication store
- event versioning
- exactly-once-like processing patterns
Rate limiting and quotas
Per tenant / per integration:
- API call limits
- concurrent job limits
- sync frequency caps
- payload size limits
Prevents one tenant from exhausting shared resources.
Retry and dead-letter handling
Use:
- exponential backoff
- circuit breakers
- dead-letter queues
- manual replay tools
Also distinguish:
- transient errors
- auth errors
- validation errors
- permission errors
Schema mapping and transformation
You’ll likely need:
- field mapping
- custom rules
- enrichment
- filtering
- normalization
- versioned schemas
A “mapping engine” is often a core product feature in integration SaaS.
8) Security considerations
- Encrypt data in transit and at rest
- Use least-privilege OAuth scopes
- Rotate credentials
- Validate webhook signatures
- Prevent cross-tenant queries at the database layer
- Separate tenant data in logs/metrics/traces
- Mask secrets and sensitive PII
- Use private networking for enterprise connectors where possible
9) Observability
Track by tenant and connector:
- success/failure rate
- latency
- retry count
- throughput
- token refresh failures
- webhook delivery lag
- sync freshness / staleness
Include:
- dashboards per tenant
- audit logs
- alerting on auth expiration and failure spikes
10) Suggested service decomposition
A scalable microservice split could be:
- Tenant Service
- Auth/IAM Service
- Integration Registry
- Secrets Service
- Webhook Ingest Service
- Sync Orchestrator
- Connector Workers
- Mapping/Transformation Service
- Audit/Event Log Service
- Notification Service
You can also start as a modular monolith and split later.
11) Technology choices
Common stack choices:
- API gateway: Kong / Apigee / AWS API Gateway
- Queue: Kafka / SQS / RabbitMQ / PubSub
- Secrets: HashiCorp Vault / AWS Secrets Manager / GCP Secret Manager
- DB: Postgres with tenant partitioning
- Cache: Redis
- Workflow: Temporal / Step Functions / Durable Functions
- Observability: OpenTelemetry + Prometheus/Grafana + ELK/Splunk
12) Best-practice reference model
If you want a strong default architecture, use this:
- Shared control plane
- Metadata-driven integration registry
- Tenant-scoped secrets vault
- Async job processing via queue
- Stateless connector workers
- Per-tenant idempotency + rate limits
- Strong audit logging
- Hybrid isolation for enterprise tenants
If you want, I can also provide one of these:
- a diagram in C4 format
- a microservices breakdown
- a reference AWS/Azure/GCP architecture
- a database schema for multi-tenant integrations
- a sample event flow for OAuth-based integrations