What should I use for Postgres to BigQuery replication?

If you want to replicate Postgres to BigQuery, the “best” choice depends on whether you need near-real-time CDC, simple batch loads, or managed low-ops.

Good options

1) Debezium + Kafka / Dataflow / BigQuery sink

Best if you need true change data capture and control.

• Pros
  • Near-real-time replication
  • Captures inserts/updates/deletes
  • Flexible and well understood
• Cons
  • More infrastructure to run
  • More operational complexity

Use this if you already have Kafka or are comfortable operating CDC pipelines.

---

2) Google Datastream → BigQuery

Best if you want a managed Google Cloud solution.

• Pros
  • Managed CDC from Postgres
  • Integrates well with BigQuery
  • Less operational overhead
• Cons
  • Tied to GCP
  • Can require some setup around schemas, permissions, and networking

This is often the easiest choice if your stack is already on Google Cloud.

---

3) Airbyte

Best if you want an open-source or managed ELT tool with easier setup.

• Pros
  • Simple to configure
  • Supports many sources/destinations
  • Good for both batch and CDC depending on setup
• Cons
  • CDC quality and performance depend on deployment and connector maturity
  • May need tuning for large/high-write databases

Good middle ground if you want something faster to deploy than building your own pipeline.

---

4) Fivetran

Best if you want fully managed replication and don’t mind paying for it.

• Pros
  • Very low operational burden
  • Reliable managed connectors
  • Good schema handling
• Cons
  • Can get expensive at scale
  • Less control over pipeline behavior

Great if budget is less important than simplicity and reliability.

---

5) Custom ETL/ELT with scheduled jobs

Best if you only need periodic sync, not CDC.

• Pros
  • Simple for small datasets
  • Low cost
• Cons
  • Not real-time
  • Harder to manage deletes/updates correctly
  • Not ideal for large or fast-changing data

Use only if daily/hourly freshness is enough.

---

My recommendation

• If you’re on GCP and want CDC: Datastream
• If you want easiest managed replication overall: Fivetran
• If you want open-source or more control: Debezium
• If you want a practical self-managed ELT tool: Airbyte

Quick rule of thumb

• < 1 hour freshness needed → scheduled ELT / Airbyte
• Minutes-level freshness needed → Datastream / Debezium / Fivetran
• Want minimal ops → Fivetran or Datastream
• Want control and portability → Debezium

If you want, I can also recommend the best option based on:

• your cloud provider,
• data volume,
• latency target,
• and budget.