How do I load data from multiple sources into Snowflake?

You can load data into Snowflake from multiple sources in a few common ways, depending on where the data lives and how automated you want the process to be.

Common approaches

1) Load files from cloud storage or local files

Typical sources:

• Amazon S3
• Azure Blob Storage / ADLS
• Google Cloud Storage
• Local files via PUT

Basic pattern:

1. Create a stage for each source.
2. Define a file format for each data type.
3. Load into a target table using COPY INTO.

Example:

CREATE OR REPLACE STAGE my_s3_stage
  URL='s3://my-bucket/data/'
  STORAGE_INTEGRATION = my_s3_int;

CREATE OR REPLACE FILE FORMAT my_csv_fmt
  TYPE = CSV
  SKIP_HEADER = 1
  FIELD_OPTIONALLY_ENCLOSED_BY = '"';

COPY INTO my_table
FROM @my_s3_stage
FILE_FORMAT = (FORMAT_NAME = my_csv_fmt);

Repeat for each source/stage, or load each source into its own landing table first.

---

2) Load from multiple external locations into one table

If the sources are similar, you can use:

• one table
• multiple stages or directories
• multiple COPY INTO commands
• or a single COPY INTO with a file pattern if the files are in one stage

Example with multiple stages:

COPY INTO my_table
FROM @stage_a
FILE_FORMAT = (FORMAT_NAME = csv_fmt);

COPY INTO my_table
FROM @stage_b
FILE_FORMAT = (FORMAT_NAME = csv_fmt);

---

3) Use Snowpipe for continuous ingestion

For near real-time loading from cloud storage:

• Set up external stages
• Configure event notifications
• Create a Snowpipe pipe that auto-ingests new files

Good for:

• many small files
• ongoing ingestion
• reducing manual COPY INTO runs

---

4) Use Snowflake connectors or ETL tools

If your data comes from databases, APIs, SaaS apps, or streaming systems, use:

• Snowflake JDBC/ODBC connectors
• Python connector
• Fivetran
• Airbyte
• Matillion
• dbt + orchestration tools
• Kafka connector / Snowpipe Streaming

These tools can extract from multiple sources and land data in Snowflake automatically.

---

Recommended pattern for multiple sources

A scalable approach is:

1. Land each source separately

   • raw tables like raw_salesforce_*, raw_s3_*, raw_api_*

2. Standardize into staging tables

   • clean datatypes, deduplicate, normalize columns

3. Merge into final models

   • use MERGE or transformation pipelines

This makes troubleshooting much easier than trying to combine everything during ingestion.

---

Example pipeline design

Source A: S3 CSV

COPY INTO raw_sales
FROM @sales_stage
FILE_FORMAT = (TYPE = CSV SKIP_HEADER = 1);

Source B: Azure JSON

COPY INTO raw_events
FROM @events_stage
FILE_FORMAT = (TYPE = JSON);

Combine later

INSERT INTO unified_fact_table
SELECT ... FROM raw_sales
UNION ALL
SELECT ... FROM raw_events;

---

Key considerations

• Make sure each source has the correct file format
• Use separate staging tables if schemas differ
• Watch for duplicate loads
• Use COPY INTO ... ON_ERROR = CONTINUE only when appropriate
• Prefer incremental ingestion for large or frequently updated sources

---

If you want, I can show you:

1. a step-by-step Snowflake SQL example for multiple sources, or
2. the best architecture for your specific sources (S3, API, databases, etc.).