The most cost-effective way to load high-volume batch data with an ELT tool is usually:

1. Land raw files in cloud object storage first

Use the cheapest durable storage available (e.g. S3, GCS, Azure Blob) and load from there into your warehouse.

• Avoid pushing row-by-row over APIs if possible
• Prefer bulk file loads over streaming
• Use compressed columnar files when supported, especially Parquet or Avro

2. Use the warehouse’s native bulk loader

Most ELT tools are most efficient when they orchestrate the warehouse’s own ingestion path:

• Snowflake: COPY INTO from stage files
• BigQuery: batch load jobs from GCS
• Redshift: COPY from S3
• Databricks / Spark-based: write optimized files and use bulk ingest patterns

This is cheaper than many small inserts because it reduces compute time and API overhead.

3. Batch in larger files, not many tiny files

A very common cost mistake is loading thousands of tiny files.

• Aim for moderately large files, often 100 MB to 1 GB compressed depending on the platform
• Too many small files increase orchestration and metadata overhead
• Too-large files can hurt parallelism, so there’s a sweet spot

4. Incremental loads over full reloads

For recurring batch pipelines:

• Load only new or changed data
• Use watermarks, change data capture (CDC), or partition-based loads
• Avoid truncating and reloading large tables unless necessary

This is often the biggest cost saver.

5. Minimize transformations during ingestion

Since ELT means transform later:

• Keep ingestion simple
• Land raw data first
• Do transformations in the warehouse only after loading
• Avoid expensive parsing or enrichment in the ELT tool itself if the warehouse can do it more efficiently

6. Use autoscaling / elastic execution where possible

If your ELT tool supports it:

• Run loads in short, scheduled bursts
• Use ephemeral workers rather than always-on infrastructure
• Shut down unused compute immediately after batch completion

7. Compress and partition smartly

• Compress files before upload
• Partition data by date or another natural key for easier incremental loads
• Don’t over-partition, or you’ll create too many small files

---

Practical rule of thumb

For high-volume batch ELT, the cheapest pattern is usually:

Source system → compressed batch files in object storage → warehouse bulk load → transformations in the warehouse

---

What usually makes it expensive

Avoid these if you want low cost:

• Row-by-row inserts
• Frequent micro-batches when not needed
• Loading through the ELT tool’s API when bulk storage load is available
• Tiny files
• Reprocessing full history every run
• Doing heavy transforms before landing data

If you tell me which ELT tool and warehouse you’re using (e.g. Fivetran + Snowflake, Airbyte + BigQuery, dbt + Redshift), I can give a more specific cost-optimized setup.