I'm building a media library and need bulk image compression, what approach works best?

For a media library, the best bulk image compression approach is usually:

1) Use lossy compression for delivery, lossless/originals for archive

Keep the original files untouched, and generate optimized derivatives for display/use.

• Archive/master: store original upload
• Web/preview: generate compressed versions
• Thumbnails: separate very small derivatives

This gives you quality preservation plus fast load times.

2) Use modern formats where possible

Best practical formats today:

• AVIF: smallest files, best compression, slower encode
• WebP: excellent balance of size, quality, speed
• JPEG: still useful for compatibility, especially photos
• PNG: keep for transparency and lossless needs, but compress carefully

Typical rule:

• Photos: AVIF or WebP
• UI graphics/logos: WebP/PNG
• Transparency needed: WebP or PNG
• Very old browser support: JPEG/PNG fallback

3) Resize before compressing

Don’t compress a 6000px image down to a tiny file and then display it at 300px.
First scale to the maximum size you actually need.

This usually saves more than compression alone.

4) Use an automated pipeline

For bulk processing, create a batch job that:

1. Detects image type
2. Resizes to target dimensions
3. Converts to the preferred format
4. Applies quality settings
5. Generates multiple sizes if needed
6. Stores metadata and original file reference

Good libraries/tools:

• sharp (Node.js, very fast, great for batch processing)
• libvips (underneath sharp, very efficient)
• ImageMagick / GraphicsMagick (flexible, heavier)
• Squoosh CLI or cwebp/avifenc for command-line workflows

5) Use sensible quality settings

Starting points:

• JPEG/WebP: quality around 75–85
• AVIF: quality around 45–60 depending on content
• PNG: use lossless optimization only, unless converting to another format

Then test visually on a sample set.

6) Consider perceptual optimization

For libraries with lots of photos:

• Strip metadata unless needed
• Optimize chroma subsampling
• Remove unused ICC profiles if acceptable
• Use perceptual comparison / SSIM testing to tune quality

7) Make it async and scalable

For bulk libraries:

• Put uploads into a queue
• Process images in workers
• Store output in object storage/CDN
• Cache generated sizes
• Keep processing idempotent so re-runs are safe

Recommended default setup

If you want a practical default:

• Store originals
• Generate:
  • AVIF for modern clients
  • WebP as fallback
  • JPEG only if required
• Resize to a few standard widths, like:
  • 320px
  • 640px
  • 1280px
• Strip metadata unless it matters
• Process with sharp/libvips

If you want the simplest “best” answer

Use sharp to resize and convert images to WebP or AVIF in multiple sizes, keep originals, and run compression asynchronously in a worker queue.

If you tell me your stack (Node, Python, Go, etc.) and whether this is for web display or archival, I can suggest an exact pipeline and quality settings.