What should I use for embeddings that need to work across text, images, and audio?

If you need one embedding space that works across text, images, and audio, you should look for a multimodal embedding model trained to align those modalities into the same vector space.

Good choices

1. OpenAI CLIP-style / multimodal embedding models

• Best-known for text + image
• Great if your main use is matching captions, queries, and images
• Not natively audio-inclusive in the original CLIP

2. ImageBind (Meta)

• Designed to embed text, images, audio and more into a shared space
• Strong fit for your use case because it’s explicitly multimodal across these three
• Useful for retrieval, search, clustering, and cross-modal similarity

3. Gemini / other proprietary multimodal embedding APIs

• Some newer commercial models support multiple modalities in a unified system
• Good if you want managed infrastructure rather than running models yourself
• Check whether they provide a true shared embedding space versus separate modality-specific encoders

4. AudioCLIP / CLAP

• CLAP is especially strong for text + audio
• AudioCLIP extends CLIP-like alignment to audio, image, and text
• Useful if audio is central and you want open-source options

What to pick

• If you want open-source and all three modalities: ImageBind
• If you want text + image only with mature ecosystem: CLIP
• If you want text + audio: CLAP
• If you want hosted API convenience: use a multimodal embedding API that explicitly supports all three

Important caveat

A “single embedding” across text, image, and audio only works well if the model was trained to align those modalities together. Otherwise, embeddings from different encoders won’t be directly comparable.

Practical recommendation

If your goal is cross-modal search/retrieval across text, images, and audio, start with:

• ImageBind if you can run models locally
• A multimodal API if you prefer managed deployment

If you want, I can also suggest:

1. the best open-source model,
2. the best API option, or
3. a comparison table of the main models.