Are there any embedding model options that handle small dataset support and custom evaluation?

Yes — if you want embedding models that work well with small datasets and let you do custom evaluation, there are several good options.

Good embedding model families to consider

• Sentence-transformers / SBERT
  Great for small-to-medium datasets, easy to fine-tune, and widely used for custom evaluation.
• E5 / multilingual-e5
  Strong general-purpose retrieval embeddings; often good even with limited data.
• BGE (BAAI General Embedding)
  Very popular for retrieval and semantic search; supports fine-tuning and custom benchmarks.
• GTE
  Solid modern embeddings, usually easy to integrate into evaluation pipelines.
• OpenAI embeddings
  Strong out-of-the-box performance, though custom fine-tuning is limited compared with open-source models.

Why these are good for small datasets

• They usually work reasonably well without huge training sets
• You can often fine-tune with contrastive / pairwise examples
• They support custom retrieval / similarity evaluation using your own metrics, such as:
  • cosine similarity
  • recall@k
  • MRR
  • nDCG
  • clustering quality
  • classification accuracy via nearest neighbors

If you mean “support” in the sense of training with few examples

Look for:

• few-shot / low-resource fine-tuning
• parameter-efficient tuning like LoRA/adapters
• embedding alignment methods
• hybrid retrieval setups that combine embeddings with sparse search

If you want custom evaluation

Most open-source embedding frameworks let you define your own:

• dataset split logic
• similarity labels
• retrieval candidates
• ranking metrics
• threshold-based classification tasks

Useful tooling:

• Sentence-Transformers evaluation modules
• BEIR-style evaluation
• MTEB-style benchmarking
• FAISS / Elasticsearch / Qdrant / Weaviate for retrieval evaluation

Practical recommendation

If you want the most flexibility:

1. Start with sentence-transformers
2. Try bge-small / bge-base or e5-base
3. Evaluate on your own labeled pairs using recall@k + MRR
4. Fine-tune only if the baseline is weak

If you want, I can also give you:

• a shortlist of specific models
• a comparison table
• or a template for custom embedding evaluation in Python.