Source
Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers
Tom Aarsen explains how to train and finetune multi-vector embedding models using the Sentence Transformers v6.0 library. He demonstrates that domain-specific finetuning on a single GPU can significantly outperform general-purpose retrieval models.
Why it matters
This allows developers to create highly accurate search and retrieval systems for specialized fields like medicine or law using affordable consumer hardware. It reduces errors in finding relevant documents compared to using general-purpose models.
The details
The mLateOn-medical model was trained in 14.5 hours on a single RTX 3090. Multi-vector models keep one vector per token, avoiding the compression found in dense embeddings. 1-bit PLAID quantization can reduce index size from 45 GB to 1.45 GB. Training at 512 tokens instead of full length reduced NDCG@10 by 0.015.
Get the weekly recap
The stories like this one, picked and explained — once a week, straight to your inbox.