Turkish Embedding Leaderboard

This is a leaderboard comparing Turkish performances of embedding models, using MTEB-TR benchmark developed by Selman Baysan in his MS Thesis.

Paper: https://aclanthology.org/2025.findings-emnlp.471/

Repo: https://github.com/selmanbaysan/mteb_tr

Rank Model Parameters Mean (Task) Mean (Type) Retrieval Bitext Classification Clustering Pair Classification STS
1 Qwen/Qwen3-Embedding-8B 8B 67.93 73.27 62.82 98.07 72.81 63.13 62.78 80.01
2 intfloat/multilingual-e5-large 0.6B 66.74 72.95 60.62 99.43 71.79 60.58 64.12 81.18
3 Qwen/Qwen3-Embedding-4B 4B 66.12 71.35 61.76 97.86 70.32 61.08 60.10 76.96
4 ytu-ce-cosmos/turkish-e5-large 0.6B 65.96 72.33 58.65 99.24 72.62 60.77 62.72 80.00
5 google/embeddinggemma-300m 0.3B 65.20 70.52 58.60 96.84 71.81 62.36 60.57 72.93
6 microsoft/harrier-oss-v1-0.6b 0.6B 64.29 70.57 56.90 98.58 71.18 63.60 58.63 74.54
7 microsoft/harrier-oss-v1-270m 0.27B 62.61 69.54 54.45 98.30 69.88 62.19 57.42 75.01
8 Qwen/Qwen3-Embedding-0.6B 0.6B 60.97 66.47 54.87 92.43 65.82 60.62 58.16 66.91
9 sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 0.1B 58.84 68.29 46.57 93.67 65.76 56.56 68.80 78.41
10 sentence-transformers/LaBSE 0.5B 57.48 66.43 46.47 99.53 65.60 58.42 56.80 71.75
11 emrecan/bert-base-turkish-cased-mean-nli-stsb-tr 0.1B 54.82 57.75 42.46 31.47 66.44 56.79 66.23 83.13
12 ytu-ce-cosmos/modernbert-tr-base-1k 0.15B 48.89 46.78 32.66 12.29 71.41 64.20 49.86 50.24
13 boun-tabilab/TabiBERT 0.15B 41.10 42.53 19.49 8.58 65.42 60.76 49.94 50.97
14 sentence-transformers/all-MiniLM-L6-v2 0.02B 37.01 36.95 23.25 6.78 52.46 40.52 50.14 48.55
15 answerdotai/ModernBERT-base 0.15B 27.73 31.65 5.53 0.62 47.94 48.23 46.01 41.57