Datalad tool
Embedding Similarity Explorer
Retrieval comes down to turning text into numbers and measuring the angle between them. This does that in front of you: real vectors, real cosine similarity, and the per word arithmetic behind every score. It also shows the exact point where matching on words alone falls over, which is the reason dense embeddings exist at all.
Your documents and your query
The vectors are TF-IDF, built with the same formula scikit-learn uses, so the numbers here are the numbers you would get in a notebook. The geometry is identical to the geometry a dense embedding model uses. What differs is where the numbers come from, and that difference is the whole point of the example loaded below.
How the documents rank
Three ways of measuring the same vectors. They do not agree, and knowing why is most of what there is to know about retrieval scoring.
[…] live See exactly where the top score came from Shows where word overlap stops being enough Open the explorer Free · runs in your browser · no […]
[…] them rank See exactly where the top score came from Shows why word overlap is not understanding Open the explorer Free · runs in your browser · no […]
[…] a query Trace exactly where the top score came from The same vectors NMF and clustering work on Open the explorer Free · runs in your browser · no […]
[…] live See where the top score actually came from Shows where word matching breaks and models help Open the explorer Free · runs in your browser · no […]
[…] How Text Becomes NumbersEmbedding explorerArticleMathCodeCode […]