Tech News
← Home  ·  All topics

Gram Matrix

1 GoKawiil brief on this topic

Researcher bets he can find random words that mimic circular embedding geometry

A blog post describes a viral paper by Dhruva Karkada showing that LLM and word2vec embeddings of month names project onto a near-perfect circle, with their Gram matrix closely matching a circulant matrix predicted by a simple analytical theory. The author, skeptical that this pattern is unique to semantically related word sets like months, wagered a beer that he could find ten seemingly unrelated words whose word2vec embeddings would also form a clear circle and circulant structure.