Cite
Vyshnyvetska, Maryna. Information Gravity: A Field-Theoretic Model for Token Selection in Large Language Models. 25 Apr. 2025. arXiv.org, https://doi.org/10.5281/zenodo.15289890.
Metadata
Title: Information Gravity: A Field-Theoretic Model for Token Selection in Large Language Models Authors: Maryna Vyshnyvetska Cite key: vyshnyvetska2025Informationa
Links
Abstract
We propose a theoretical model called “information gravity” to describe the text generation process in large language models (LLMs). The model uses physical apparatus from field theory and spacetime geometry to formalize the interaction between user queries and the probability distribution of generated tokens. A query is viewed as an object with “information mass” that curves the semantic space of the model, creating gravitational potential wells that “attract” tokens during generation. This model offers a mechanism to explain several observed phenomena in LLM behavior, including hallucinations (emerging from low-density semantic voids), sensitivity to query formulation (due to semantic field curvature changes), and the influence of sampling temperature on output diversity.
Notes
From Obsidian
(As notes and annotations from Zotero are one-way synced, this section include a link to another note within Obsidian to host further notes)
Information-Gravity---A-Field-Theoretic-Model-for-Token-Selection-in-Large-Language-Models
From Zotero
(one-way sync from Zotero) Imported: 2025-06-25 Comment: 12 pages, 1 figure View in local Zotero
Annotations
Highlighting colour codes
Link to original
- Note: highlights for quicker reading or comments stemmed from reading the paper but might not be too related to the paper
- External Insight: Insights from other works but was mentioned in the paper
- Question/Critic: questions or comments on the content of paper
- Claim: what the paper claims to have found/achieved
- Finding: new knowledge presented by the paper
- Important: anything interesting enough (findings, insights, ideas, etc.) that’s worth remembering
From Zotero
(one-way sync from Zotero)