docs/research/gmi-vs-agent/references.md

Rama master · réplica de lectura del repositorio original.

# References

The numbered references of [the paper](paper.md). Each entry gives the source's authors, title and venue as its own record gives them, its URL, the date it was read and, for an arXiv preprint, the version read; "per the arXiv record" marks a venue taken from an arXiv comment or journal reference. Under each entry: the sections of the paper that cite it, what they cite it for, and the part of the source that supports it.

[1] Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, Yuwei Fang. Evaluating Very Long-Term Conversational Memory of LLM Agents. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13851-13870, 2024. https://doi.org/10.18653/v1/2024.acl-long.747 and https://arxiv.org/abs/2402.17753 (read 2026-10-11; arXiv version v1).
Cited in Sections 1, 3.1 and 4.3: LOCOMO, its dialogues of about 300 turns and 9K tokens on average over up to 35 sessions, and its question answering, event summarization and multimodal dialogue generation tasks. Source part: the abstract; the venue and pages from the Crossref record of the DOI.

[2] Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, Dong Yu. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. ICLR 2025 (per the arXiv record). https://arxiv.org/abs/2410.10813 (read 2026-10-11; arXiv version v2).
Cited in Sections 1, 3.1, 4.3 and 5.3: LongMemEval's 500 questions and its five abilities (information extraction, multi-session reasoning, temporal reasoning, knowledge updates, abstention); in 4.3, its grading prompts, chosen by question type; in 5.3, the sources of its histories' sessions: simulated chats, and ShareGPT and UltraChat as publicly released chat data. Source part: the abstract; for 4.3, section 3.3 ("Question Answering") and appendix A.4 ("Evaluation Metric Building", the judge's instructions by question type), read in the HTML version on 2026-10-13; for 5.3, section 3.2 ("History Compilation") and appendix A.2 ("Session sampling", the mixture of 25% ShareGPT, 25% UltraChat and 50% simulated sessions), read in the HTML version on 2026-10-12, https://arxiv.org/html/2410.10813v2.

[3] Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez. MemGPT: Towards LLMs as Operating Systems. arXiv preprint, 2023. https://arxiv.org/abs/2310.08560 (read 2026-10-11; arXiv version v2).
Cited in Sections 1 and 3.2: memory tiers that give a model context beyond its window, and interrupts that manage control flow. Source part: the abstract.

[4] Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein. Generative Agents: Interactive Simulacra of Human Behavior. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST '23), pp. 1-22, 2023. https://doi.org/10.1145/3586183.3606763 and https://arxiv.org/abs/2304.03442 (read 2026-10-11; arXiv version v2). The Crossref record lists two of the authors as "Joseph O'Brien" and "Carrie Jun Cai"; the names here are the arXiv record's.
Cited in Sections 1, 2.2 and 3.2: a complete record of experience in natural language, synthesized into reflections and retrieved to plan; persona, memory and reflection in one agent. Source part: the abstract.

[5] Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, Thomas L. Griffiths. Cognitive Architectures for Language Agents. Transactions on Machine Learning Research (per the arXiv record: "v3 is TMLR camera ready version"); arXiv preprint, 2023. https://arxiv.org/abs/2309.02427 (read 2026-10-11; arXiv version v3).
Cited in Sections 1, 2.2 and 3.2: modular memory, a structured action space and a decision process (the abstract); working memory and the long-term episodic, semantic and procedural memories (section 4.1, read in the HTML version, https://arxiv.org/html/2309.02427v3).

[6] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, Ion Stoica. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS 2023 Datasets and Benchmarks Track (per the arXiv record). https://arxiv.org/abs/2306.05685 (read 2026-10-11; arXiv version v4).
Cited in Sections 3.4 and 8: position, verbosity and self-enhancement biases, and over 80% agreement of strong judges with human preferences. Source part: the abstract.

[7] Michael C. Ashton, Kibeom Lee. Empirical, Theoretical, and Practical Advantages of the HEXACO Model of Personality Structure. Personality and Social Psychology Review 11(2), 150-166, 2007. https://doi.org/10.1177/1088868306294907 (read 2026-10-11).
Cited in Section 2.2: HEXACO's six factors. Source part: the abstract in the Crossref record.

[8] dial481 (repository owner). locomo-audit: LoCoMo Benchmark Audit, repository README. GitHub, not peer reviewed, undated page. https://github.com/dial481/locomo-audit (read 2026-10-11).
Cited in Sections 3.1 and 8: 99 of 1,540 questions with wrong golden answers, and 62.81% of intentionally wrong, vague but topical answers accepted by the judge. Source part: the README's "Key Findings" table.

[9] Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, Deshraj Yadav. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv preprint, 2025. https://arxiv.org/abs/2504.19413 (read 2026-10-11; arXiv version v1).
Cited in Section 3.1: a memory layer that reports on LOCOMO. Source part: the abstract.

[10] Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, Daniel Chalef. Zep: A Temporal Knowledge Graph Architecture for Agent Memory. arXiv preprint, 2025. https://arxiv.org/abs/2501.13956 (read 2026-10-11; arXiv version v1).
Cited in Section 3.1: a memory layer that reports on LongMemEval. Source part: the abstract.

[11] Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, Yanlin Wang. MemoryBank: Enhancing Large Language Models with Long-Term Memory. arXiv preprint, 2023. https://arxiv.org/abs/2305.10250 (read 2026-10-11; arXiv version v3).
Cited in Section 3.2: forgetting and reinforcement along the Ebbinghaus forgetting curve, applied in a long-term companion. Source part: the abstract.

[12] Vinay Samuel, Henry Peng Zou, Yue Zhou, Shreyas Chaudhari, Ashwin Kalyan, Tanmay Rajpurohit, Ameet Deshpande, Karthik Narasimhan, Vishvak Murahari. PersonaGym: Evaluating Persona Agents and LLMs. Findings of EMNLP 2025 (per the arXiv record). https://arxiv.org/abs/2407.18416 (read 2026-10-11; arXiv version v5).
Cited in Section 3.3: persona agents evaluated across environments with a score grounded in decision theory. Source part: the abstract.

[13] Quan Tu, Shilong Fan, Zihang Tian, Rui Yan. CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation. arXiv preprint, 2024. https://arxiv.org/abs/2401.01275 (read 2026-10-11; arXiv version v2).
Cited in Section 3.3: a Chinese benchmark for role-playing agents with thirteen metrics in four dimensions. Source part: the abstract.

[14] Hannah Rashkin, Eric Michael Smith, Margaret Li, Y-Lan Boureau. Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset. ACL 2019, long paper (per the arXiv record). https://arxiv.org/abs/1811.00207 (read 2026-10-11; arXiv version v5).
Cited in Section 3.3: dialogue grounded in emotional situations. Source part: the abstract.

[15] Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, Zhifang Sui. Large Language Models are not Fair Evaluators. arXiv preprint, 2023. https://arxiv.org/abs/2305.17926 (read 2026-10-11; arXiv version v2).
Cited in Sections 3.4 and 8: the order of two answers changes a judge's verdict, and aggregating over both orders (balanced position calibration) calibrates for it. Source part: the abstract.

[16] Lin Shi, Chiyu Ma, Wenhua Liang, Xingjian Diao, Weicheng Ma, Soroush Vosoughi. Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge. AACL-IJCNLP 2025 (per the arXiv record). https://arxiv.org/abs/2406.07791 (read 2026-10-11; arXiv version v9).
Cited in Sections 3.4 and 8: position bias that varies across judges and tasks, and its three measures (repetition stability, position consistency, preference fairness). Source part: the abstract.

[17] Arjun Panickssery, Samuel R. Bowman, Shi Feng. LLM Evaluators Recognize and Favor Their Own Generations. arXiv preprint, 2024. https://arxiv.org/abs/2404.13076 (read 2026-10-11; arXiv version v1).
Cited in Sections 3.4 and 8: evaluators recognize their own generations, and self-preference grows with that ability. Source part: the abstract.

[18] Yann Dubois, Balázs Galambosi, Percy Liang, Tatsunori B. Hashimoto. Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators. COLM 2024 (per the arXiv record). https://arxiv.org/abs/2404.04475 (read 2026-10-11; arXiv version v2).
Cited in Sections 3.4, 4.6 and 8: a regression that predicts the preference at zero length difference. Source part: the abstract.

[19] Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, Benyou Wang. Humans or LLMs as the Judge? A Study on Judgement Biases. EMNLP 2024 (per the arXiv record). https://arxiv.org/abs/2402.10669 (read 2026-10-11; arXiv version v5).
Cited in Section 3.4: a catalogue of judge biases. Source part: the abstract.

[20] Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, Dongyeop Kang. Benchmarking Cognitive Biases in Large Language Models as Evaluators. ACL 2024 (per the arXiv record, which reads "Publishsed at ACL 2024"; whether the main proceedings or Findings is not stated there). https://arxiv.org/abs/2309.17012 (read 2026-10-11; arXiv version v3).
Cited in Section 3.4: a benchmark of six cognitive biases in model evaluators. Source part: the abstract.

[21] Rickard Stureborg, Dimitris Alikaniotis, Yoshi Suhara. Large Language Models are Inconsistent and Biased Evaluators. arXiv preprint, 2024. https://arxiv.org/abs/2405.01724 (read 2026-10-11; arXiv version v1).
Cited in Section 3.4: evaluators that are biased and inconsistent. Source part: the abstract.

[22] Aman Singh Thakur, Kartik Choudhary, Venkat Srinik Ramayapally, Sankaran Vaidyanathan, Dieuwke Hupkes. Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges. Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM²), pp. 404-430, 2025 (the arXiv journal reference). https://arxiv.org/abs/2406.12624 (read 2026-10-11; arXiv version v6).
Cited in Section 3.4: judges' alignment with human graders, and a tendency toward leniency. Source part: the abstract.

[23] Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Y. Tang, Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, Ion Stoica. JudgeBench: A Benchmark for Evaluating LLM-based Judges. ICLR 2025 (per the arXiv record). https://arxiv.org/abs/2410.12784 (read 2026-10-11; arXiv version v2).
Cited in Section 3.4: a benchmark of judges on hard response pairs. Source part: the abstract.

[24] Sumanth Doddapaneni, Mohammed Safi Ur Rahman Khan, Sshubam Verma, Mitesh M. Khapra. Finding Blind Spots in Evaluator LLMs with Interpretable Checklists. EMNLP 2024 (per the arXiv record). https://arxiv.org/abs/2406.13439 (read 2026-10-11; arXiv version v2).
Cited in Sections 3.4 and 8: targeted perturbations whose quality drop evaluator models miss in over 50% of cases on average. Source part: the abstract.

[25] Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6(2), 65-70, 1979 (zbMATH 0402.62058; the issue number from Wikipedia's citation of the article, since JSTOR's page answered with a bot check). https://www.jstor.org/stable/4615733 and https://zbmath.org/?q=an:0402.62058 (read 2026-10-11).
Cited in Sections 3.5, 4.6 and 8: a sequentially rejective procedure for several tests that controls the family-wise error rate. Source part: the title (zbMATH record) and Wikipedia's article on the Holm-Bonferroni method for the error rate it controls.

[26] C. A. Field, A. H. Welsh. Bootstrapping Clustered Data. Journal of the Royal Statistical Society Series B: Statistical Methodology 69(3), 369-390, 2007. https://doi.org/10.1111/j.1467-9868.2007.00593.x (read 2026-10-11).
Cited in Sections 3.5, 4.6 and 8: the cluster bootstrap's consistent variance estimates under both the random-effect and the transformation models. Source part: the summary in the Crossref record.

[27] Evan Miller. Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations. arXiv preprint, 2024. https://arxiv.org/abs/2411.00640 (read 2026-10-11; arXiv version v1; sections 2.2 and 4.2 read in the HTML version, https://arxiv.org/html/2411.00640v1).
Cited in Sections 3.5, 4.6 and 8: clustered questions and paired comparisons in model evaluations. Source part: the abstract, and sections 2.2 "Clustered questions" and 4.2 "Paired analysis" of the HTML version.

[28] Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, David T. Mellor. The preregistration revolution. Proceedings of the National Academy of Sciences 115(11), 2600-2606, 2018. https://doi.org/10.1073/pnas.1708274114 (read 2026-10-11).
Cited in Sections 3.5 and 4.9: defining the questions and the analysis plan before observing the outcomes separates tests of predictions from postdictions. Source part: the abstract in the Crossref record.

[29] Ron Kohavi, Diane Tang, Ya Xu. Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press, 2020; chapter 19, "The A/A Test", pp. 200-208. https://doi.org/10.1017/9781108653985 and https://doi.org/10.1017/9781108653985.025 (read 2026-10-11).
Cited in Section 3.5: A/A tests as part of establishing trust in an experimentation platform, and their frequent failures in practice. Source part: the chapter's summary on the publisher's page; the subtitle and pages from Crossref.

[30] Anthropic. OpenAI SDK compatibility. Claude Platform Docs, vendor documentation, undated page. https://platform.claude.com/docs/en/cli-sdks-libraries/libraries/openai-sdk (read 2026-10-11).
Cited in Sections 4.1 and 8: "primarily intended to test and compare model capabilities, and is not considered a long-term or production-ready solution for most use cases"; "Prompt caching is not supported"; system and developer messages hoisted and joined with a single newline; `reasoning_effort` ignored. Source part: the page's introduction, its list of limitations and its field tables.