 [2406.18312v1] AI-native Memory: A Pathway from LLMs Towards AGI




























  








Skip to main content





Grab your spot at the free arXiv Accessibility Forum
Forum Schedule

We gratefully acknowledge support fromthe Simons Foundation, Stockholm University, and all contributors. Donate





 > cs > arXiv:2406.18312v1
  





Help | Advanced Search




All fields
Title
Author
Abstract
Comments
Journal reference
ACM classification
MSC classification
Report number
arXiv identifier
DOI
ORCID
arXiv author ID
Help pages
Full text




Search















open search






GO



open navigation menu


quick links

Login
Help Pages
About












Computer Science > Computation and Language


arXiv:2406.18312v1 (cs)
    




  [Submitted on 26 Jun 2024 (this version), latest version 19 Jul 2024 (v2)]
Title:AI-native Memory: A Pathway from LLMs Towards AGI
Authors:Jingbo Shang, Zai Zheng, Xiang Ying, Felix Tao, Mindverse Team View a PDF of the paper titled AI-native Memory: A Pathway from LLMs Towards AGI, by Jingbo Shang and 4 other authors
View PDF

Abstract:Large language models (LLMs) have demonstrated the world with the sparks of artificial general intelligence (AGI). One opinion, especially from some startups working on LLMs, argues that an LLM with nearly unlimited context length can realize AGI. However, they might be too optimistic about the long-context capability of (existing) LLMs -- (1) Recent literature has shown that their effective context length is significantly smaller than their claimed context length; and (2) Our reasoning-in-a-haystack experiments further demonstrate that simultaneously finding the relevant information from a long context and conducting (simple) reasoning is nearly impossible. In this paper, we envision a pathway from LLMs to AGI through the integration of \emph{memory}. We believe that AGI should be a system where LLMs serve as core processors. In addition to raw data, the memory in this system would store a large number of important conclusions derived from reasoning processes. Compared with retrieval-augmented generation (RAG) that merely processing raw data, this approach not only connects semantically related information closer, but also simplifies complex inferences at the time of querying. As an intermediate stage, the memory will likely be in the form of natural language descriptions, which can be directly consumed by users too. Ultimately, every agent/person should have its own large personal model, a deep neural network model (thus \emph{AI-native}) that parameterizes and compresses all types of memory, even the ones cannot be described by natural languages. Finally, we discuss the significant potential of AI-native memory as the transformative infrastructure for (proactive) engagement, personalization, distribution, and social in the AGI era, as well as the incurred privacy and security challenges with preliminary solutions.
    



Subjects:

Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Cite as:
arXiv:2406.18312 [cs.CL]


 
(or 
arXiv:2406.18312v1 [cs.CL] for this version)
          
 
 

https://doi.org/10.48550/arXiv.2406.18312



Focus to learn more




                arXiv-issued DOI via DataCite
              







Submission history From: Jingbo Shang [view email]       [v1]
        Wed, 26 Jun 2024 12:51:37 UTC (441 KB)
[v2]
        Fri, 19 Jul 2024 02:37:42 UTC (445 KB)



 

Full-text links:
Access Paper:


View a PDF of the paper titled AI-native Memory: A Pathway from LLMs Towards AGI, by Jingbo Shang and 4 other authorsView PDFOther Formats


view license


 
    Current browse context: cs.CL


< prev

  |   
next >


new
 | 
recent
 | 2024-06

    Change to browse by:
    
cs
cs.AI




References & Citations

NASA ADSGoogle Scholar
Semantic Scholar




a
export BibTeX citation
Loading...




BibTeX formatted citation
×


loading...


Data provided by: 




Bookmark





 




Bibliographic Tools

Bibliographic and Citation Tools






Bibliographic Explorer Toggle



Bibliographic Explorer (What is the Explorer?)







Litmaps Toggle



Litmaps (What is Litmaps?)







scite.ai Toggle



scite Smart Citations (What are Smart Citations?)








Code, Data, Media

Code, Data and Media Associated with this Article






Links to Code Toggle



CatalyzeX Code Finder for Papers (What is CatalyzeX?)







DagsHub Toggle



DagsHub (What is DagsHub?)







GotitPub Toggle



Gotit.pub (What is GotitPub?)







Links to Code Toggle



Papers with Code (What is Papers with Code?)







ScienceCast Toggle



ScienceCast (What is ScienceCast?)











Demos

Demos






Replicate Toggle



Replicate (What is Replicate?)







Spaces Toggle



Hugging Face Spaces (What is Spaces?)







Spaces Toggle



TXYZ.AI (What is TXYZ.AI?)








Related Papers

Recommenders and Search Tools






Link to Influence Flower



Influence Flower (What are Influence Flowers?)







Connected Papers Toggle



Connected Papers (What is Connected Papers?)







Core recommender toggle



CORE Recommender (What is CORE?)





Author
Venue
Institution
Topic














        About arXivLabs
      



arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.










Which authors of this paper are endorsers? |
    Disable MathJax (What is MathJax?)
    












About
Help





contact arXivClick here to contact arXiv
 Contact


subscribe to arXiv mailingsClick here to subscribe
 Subscribe











Copyright
Privacy Policy




Web Accessibility Assistance


arXiv Operational Status 
                    Get status notifications via
                    email
                    or slack





 





