Abstract
Large Language Models (LLMs) have catalyzed significant advancements inNatural Language Processing (NLP), yet they encounter challenges such ashallucination and the need for domain-specific knowledge. To mitigate these,recent methodologies have integrated information retrieved from externalresources with LLMs, substantially enhancing their performance across NLPtasks. This survey paper addresses the absence of a comprehensive overview onRetrieval-Augmented Language Models (RALMs), both Retrieval-AugmentedGeneration (RAG) and Retrieval-Augmented Understanding (RAU), providing anin-depth examination of their paradigm, evolution, taxonomy, and applications.The paper discusses the essential components of RALMs, including Retrievers,Language Models, and Augmentations, and how their interactions lead to diversemodel structures and applications. RALMs demonstrate utility in a spectrum oftasks, from translation and dialogue systems to knowledge-intensiveapplications. The survey includes several evaluation methods of RALMs,emphasizing the importance of robustness, accuracy, and relevance in theirassessment. It also acknowledges the limitations of RALMs, particularly inretrieval quality and computational efficiency, offering directions for futureresearch. In conclusion, this survey aims to offer a structured insight intoRALMs, their potential, and the avenues for their future development in NLP.The paper is supplemented with a Github Repository containing the surveyedworks and resources for further study:https://github.com/2471023025/RALM_Survey.