Research Article
Evaluating Precision and Recall at Retrieval Time in Retrieval-Augmented Generation (RAG) Systems
Gopichand Agnihotram*
,
Joydeep Sarkar
Issue:
Volume 8, Issue 4, December 2025
Pages:
174-180
Received:
12 September 2025
Accepted:
23 September 2025
Published:
18 October 2025
Abstract: Retrieval-Augmented Generation (RAG) systems signify a pivotal advancement in natural language processing, merging information retrieval with large language models (LLMs) to ground responses in external knowledge. This hybrid approach enhances the factual accuracy and currency of generated content, mitigating common issues like hallucination. The efficacy of a RAG system, however, is fundamentally dependent on the performance of its retrieval component. This paper provides a detailed analysis of precision and recall as critical metrics for evaluating and optimizing this retrieval step. We explore the distinct roles and inherent trade-offs of these metrics within a RAG pipeline, demonstrating their direct influence on the quality of the final output. Through a series of experiments comparing sparse (BM25), dense (DPR), and hybrid retrieval methods, we quantify their performance characteristics. The analysis is further enriched with real-world examples from finance, law, and healthcare, illustrating the practical implications of retrieval quality. Additionally, we outline advanced strategies for improving retrieval effectiveness, such as multi-stage architecture involving rerankers and the use of query transformations. The paper concludes with a set of best practices for deploying robust, enterprise-grade RAG systems, emphasizing the need for continuous evaluation and sophisticated retrieval strategies. By focusing on the systematic optimization of precision and recall, organizations can build more reliable and trustworthy AI applications.
Abstract: Retrieval-Augmented Generation (RAG) systems signify a pivotal advancement in natural language processing, merging information retrieval with large language models (LLMs) to ground responses in external knowledge. This hybrid approach enhances the factual accuracy and currency of generated content, mitigating common issues like hallucination. The e...
Show More
Research Article
Beyond Static Retrieval: A Reinforcement Learning Framework for Dynamic and Adaptive RAG
Gopichand Agnihotram,
Joydeep Sarkar,
Magesh Kasthuri*
Issue:
Volume 8, Issue 4, December 2025
Pages:
181-188
Received:
12 September 2025
Accepted:
23 September 2025
Published:
18 October 2025
Abstract: Retrieval-Augmented Generation (RAG) is a widely adopted technique that enhances large language models (LLMs) by grounding their outputs in external knowledge sources. This approach reduces hallucinations, increases factual accuracy, and adapts well to rapidly evolving domains. Despite these strengths, traditional RAG implementations rely on static, heuristic-based retrieval strategies that operate independently of feedback or contextual learning. In today’s fast-changing information landscape, it’s crucial for language models to go beyond static retrieval when grounding their responses. That’s where a RL framework comes into play for RAG. Rather than sticking to fixed, rule-based selection methods, RL allows the retrieval component to learn and adapt over time—much like how a person refines their search strategies with experience and feedback. By framing the process of document selection as a Markov Decision Process (MDP), the system can make context-aware choices that consider both immediate and future gains. This white paper explores how Retrieval-Augmented Generation can be significantly enhanced by integrating Markov Decision Processes (MDPs) and Reinforcement Learning (RL). We present a conceptual framework that models retrieval as a sequential decision-making problem. By treating document selection as an MDP and employing RL algorithms to optimize retrieval strategies, we introduce adaptivity, context sensitivity, and long-term reasoning into the RAG pipeline, leading to demonstrably more accurate and relevant generated content. The paper also outlines applications, implementation strategies, and future research directions that combine symbolic and neural methods for improved decision-making and document relevance.
Abstract: Retrieval-Augmented Generation (RAG) is a widely adopted technique that enhances large language models (LLMs) by grounding their outputs in external knowledge sources. This approach reduces hallucinations, increases factual accuracy, and adapts well to rapidly evolving domains. Despite these strengths, traditional RAG implementations rely on static...
Show More
Research Article
An Integrated Jacket–Helmet Assistive System for Visually Impaired Individuals Using YOLO-Based Object Detection, Depth Estimation, and OCR
Kashvi Ruparelia,
Priyam Parikh*,
Parth Atulkumar Shah
Issue:
Volume 8, Issue 4, December 2025
Pages:
189-205
Received:
12 September 2025
Accepted:
23 September 2025
Published:
30 October 2025
DOI:
10.11648/j.ajcst.20250804.13
Downloads:
Views:
Abstract: This paper presents the design and evaluation of a jacket–helmet assistive system for visually impaired individuals in India. The system integrates a Raspberry Pi 4B with a USB web camera, USB microphone, vibration motor cluster, earphone, pushbuttons, and a rechargeable 7.4 V, 10,000 mAh battery. Two primary functions are implemented: (i) object detection and distance estimation using YOLO algorithms with 2D depth estimation, and (ii) text recognition on posters and hoardings using optical character recognition (OCR). Comparative analysis of YOLOv5, YOLOv7, and YOLOv8 models demonstrated that YOLOv8 achieved the highest mean Average Precision (mAP) of 92.4%, outperforming YOLOv7 (89.6%) and YOLOv5 (87.3%). For monocular 2D depth estimation, MiDaS achieved the lowest mean absolute relative error (0.124) compared to Monodepth2 (0.156) and DPT (0.139). Speech-to-text efficiency was tested across Google Speech Recognition, Vosk, and CMU Sphinx, with Google achieving 94.1% accuracy, followed by Vosk (88.3%) and CMU Sphinx (81.6%). User trials were conducted with ten visually impaired individuals across diverse environments (bus stand, garden, bungalow, and home settings). System usability was measured using the System Usability Scale (SUS), yielding an overall average score of 84.6, indicating “excellent” usability. The proposed system demonstrates high accuracy, robustness, and practicality for real-world navigation and reading assistance, thus contributing to improved autonomy and quality of life for visually impaired users.
Abstract: This paper presents the design and evaluation of a jacket–helmet assistive system for visually impaired individuals in India. The system integrates a Raspberry Pi 4B with a USB web camera, USB microphone, vibration motor cluster, earphone, pushbuttons, and a rechargeable 7.4 V, 10,000 mAh battery. Two primary functions are implemented: (i) object d...
Show More
