WNBF: reducing hallucination and boosting generalization in LLMs via weighted norm-based filtering

Abstract

Hallucinations in Large Language Models (LLMs) remain a critical challenge, particularly in applications demanding factual accuracy and knowledge consistency. While existing methods, such as retrieval-based augmentation and fine-tuning, have improved factual correctness, they often require extensive external resources, making them difficult to scale and adapt to diverse reasoning tasks. This limits their generalization across commonsense knowledge domains. This paper introduces a lightweight and scalable method, Weighted Norm-Based Filtering (WNBF), which exploits the latent potential of attention head norms to enhance factual accuracy and knowledge generalization. WNBF dynamically assigns weights to truthcorrelated attention heads and applies an adaptive thresholding mechanism, enabling reliable output filtering without additional retrieval or retraining. This inferenceonly approach ensures efficiency while significantly mitigating hallucinations. On TruthfulQA, CommonsenseQA2, and HellaSwag, WNBF achieves substantial improvements, consistently enhancing factual reliability and reasoning capabilities. Notably, WNBF surpasses the current state-of-the-art (SOTA) by in average 1.5% across different LLMs, demonstrating robust generalization across multiple domains. By leveraging attention head norms, WNBF opens new directions in LLM interpretability, robustness, and factual reliability, paving the way for more trustworthy AI systems.