A Framework for Hallucination-Mitigating Manufacturing Anomaly Detection

Visual anomaly detection techniques are used to detect various defects that occur during the manufacturing process by identifying abnormal patterns in images. In particular, anomaly detection techniques based on large vision-language models can detect anomalies without thresholds and provide interpretation of the detection results. However, they cannot perform pixel-level fine defect judgement and lack expert domain knowledge. These two limitations lead to hallucination when it comes to justifying defect detection. In this paper, we propose a novel framework to mitigate hallucinations by utilizing anomaly maps containing defect locations and retrieval augmented generation-based knowledge augmentation. In various experiments using several benchmark datasets, we demonstrate that the proposed framework can generate more reliable defect judgment rationale with 50.2% hallucination reduction compared to the existing method (GPT-LAD) and up to 54.1% compared to the baseline method (GPT-4o).