A Hybrid AI System for Fusion of Object and Context Information: Application to the Rail Line Defect Detection

A hybrid artificial intelligence (Hybrid AI) which represents a convergence of a classical (symbolic) AI with recent machine learning approaches has become a very quickly developing research axis. The combination of rule-based reasoning and statistical learning is required whenever the domain knowledge has to be incorporated in the decision system. In this work we present a system on the basis of Deep Neural Networks (DNNs) as object detectors, such as You Only Look Once version 8 (YOLOv8), transformers and logical rules which link objects and their context in the problem of rail line defect detection. Fusion of information is performed at the intermediate level - in the feature space, mixing sets of elements of this space delimited due to the object and context element detectors. Combination of objects and context elements is performed accordingly to the domain-defined rules, and fusion is ensured by a vision transformer. Experiments have been conducted on the domainrecorded dataset of rail defects. The proposed hybrid system outperforms base-line objects detection up to 0.28 of accuracy increase.