Автоматизована перевірка тверджень з використанням RAG-механізму та класифікації ознак

Abstract

The object of the study is the problem of automatic fact verification in a digital environment saturated with disinformation. The paper analyzes modern approaches to fake news detection, including transformer architectures, neurosemantic and graph models. Additionally, the limitations of existing methods are identified, in particular, the popularity of the use of static features and poor generalization ability in a constant dynamic flow of information. The author proposes his own architecture of a multimodal model that combines style classification, AI text detection and a fact-checking module, supported by the search for relevant evidence through the RAG mechanism. The results of experiments on a test set of 1660 examples showed that the model achieves a high Recall indicator (84.6 %), while maintaining an acceptable balance of accuracy (Accuracy – 78.6 %, Precision – 74.4 %, F1 – 80.8 %). The obtained results indicate sufficient effectiveness of multi-task learning in truth-checking systems. This allows for effective detection of fake news from various sources, albeit with a certain number of false positives, but the balance between high Recall and lower Precision is justified, since the system is focused on reducing the possibility of missing fake news. The proposed model is suitable for use in real-world monitoring of the information space, in particular in the context of countering information threats. The effectiveness of the model is explained by the combination of several independent features (style, origin, factuality) and a flexible signal integration system. In addition, the use of the RAG mechanism provides an additional level of interpretability of the results obtained with reference to external sources. It can be used in online platforms with a large number of unstructured messages. The approach can be expanded with multimedia analysis and adapted for another specific language environment.