Capture of focal multidimensional data in human and LLM-based fact-check summaries - Macák, Matouš; Svátek, Vojtěch, 1967 prosinec 1.-; Chudán, David; Vysoká škola ekonomická v Praze. Fakulta informatiky a statistiky (2025)
In the contemporary information landscape, marked by a surge in data and the rise of "post-information age" challenges, fact-checking journalism has become essential to combat the spread of disinformation by contextualizing public claims. As the volume of fact-checking articles grows, effective summarization is crucial to make these detailed analyses accessible. However, this presents a significant problem: both human and automated summaries risk omitting critical, multidimensional data—such as time, location, and methodology—thereby undermining the factual grounding and value of the summary. This thesis addresses this challenge by first defining "focal multidimensional data" within fact-checking articles. It then develops a novel suite of custom metrics designed to measure the capture of this data, including Data Point Coverage, Dimensional Completeness, and Weighted Data Point Coverage. This framework was applied to a curated dataset of 14 climate-related fact-checks from PolitiFact and their corresponding summaries generated by the gpt-4o-mini large language model (LLM). Our findings indicate that while LLM-generated summaries achieve high semantic coherence (0.86 BERTScore F1), they fail to preserve the underlying evidence, capturing only 37% of the most critical, weighted data points. The analysis further reveals that while dimensions like subject and source are well-preserved, summaries frequently omit the precise quantitative value and, most critically, the temporal context of the data. This study concludes that standard summarization metrics are insufficient for evaluating content in high-stakes, factual domains. The significant gap between semantic fluency and factual integrity highlights the urgent need for dimension-aware evaluation and the development of summarization models explicitly optimized to preserve the dimensional context that underpins factual claims.
Vedoucí práce: Vojtěch Svátek
Bakalářská práce (Bc.)—Vysoká škola ekonomická v Praze. Fakulta informatiky a statistiky, 2025
Obsahuje bibliografii
Textový (vysokoškolská kvalifikační práce)
Rok obhajoby 2025
Způsob přístupu: Internet
Keywords: data analytics [obor bakal. práce]; bakalářské práce; bachelor's theses