Logo del repository
  1. Home
 
Opzioni

The Magnitude of Truth: On Using Magnitude Estimation for Truthfulness Assessment

Soprano, Michael
•
Tapu, Denis Eduard
•
La Barbera, David
altro
Mizzaro, Stefano
2025
  • conference object

Abstract
Evaluating the truthfulness of online content is critical for combating misinformation. This study examines the efficiency and effectiveness of crowdsourced truthfulness assessments through a comparative analysis of two approaches: one involving full-length webpages as evidence for each claim, and another using summaries for each evidence document generated with an LLM. Using an A/B testing setting, we engage a diverse pool of participants tasked with evaluating the truthfulness of statements under these conditions.Our analysis explores both the quality of assessments and the behavioral patterns of participants. The results reveal that relying on summarized evidence offers comparable accuracy and error metrics to the standard modality while significantly improving efficiency. Workers in the Summary setting complete a significantly higher number of assessments, reducing task duration and costs. Additionally, the Summary modality maximizes internal agreement and maintains consistent reliance on and perceived usefulness of evidence, demonstrating its potential to streamline large-scale truthfulness evaluations.
DOI
10.1145/3726302.3730091
Archivio
https://hdl.handle.net/11390/1309147
info:eu-repo/semantics/altIdentifier/scopus/2-s2.0-105011817014
https://dl.acm.org/doi/10.1145/3726302.3730091
Diritti
open access
license:creative commons
license uri:http://creativecommons.org/licenses/by/4.0/
Soggetti
  • crowdsourcing, fact-c...

google-scholar
Get Involved!
  • Source Code
  • Documentation
  • Slack Channel
Make it your own

DSpace-CRIS can be extensively configured to meet your needs. Decide which information need to be collected and available with fine-grained security. Start updating the theme to match your nstitution's web identity.

Need professional help?

The original creators of DSpace-CRIS at 4Science can take your project to the next level, get in touch!

Realizzato con Software DSpace-CRIS - Estensione mantenuta e ottimizzata da 4Science

  • Impostazioni dei cookie
  • Informativa sulla privacy
  • Accordo con l'utente finale
  • Invia il tuo Feedback