The cognitive effectiveness of subtitle processing

In an experimental study, we analyzed the cognitive processing of a subtitled film excerpt by adopting a new approach based on the integration of a variety of measures: eye-movement data, word recognition, and visual scene recognition. We tested the hypothesis that the cognitive processing of subtitles is cognitively efficient, i.e., it does not require a significant trade-off between image processing and text processing. Following indications in the psycholinguistic literature, we also tested the hypothesis that two-line subtitles whose segmentation is syntactically incoherent can have a disruptive effect on information processing and recognition performance. The results highlighted the efficiency of subtitle processing, regardless of the quality of line segmentation: participants achieved good levels of performance in both word and scene recognition, and no trade-off was detected. Eye-movement analysis also enabled us to further characterize the processing of subtitled films.