The effects of emotional films and subtitle types on eye movement patterns

Background: In Taiwan, the use of subtitle is common in TV programs and movies. However, studies on subtitles mostly focus on foreign language learning and film subtitle translation. Few studies address how subtitle types and emotion-laden films affect the viewers' eye movement patterns. Purpose: We aim to examine how the emotion type of film (happy, sad, angry, fear, or neutral) and subtitle type (meaningful subtitle, no subtitle, or meaningless subtitle) affect the dwell times and fixation counts in the subtitle area. Methods: This study is a 5 (emotion type of film) × 3 (subtitle type) between-participants design. There were 15 participants per condition, resulting in a total of 225 participants. After watching a film, participants filled out a self-reported questionnaire regarding this film. Results: The subtitled films have more fixation counts and dwell time for the meaningful subtitle compared to meaningless subtitle and no subtitle. The dwell time was longer on the subtitle area for the sad film than the neutral and happy films. Also, the dwell time was longer on the subtitle area for the fear film than the happy film. There were more fixation counts on the subtitle area for the sad film than the angry and happy films. Conclusions: The subtitle meaning is critical in directing overt attention. Also, overt attention directed to the subtitle area is affected by the different emotion types of films.