Search

Multimodal Patterns of Speech Disfluencies Compensators (Based on Student Dialogic Discourse) // Tomsk State Pedagogical University Bulletin. 2023. Issue 1 (225). P. 9-18

Speech as a multidimensional phenomenon still remains relevant in linguistics studies. It is known that verbal communication evolved later than non-verbal one, however, a multimodal approach to language communication modeling based on spontaneous dialogues is only gaining momentum in Russian linguistics studies. The aim of the article is to provide data on principles of verbal and non-verbal correlation at times of difficulty during the development of spoken discourse between two speakers, as well as to systematize them in the most frequent multimodal patterns. The material of the article is 16 video clips of spontaneous dialogues of Russian native speakers, the total timing is 1.5 hours. The annotated video materials are students’ spontaneous dialogic discourse. The topics of dialogues can be divided into two groups – communication between friends/colleagues and communication between the client and service staff. Each video clip is annotated in ELAN v.6.2. The presented results are based on general linguistic methods of description and comparison, as well as a method of analyzing a cognitive event. Analyzing cognitive events and providing statistics on the subject, we describe the main multimodal patterns, which have been classified by the frequent nature of some speech disfluencies and non-verbal response to them. Our study is based on the gestural repetition concept. As a result, the most common speech disfluencies have been identified and divided into five large blocks. Each of the blocks is accompanied by the main multimodal patterns, implemented in different channels – manual channel, gaze channel and cephalic channel. The number of speech disfluencies recorded at the external level directly depends on the topic and not always on the level of competence of the speakers, since all dialogues are improvised by students who only have presuppositions regarding the deployment of discourse depending on the given role. Speech disfluencies and the principles of their compensation in different multimodal channels depend on both internal (cognitive) and external factors (overlapping and interruptions), which represent an interpenetrating indivisible system.

Keywords: multimodal pattern, speech disfluency, manual channel, gaze channel, cephalic channel, cognitive event, overlapping