Brings together a great many contributions regarding aspects of natural and multimodal interaction written by many of the important actors in the field
A timely update of Multimodality in Language and Speech Systems by Björn Granström, David House and Inger Karlsson and, at the same time, it presents a much broader overview of the field
Its 17 chapters provide a broad and detailed impression of where the fairly new field of natural and multimodal interactivity engineering stands today
The main topic of this volume is natural multimodal interaction. The book is unique in that it brings together a great many contributions regarding aspects of natural and multimodal interaction written by many of the important actors in the field. Topics addressed include talking heads, conversational agents, tutoring systems, multimodal communication, machine learning, architectures for multimodal dialogue systems, systems evaluation, and data annotation.
|References 74 Part II Annotation and Analysis of Multimodal Data: Speech and Gesture 4 FORM 79 Craig H. Martell 1. Introduction 79 2. Structure of FORM 80 3. Annotation Graphs 85 4. Annotation Example 86 5. Preliminary Inter-Annotator Agreement Results 88 6. Conclusion: Applications to HLT and HCI? 90 Appendix: Other Tools, Schemes and Methods of Gesture Analysis 91 References 95 5 97 On the Relationships among Speech, Gestures, and Object Manipulation in Virtual Environments: Initial Evidence Andrea Corradini and Philip R. Cohen 1. Introduction 97 2. Study 99 3. Data Analysis 101 4. Results 103 5. Discussion 106 6. Related Work 106 7. Future Work 108 8. Conclusions 108 Appendix: Questionnaire MYST III - EXILE 110 References 111 6 113 Analysing Multimodal Communication Patrick G. T. Healey, Marcus Colman and Mike Thirlwell 1. Introduction 113 2. Breakdown and Repair 117 3. Analysing Communicative Co-ordination 125 4. Discussion 126 References 127 7 131 Do Oral Messages Help Visual Search? Noëlle Carbonell and Suzanne Kieffer 1. Context and Motivation 131 2. Methodology and Experimental Set-Up 134 3. Results: Presentation and Discussion 141 4. Conclusion 153 References 154 Contents vii 8 159 Geometric and Statistical Approaches to Audiovisual Segmentation Trevor Darrell, John W. Fisher III, Kevin W. Wilson, and Michael R. Siracusa 1. Introduction 159 2. Related Work 160 3. Multimodal Multisensor Domain 162 4. Results 166 5. Single Multimodal Sensor Domain 167 6.
Preface Natural and Multimodal Interactivity Engineering - Directions and Needs / NIELS OLE BERNSEN and LAILA DYBKJÆR Part I Making Dialogues More Natural: Empirical Work and Applied Theory Social Dialogue with Embodied Conversational Agents / TIMOTHY BICKMORE and JUSTINE CASSELL A First Experiment in Engagement for Human-Robot Interaction in Hosting Activities / CANDACE L. SIDNER and MYROSLAVA DZIKOVSKA Part II Annotation and Analysis of Multimodal Data: Speech and Gesture FORM / CRAIG H. MARTELL On the Relationships among Speech, Gestures, and Object Manipulation in Virtual Environments: Initial Evidence / ANDREA CORRADINI and PHILIP R. COHEN Analysing Multimodal Communication / PATRICK G. T. HEALEY, MARCUS COLMAN and MIKE THIRLWELL Do Oral Messages Help Visual Search? / NOËLLE CARBONELL amd SUZANNE KIEFFER Geometric and Statistical Approaches to Audiovisual Segmentation / TREVOR DARRELL, JOHN W. FISHER III, KEVIN W. WILSON, and MICHAEL R. SIRACUSA Part III Animated Talking Heads and Evaluation The Psychology and Technology of Talking Heads: Applications in Language Learning / DOMINIC W. MASSARO Effective Interaction with Talking Animated Agents in Dialogue Systems / BJÖRN GRANSTRÖM and DAVID HOUSE Controlling the Gaze of Conversational Agents / DIRK HEYLEN, IVO VAN ES, ANTON NIJHOLT and BETSY VAN DIJK Part IV Architectures and Technologies for Advanced and Adaptive Multimodal Dialogue Systems MIND: A Context-Based Multimodal Interpretation Framework in Conversational Systems / JOYCE Y. CHAI, SHIMEI PAN and MICHELLE X. ZHOU A General Purpose Architecture for Intelligent Tutoring Systems / BRADY CLARK, OLIVER LEMON, ALEXANDER GRUENSTEIN, ELIZABETH OWEN BRATT, JOHN FRY, STANLEY PETERS,HEATHER PON-BARRY, KARL SCHULTZ, ZACK THOMSEN-GRAY and PUCKTADA TREERATPITUK MIAMM – A Multimodal Dialogue System using Haptics / NORBERT REITHINGER, DIRK FEDELER, ASHWANI KUMAR, CHRISTOPH LAUER, ELSA PECOURT and LAURENT ROMARY Adaptive Human-Computer Dialogue / SORIN DUSAN and JAMES FLANAGAN Machine Learning Approaches to Human Dialogue Modelling / YORICK WILKS, NICK WEBB, ANDREA SETZER, MARK HEPPLE and ROBERTA CATIZONE Index
Natural and Multimodal Interactivity Engineering - Directions and Needs.- Natural and Multimodal Interactivity Engineering - Directions and Needs.- Making Dialogues More Natural: Empirical Work and Applied Theory.- Social Dialongue with Embodied Conversational Agents.- A First Experiment in Engagement for Human-Robot Interaction in Hosting Activities.- Annotation and Analysis of Multimodal Data: Speech and Gesture.- Form.- On the Relationships Among Speech, Gestures, and Object Manipulation in Virtual Environments: Initial Evidence.- Analysing Multimodal Communication.- Do Oral Messages Help Visual Search?.- Geometric and Statistical Approaches to Audiovisual Segmentation.- Animated Talking Heads and Evaluation.- The Psychology and Technology of Talking Heads: Applications in Language Learning.- Effective Interaction with Talking Animated Agents an Dialogue Systems.- Controlling the Gaze of Conversational Agents.- Architectures and Technologies for Advanced and Adaptive Multimodal Dialogue Systems.- Mind: A Context-Based Multimodal Interpretation Framework in Conversational Systems.- A General Purpose Architecture for Intelligent Tutoring Systems.- Miamm - A Multimodal Dialogue System Using Haptics.- Adaptive Human-Computer Dialogue.- Machine Learning Approaches to Human Dialogue Modelling.
Inhaltsverzeichnis
Natural and Multimodal Interactivity Engineering - Directions and Needs.- Natural and Multimodal Interactivity Engineering - Directions and Needs.- Making Dialogues More Natural: Empirical Work and Applied Theory.- Social Dialongue with Embodied Conversational Agents.- A First Experiment in Engagement for Human-Robot Interaction in Hosting Activities.- Annotation and Analysis of Multimodal Data: Speech and Gesture.- Form.- On the Relationships Among Speech, Gestures, and Object Manipulation in Virtual Environments: Initial Evidence.- Analysing Multimodal Communication.- Do Oral Messages Help Visual Search?.- Geometric and Statistical Approaches to Audiovisual Segmentation.- Animated Talking Heads and Evaluation.- The Psychology and Technology of Talking Heads: Applications in Language Learning.- Effective Interaction with Talking Animated Agents an Dialogue Systems.- Controlling the Gaze of Conversational Agents.- Architectures and Technologies for Advanced and Adaptive Multimodal Dialogue Systems.- Mind: A Context-Based Multimodal Interpretation Framework in Conversational Systems.- A General Purpose Architecture for Intelligent Tutoring Systems.- Miamm - A Multimodal Dialogue System Using Haptics.- Adaptive Human-Computer Dialogue.- Machine Learning Approaches to Human Dialogue Modelling.
Klappentext
References 74 Part II Annotation and Analysis of Multimodal Data: Speech and Gesture 4 FORM 79 Craig H. Martell 1. Introduction 79 2. Structure of FORM 80 3. Annotation Graphs 85 4. Annotation Example 86 5. Preliminary Inter-Annotator Agreement Results 88 6. Conclusion: Applications to HLT and HCI? 90 Appendix: Other Tools, Schemes and Methods of Gesture Analysis 91 References 95 5 97 On the Relationships among Speech, Gestures, and Object Manipulation in Virtual Environments: Initial Evidence Andrea Corradini and Philip R. Cohen 1. Introduction 97 2. Study 99 3. Data Analysis 101 4. Results 103 5. Discussion 106 6. Related Work 106 7. Future Work 108 8. Conclusions 108 Appendix: Questionnaire MYST III - EXILE 110 References 111 6 113 Analysing Multimodal Communication Patrick G. T. Healey, Marcus Colman and Mike Thirlwell 1. Introduction 113 2. Breakdown and Repair 117 3. Analysing Communicative Co-ordination 125 4. Discussion 126 References 127 7 131 Do Oral Messages Help Visual Search? Noëlle Carbonell and Suzanne Kieffer 1. Context and Motivation 131 2. Methodology and Experimental Set-Up 134 3. Results: Presentation and Discussion 141 4. Conclusion 153 References 154 Contents vii 8 159 Geometric and Statistical Approaches to Audiovisual Segmentation Trevor Darrell, John W. Fisher III, Kevin W. Wilson, and Michael R. Siracusa 1. Introduction 159 2. Related Work 160 3. Multimodal Multisensor Domain 162 4. Results 166 5. Single Multimodal Sensor Domain 167 6.
Brings together a great many contributions regarding aspects of natural and multimodal interaction written by many of the important actors in the field
A timely update of Multimodality in Language and Speech Systems by Björn Granström, David House and Inger Karlsson and, at the same time, it presents a much broader overview of the field
Its 17 chapters provide a broad and detailed impression of where the fairly new field of natural and multimodal interactivity engineering stands today
