Introduction

Understanding how people exploit nonverbal aspects of their communication to coordinate their activities and social relationships is a fundamental scientific challenge. Deeper insights into nonverbal communication can have a profound impact on how we link theories of perception, learning, cognition and action to models of interactions and groups at the social level. Models of nonverbal behaviors in interaction are essential for collaboration tools, human-computer and virtual interaction and other assistive technologies designed to support people in real-world activities. This knowledge is also useful to develop models of the deficits of specific populations, such as autistic children, and interventions that bring them into fuller participation in communities. In general, nonverbal communication research offers high-level principles that might explain how people organize, display, adapt and understand such behaviors for communicative purposes and social goals. However, the specifics are generally not fully understood, nor is the way to translate these principles into algorithms and computer-aided communication technologies such as intelligent agents.

To model such complex dynamic processes effectively, novel computer vision and learning algorithms are needed that take into account both the heterogeneity and the dynamicity intrinsic to behavior data. As one of the most active research areas in computer vision, human motion analysis has become a widely-used tool in this area. It uses image sequences to detect and track people, and also to interpret human activities. Emerging automated methods for analyzing motion have been studied and developed to enable tracking diverse human movements precisely and robustly as well as correlating multiple people's movements in interaction. Some of the applications of using motion analysis methods for Nonverbal Communication Computing include deception detection, expression recognition, sign language recognition, behavior analysis, and group activity recognition.

Downloads

• Code for face tracking


• Dataset and Annotation Tools for American Sign Language (Boston University)


• Slides

BibTeX

@article{Metaxas2013421,
title = "A review of motion analysis methods for human Nonverbal Communication Computing ",
journal = "Image and Vision Computing ",
volume = "31",
number = "6-7",
pages = "421-433",
year = "2013",
author = "Dimitris Metaxas and Shaoting Zhang",
}

Relevant publications

Dimitris Metaxas, Shaoting Zhang: A Review of Motion Analysis Methods for Human Nonverbal Communication Computing. Image and Vision Computing, special issue on Machine learning in motion analysis: New advances, Volume 31, Issues 6-7, Pages 421-433, 2013.

Xiang Yu, Junzhou Huang, Shaoting Zhang, Wang Yan, Dimitris Metaxas: Pose-free Facial Landmark Fitting via Optimized Part Mixtures and Deformable Shape Model. International Conference on Computer Vision, 2013.

Jingjing Liu, Bo Liu, Shaoting Zhang, Fei Yang, Peng Yang, Dimitris N. Metaxas and Carol Neidle: Recognizing Eyebrow and Periodic Head Gestures Using CRFs for Non-Manual Grammatical Marker Detection in ASL. IEEE International Conference on Automatic Face and Gesture Recognition, 2013. oral presentation.

X Yu, S Zhang, Z Yan, F Yang, J Huang, N Dunbar, M Jensen, J Burgoon, D Metaxas: Is Interactional Dissynchrony a Clue to Deception: Insights from Automated Analysis of Nonverbal Visual Cues. Hawaii International Conference on System Sciences (HICSS), 2013