Best Student Paper Award @ IVA 2016
2016.September.23:
Randy Zhao and Tanmay Sinha, PhD students at Language Technologies Institute in the School of Computer Science, won the best student paper award at 16th International Conference on Intelligent Virtual Agents (IVA) for their research on automatic estimation of interpersonal rapport using temporally co-occurring and contingent patterns of multimodal behavior (link to the paper)
By mining a reciprocal peer tutoring corpus reliably annotated for nonverbals like eye gaze and smiles, conversational strategies like self-disclosure and social norm violation, and for rapport (in 30 second thin slices), this work performs fine-grained investigation of how the temporal profiles of sequences of interlocutor behaviors predict increases and decreases of rapport, and how this rapport management manifests differently in friends and strangers. The work also validates the discovered behavioral patterns by predicting rapport against ground truth via a forecasting model involving two-step fusion of learned temporal associated rules. The framework was shown to have significant performance improvement, compared to a baseline linear regression method that does not encode temporal information among behavioral features.
These results have been integrated into a real-time end-to-end socially aware dialog system (SARA). SARA is capable of automatically detecting conversational strategies based on verbal, nonverbal, and acoustic features in the user’s input, relying on the conversational strategies detected in order to accurately estimate rapport between the interlocutors, reasoning about what conversational strategy to respond with as the next turn, and generating those appropriate responses in the service of more effectively carrying out her task duties. To our knowledge, SARA is the first socially-aware dialog system that relies on visual, verbal, and vocal cues to detect user social and task intent, and generates behaviors in those same channels to achieve her social and task goals.
Categories :
Archives :