We present and discuss a fully-automated collaboration system, CoCo, that allows multiple participants to video chat and receive real-time and post feedback.

We present two different variations of improving team dynamics during a video call.

In the first version, we developed a virtual feedback assistant (CoCo) that provides insights into the conversation to participants. CoCo automatically pulls audial and visual data during conversations and analyzes the extracted streams for affective features, including smiles, engagement, attention, speech overlap, and turn-taking. In an experiment with 39 participants,  CoCo can create balanced participation; that is, everyone spoke for an equal amount of time.

In the second version, we developed a video chat framework that can automatically analyze audio-video data of the participants and provide real-time feedback on participation, interruption, volume, and facial emotion. We analyze the immediate and reflective effects of real-time feedback on participants. Our findings show that while real-time feedback can make the ongoing discussion significantly less spontaneous, its effects propagate to successive sessions bringing significantly more expressiveness
to the team.

Related papers

S. Samrose, E. Hoque, Suggestive Motivational Interviewing Agent for Remote Group Discussion, ACM International Conference on Supporting Group Work (Group 22), Jan 2022.

S. Samrose, R. Zhao, J. White, V. Li, L. Nova, Y. Lu, M. R. Ali and M. E. Hoque. CoCo: Collaboration Coach for Understanding Team Dynamics during Video Conferencing. In the Proceedings of the ACM Interactive, Mobile, Wearable, and Ubiquitous Computing (UbiComp-IMWUT), Singapore, 2018.

S. Samrose, R. Rawassizadeh, E. Hoque, Immediate or Reflective?: Effects of Real-time Feedback on Group Discussions over Videochat, arxiv

Code

The code is now released:
https://github.com/ROC-HCI/CollaborationCoach_PostFeedback

This research is funded by the National Science Foundation and Google.