The LISSA system aims to help individuals improve their conversational skills by simulating face-to-face conversation and providing both real-time as well as post-session summary feedback on users’ nonverbal behavior. At the same time, it manages a fairly open-ended conversation with some degree of understanding. We have developed an HMM based model for generating real-time feedback.

We evaluated our system using a speed-dating study design. We invited 47 undergraduate male students to interact with staff and randomly assigned them to intervention with LISSA or a selfhelp control group. Results suggested participants who practiced with the LISSA system were rated as significantly better in nodding when compared to a self-help control group and marginally better in eye contact and gesturing. The system usability and surveys showed that participants found the feedback provided by the system useful, unobtrusive, and easy to understand

Related papers:

Code

The LISSA code is now released:
https://github.com/mali7/lissa

Contributions appreciated.