LISSA – Live Interactive Social Skills Assistance
The LISSA system aims to help individuals improve their conversational skills by simulating face-to-face conversation and providing both real-time as well as post-session summary feedback on users’ nonverbal behavior. At the same time, it manages a fairly open-ended conversation with some degree of understanding. We have developed an HMM based model for generating real-time feedback.
We evaluated our system using a speed-dating study design. We invited 47 undergraduate male students to interact with staff and randomly assigned them to intervention with LISSA or a selfhelp control group. Results suggested participants who practiced with the LISSA system were rated as significantly better in nodding when compared to a self-help control group and marginally better in eye contact and gesturing. The system usability and surveys showed that participants found the feedback provided by the system useful, unobtrusive, and easy to understand
Related papers:
- Ali, M.R.; Crasta, D.; Li Jin; Baretto, A.; Pachter, J.; Rogge, R.D.; Hoque, M.E., “LISSA- Live Interactive Social Skill Assistance,” International Conference on Affective Computing and Intelligent Interaction (ACII), 2015, pp.173-179, 21-24 Sept. 2015
- M. Rafayet. Ali, Taylan Sen, Dev Crasta, Viet-Duy Nguyen, Ronald Rogge, M Ehsan Hoque, “What, When, and Why of Facial Expressions: An Objective Analysis of Conversational Skills in Speed-Dating Videos“, 2018 IEEE Conference on Automatic Face and Gesture Recognition (FG 2018).
- S. Z. Razavi, M. Ali, T. Smith, L. Schubert, M. E. Hoque, The LISSA Virtual Human and ASD Teens: An Overview of Initial Experiments, 16th International Conference on Intelligent Virtual Agent (IVA), Los Angeles, California, September 2016.
- S. Z. Razavi, L. Schubert, M. Ali, M. E. Hoque, Managing Causal Spoken Dialogue Using Flexible Schemas, Pattern Transduction Trees, and Gist Clauses, 5th Annual Conference on Advances in Cognitive Systems (ACS), Troy, NY, May 2017
- M. R. Ali, S. Z. Razavi, R. Langevin, A. A. Mamun, B. Kane, R. Rawassizadeh, L. Schubert, E. Hoque, A Virtual Conversational Agent for Teens with Autism Spectrum Disorder: Experimental Results and Design Lessons, 20th ACM International Conference on Intelligent Virtual Agent (IVA), October 2020
Code
The LISSA code is now released:
https://github.com/mali7/lissa
Contributions appreciated.