Understanding Credibility

Can you tell when someone is lying to you? Are there specific facial characteristics that may indicate when someone is not being truthful? Can you detect who is lying in the above video? Try your best!

Is there a chance that a computer can aid a human in detecting deceptive behavior? In a series of studies, we have developed a series of algorithms to understand the role of facial expressions, language, and prosody related to credibility. Additionally, we have argued for the ethical use cases of technologies that can unobtrusively measure the credibility of individuals.

Figure 1. Conceptualizing a Mental Trespass. We recommend a general use ban of accurate thought exposing technologies and an offensive use ban of accurate truth metering technologies. The idea of “accuracy” (shown as the dashed bold line in part (a) of the figure) must exceed average human level capabilities to fall under the proposed Mental Trespass Act outlined in this article. (a) Technology definitions and mental trespass concept. (b) Visualizing truth metering. (c) Visualizing thought exposure.

Deception Dataset

If you want access to the dataset described in [1], please fill out this form.

Paper Publications

[1] Sen, T., Hasan, M. K., Teicher, Z., & Hoque, M. E. (2018). Automated Dyadic Data Recorder (ADDR) Framework and Analysis of Facial Cues in Deceptive Communication. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(4), 163

[2] Sen, T., Hasan, M. K., Tran, M., Levin, M., Yang, Y., & Hoque, M. E., Say CHEESE: Common Human Emotional Expression Set Encoder and its Application to Analyze Deceptive Communication, IEEE International Conference on Automatic Face and Gesture Recognition, 2018

[3] Hasan, M.K., Rahman, W., Gerstner, L., Sen, T. K., Lee, S., Haut, K. G., Hoque, M. E. (2019). Facial Expression Based Imagination Index and a Transfer Learning Approach to Detect Deception, 8th International Conference on Affective Computing and Intelligent Interaction (ACII 2019), Cambridge, UK, September 2019

[4] Hasan, M.K., Sen, T. K., Yang, Y., Baten, R. A., Glenn, K. G., and Hoque, M. E. (2019). LIWC Into the Eyes: Using Facial Features to Contextualize Linguistic Analysis in Multimodal Communication, 8th International Conference on Affective Computing and Intelligent Interaction (ACII 2019), Cambridge, UK, September 2019

[5] G. Naven, T. K. Sen, L. Gerstner, K. G. Haut, M. Wen, M. E. Hoque, Leveraging Shared and Divergent Facial Expression Behavior Between Genders in Deception Detection, IEEE International Conference on Automated Face and Gesture Recognition (FG), Argentina, May 2020

[6] M. Tran, T K. Sen, K. G. Haut, M. A., Ali, M. E. Hoque, Are you really looking at me? A Feature-Extraction Framework for Estimating Interpersonal Eye Gaze from Conventional Video, IEEE Transactions on Affective Computing, March 2020.

[7] M. K. Hasan, S. Lee, W. Rahman, A. Zadeh, R. Mihalcea, L. P. Morency, E. Hoque, Humor Knowledge Enriched Transformer for Understanding Multimodal Humor, The Thirty-fifth AAAI Conference on Artificial Intelligence (AAAI-21), February 2021.

[8] M. K. Hasan, J. Spann, M. Hasan, M. S. Islam, K. Haut, R. Mihalcea, and E. Hoque, Hitting your MARQ: Multimodal ARgument Quality Assessment in Long Debate Video. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, Dominican Republic. Association for Computational Linguistics

[9] K. Sen, K. Haut, D. Lomakin, E. Hoque, A Mental Trespass? Unveiling Truth, Exposing Thoughts and Threatening Civil Liberties with Non-Invasive AI Lie Detection, IEEE Transactions on Technology and Society, Volume: 3, Issue: 2, June 2022