Telling a great story often involves a deliberate alteration of emotions. In this paper, we objectively measure and analyze the narrative trajectories of stories in public speaking and their impact on subjective ratings. We conduct the analysis using the transcripts of over 2000 TED talks and estimate potential audience response using over 5 million spontaneous annotations from the viewers. We use IBM Watson Tone Analyzer to extract sentence-wise emotion, language, and social scores. Our study indicates that it is possible to predict (with AUC as high as 0.88) the subjective ratings of the audience by analyzing the narrative trajectories. Additionally, we find that some trajectories (for example, a flat trajectory of joy) correlate well with some specific ratings (e.g. “Longwinded”) assigned by the viewers. Such an association could be useful in forecasting audience responses using objective analysis.

Publication

M. Iftekhar Tanveer, Samiha Samrose, Raiyan Abdul Baten, and M. Ehsan Hoque. Awe the Audience: Awe the Audience: How the Narrative Trajectories Affect Audience Perception in Public Speaking, In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI’18), Montreal, Canada, 2018

Complete Code and Data

Github Repository

Supported Operating Systems

Tested on Linux (CentOS) and OSX. Should run on Windows as well.

Installation Instructions

  1. For python distribution, you can install anaconda or miniconda (a mini version of anaconda)
  2. Download and extract the zip file and enter to the extracted folder:cd tedanalytics
  3. Install the requirements:pip install -r requirements.txt
  4. Install IBM Watson Developer Cloud using the following command:pip install –upgrade watson-developer-cloud
  5. To reproduce all the results shown in the paper, run:python ted_talk_experiments.py
  6. (Optional) If you want to extract the IBM scores yourself, you need to create an account for IBM Watson tone analyzer service (Even a free account will work). Then, edit the bluemix_key.py file and put the credentials (username and password) provided in the service key.Then run: python bluemix.py
  7. (Optional) If you want to crawl the data from TED.com, run:python ted_talk_new_crawler.py

Notes

  1. We provided the TED talk transcripts and the IBM scores within the repository (please check the “talks” folder and the “bluemix_sentiment” folder).
  2. The credentials to be written in bluemix_key.py file are different than the account login username and password.
  3. TED.com occasionally changes the format of the website and the meta content. In that case, the existing crawler will fail and that issue needs to be taken care of separately.
  4. The data is stored in pickle format. To read pickle files, use the pickle library in python: https://docs.python.org/2/library/pickle.html.
  5. The code is inline documented, so you can get the necessary information to EXACTLY replicate the experiments (even for other data sets).