How did you build your emotional dictionary ?

How did you build your emotional dictionary ?

To perform emotion recognition in customer comments, Q°emotion has developed its own methodology based on its own assets and technology.

We use unique emotional dictionary and advanced NLP algorithms that enables distinguish the emotional intensity markers with accuracy.

Let's have a look on how it was built:

The emotional dictionary from Q°emotion is a proprietary emotional corpus that have been build in 3 steps:

1. Existing affective Norms
We based the primitive dictionary on the statistical results and emotional dimensions based on the ANEW methodology (Affective Norms for English Words, Bradley and Lang (1999)).

The evaluations were done in the dimensions of valence, arousal and dominance using the Self-Assessment Manikin (SAM), and normalized. The precised methodology has been precisely defined for English (ANEW = Affective norms on English Words) and is followed for other languages (FAN=French Affective Norms, ANGST=Affective norms for German Sentiment terms). The dataset has also been replicated in more than 30 languages such as Portuguese (Soares et al., 2012), Italian (Montefinese et al., 2014), or Spanish (Redondo et al., 2007), Dutch, Japanese, etc.

For every word, the respondents have to answer and precise for them the level of the 3 dimensions (Model VAD) on a scale from 0 to :
  1. Valence
  2. Arousal
  3. Dominance
They use precisely the SAM (Self Assessment Manikin) that have been normalized:

figure1
Want to know more : pls see: https://en.wikipedia.org/wiki/Arousal

The emotional intensity indicator is calculated based the maximum arousal scoring observed in the speech, using scientific affective norms developed for each idioma.

Affective norms have the following advantages:
- International approach that enables to take into account the cultural differences with a same measurement framework.
- That can provide a big help to put relative weights between words into the same sentence.  
- Normalization rules have been discussed and challenged in cognitive psychology researchers. That enables to build a multi-idioma dictionary based on a stable and reviewable data base.
- Primary emotions may be deduced from these first results

But the affective norms have the following disadvantages: 
- Limited number of words after tokenization (to max 30 K words by idioma)
- Limited to single words : does not take into account the sentence structure
- Does not take into account the impact of the negation on the words, nor the frequency, nor the idiomatic phrases, etc.
- A same word can have several meaning depending of the context (ex: Vol in French can be Flight or Theft depending of the context)


One example:
The word Spider (EN)/ Aranha (PT) / 蜘蛛 (JN)
has the following scores: 
  1. a low valence of 2.56/9 -  2.64/9 - 2.33/9
  2. a high level of arousal of 7.8/9 -  7.2/9 - 7.9/9 
  3. a low dominance of 2.30/9 -  2.32/9 - 2.29/9
 
But we don't have the values in any of those languages for spiderman or spider web etc.
We don't have neither the values for the following sentences:
- "There is no spider in the room." 
- "My spider sense is tingling."



2. After having built this primitive dictionary, one major step has been to extend the dictionary into millions of entries with grammatical entries using available open resources and by porosity, allocate emotional values based both on human marked data set. 
This extension is based on the fact that in sentences more than 30 words, it is very rare to have unique emotional markers in the sentence. At the contrary, when expressing emotions (and particularly in reviews), the emotional markers are repeated several times, enabling to adjust the different emotional values in an automatic way by porosity, depending mainly on co-occurences.

Based on millions of occurrences, this enables to take into account the context of the sentence, and to build general rules that are valid for any languages.




This step enables to put values also on spiderman (more surprise and happiness than fear) and spiderweb (more disgust than fear) and to add rules according to co-occurrences.
This step may also differentiate words according to the grammatical values: ex to take into account the intensity of the speech according to the following elements.

The remaining rules are managed through the NLP analysis:




3. For each new data analysed, a robot based on Artificial intelligence is working in parallel and help us detect errors in data base and suggest corrective actions in the dictionary. This step can also be based on quantitative indicators (when available) to detect errors and find incongruities in each language. This can propose to automatically diminish an emotional weight, add a new emotional dimension depending of the context, etc.



All rights reserved: Qemotion France SAS (2021)

Qemotion proposes CXinsights.io, a Customer experience solution: A SaaS solution that is capturing emotions from survey comments and webreviews and that is unlocking valuable insights.

The customer experience management platform showcases the emotions along your real customer journey, the interactions with the staff described by your clients etc.

You can therefore determine in an easier way the key action levers like:

- key insights and variations,

- main irritating points, 

- main enchantment points.

The platform helps you manage and prioritize the improvement actions you want to launch in order to improve the customer experience. As a consequence, you will increase loyalty and decrease churn levels.

The platform helps you also diffuse information into your organization with automated emotional alert systems, that transform your CRM into a Customer Enriched Emotion Management system (CEEM).

More information on: https://www.qemotion.com