Sentiment and Emotion Analysis

Is sentiment analysis accurate?

It depends entirely on the corpus, and the only honest answer comes from measuring it on yours. A classifier's published accuracy was established on someone else's text, in another domain, at another length. Build a human-labelled sample of 200 to 400 items, measure agreement per class and per segment, and report the human baseline alongside: if two trained labellers agree on 78 percent of items, a machine reporting 91 percent is not outperforming them, something is wrong with the comparison.

Why does sentiment analysis get sarcasm wrong?

Because surface polarity inverts and the cue is contextual rather than lexical. It is not the only failure of this kind: negation scope, comparison ("better than the old one" says nothing absolute), conditionals ("if they fixed it I would be delighted" reads positive and is a complaint), and reported speech all defeat classification the same way. Screen a sample for each condition, record the incidence, and disclose the direction of bias with your results.

What do I do when one response contains both praise and complaint?

Preserve both. "The staff were lovely but I waited fifty minutes" is not a neutral experience, and averaging it into neutral deletes two findings at once. Score at aspect or clause level so opposing valences attach to different things, count mixed items as their own category, and report the mixed proportion, because a rising one usually means your frame is losing resolution.

Why can I not just report one sentiment number for the business?

Because it aggregates incommensurable things. A billing complaint and a compliment about a shop assistant do not average into a meaningful quantity, and the figure moves whenever the channel mix moves, independent of how anyone feels. Report by content code, by aspect and by segment, each with its own base and confidence.

My sentiment score moved three points. Is that a finding?

Usually not. Compute a noise band from the classifier's measured error plus sampling variation on your period base, publish it before the next wave lands, and report any movement inside it as no change, in those words, with no explanation attached. Narrating small movements teaches an organisation that the metric is meaningful at a resolution it does not have. If the band is wider than the movements the business cares about, the metric cannot serve as a tracker.

Can I use sentiment analysis instead of coding open ends?

No. Sentiment is downstream of coding, not a replacement for it. Valence with no subject attached cannot be acted on, because nothing follows from it: "62 percent negative" tells nobody what to change. "Negative sentiment on delivery, driven by the delivery-window code" does. Any sentiment output with no content codes behind it should not be produced.

How reliable is emotion classification?

Materially less reliable than valence. The categories are less distinct, human agreement is lower, expression varies by culture and language, and the label set is a theoretical choice rather than a natural fact. Use a small set defined for your corpus, build a separate gold set, measure agreement separately, and suppress any category that falls below your valence threshold. Absence of an emotion word is not evidence the emotion is absent, and its presence is a statement, not a diagnosis.

Should I run sentiment on translated text?

Only for triage, and marked as translation-dependent throughout. Translation systematically normalises tone: sarcasm, understatement, politeness formulas and intensifier norms are the first things lost. Sentiment on translated text measures the translator as much as the participant, and cross-market comparison of translated sentiment produces differences of ten points or more that have nothing to do with satisfaction.

What base do I need to report a sentiment percentage?

At least 100 items in the specific reporting cell, with 30 as an absolute floor below which no percentage is permitted at all. The threshold applies to every subgroup and every period separately, not to the corpus total. Cells that fail are reported as counts and verbatims, and shown as suppressed with the reason, because an empty dashboard cell reads as zero.

Does neutral sentiment mean customers are satisfied?

No, and this is the most common misreading. Neutral means no clear valence was expressed, which includes factual description, procedural answers and items that say nothing evaluative at all. The complement of negative is not satisfaction, it is everything else. Keep "no sentiment expressed" as a separate category rather than folding it into neutral, or the middle of your distribution becomes uninterpretable.