SESH FRAGRANCE ACADEMY | ADVANCED SENSORY EVALUATION
Does a mean rating of 4.2 versus 3.8 prove a winner?
These curves show how changing mean and variance affects a statistical model. They are theoretical teaching data, not skin or fragrance-panel results.
Open full-size image ↗Interpret means, dispersion, sample size, uncertainty and the distinction between statistical detection and meaningful improvement.
1. Means hide the spread
Five panelists rate F01 sweetness [4,4,4,4,4] on a defined seven-point scale, giving mean 4. F02 yields [1,2,4,6,7], also mean 4, but with much greater dispersion. Define the scale consistently and preserve individual observations. The same mean can arise from consensus or polarized perception. Display point-level data or a dispersion measure alongside the mean.
2. Standard deviation and standard error are not interchangeable
Standard deviation describes variability among observations, while standard error concerns uncertainty in estimating a mean. For independent observations an elementary approximation is SE = SD/√n. Repeated ratings from the same person are correlated and should not be treated automatically as independent n. State assessors, sessions and how repeated measures were handled. Neither SD nor SE alone establishes real consumer preference.
Effect size, uncertainty and statistical significance — a published visual reference for the topic, not SESH product-test data.
Open full-size image ↗3. Confidence intervals communicate uncertainty
A frequentist 95% confidence interval comes from a procedure designed to cover the fixed underlying parameter in 95% of repeated studies, under the model assumptions. It is not literally a posterior 95% probability for this one fixed parameter. Small samples or skewed ordinal ratings can produce broad uncertainty. Report sensible precision, the interval, and the method rather than overinterpreting tiny decimal differences.
4. Pairing changes the correct comparison
If the same assessors evaluate F01 and F02 under controlled conditions, outcomes are paired: each person contributes a within-person difference. Treating these as unrelated independent samples discards useful structure and can misstate uncertainty. If distinct groups rate each fragrance, the design differs. The statistical model must reflect scale properties, sample size and assumptions; bounded ordinal ratings are not automatically precise physical measurements.
5. Statistical significance is not effect size
A small p-value indicates how unusual the observed or more-extreme data would be under a stated null and statistical model. It does not measure product quality, the probability that the hypothesis is true, or the magnitude of the difference. A 0.1-point shift may be statistically detectable but unimportant, while a larger observed shift may remain uncertain with few assessors. Define a minimum practically important effect before the study.
6. Avoid cherry-picking dozens of attributes
If you compare 20 descriptors at three time points without a predeclared plan, some apparently significant findings are likely by chance. Predefine a primary endpoint; treat remaining measures as exploratory or use a suitable multiplicity adjustment. Do not hide an unwanted increase in sweetness while advertising a single positive comparison. Transparent presentation is stronger than cherry-picking one result among sixty.
7. Decision framework: uncertainty plus relevance
Before data collection specify who participates, the primary descriptor, pairing, the smallest meaningful improvement, scale and checkpoints. Report version means, estimated difference, relevant dispersion, uncertainty interval where appropriate and contradictions. Reasonable decisions include likely meaningful improvement, insufficient evidence requiring more data, or statistically detectable but practically negligible change. Avoid declaring a winner from one uncontextualized statistic.
Case study: mean 4 doesn't mean the same story
F01 ratings [4,4,4,4,4] and F02 [1,2,4,6,7] share mean 4 on a seven-point scale. One is consistent, the other variable. Without individual records, session details and suitable analysis we cannot distinguish calibration problems from genuine observer differences.
Professional practice exercise
Create a fictional paired dataset for six assessors rating F01 and F02. Calculate each version's mean and every within-assessor difference. Compare the result with a predeclared minimum important change. Explain why three trials from one person are not three independent assessors.
Three educational concept maps
From score to inference
- Start with individual ratings
- Mean: central tendency
- SD: observed dispersion
- SE/CI: estimation uncertainty
- Practical importance and limitations
Same mean, different spread
- F01: 4,4,4,4,4
- F02: 1,2,4,6,7
- Both means = 4
- Agreement differs dramatically
Common inference traps
- Independent n ≠ number of repeats
- p-value ≠ effect size
- Multiple comparisons
- Small effects may be unimportant
Scientific glossary
- Standard deviation
- Summary of score dispersion among observations.
- Confidence interval
- Interval produced by a procedure with stated long-run coverage properties.
- Paired design
- Design linking two product ratings from the same assessor.
Official scientific references
This educational explanation does not constitute ISO certification or full implementation of a standard. Consult the official standard for formal testing.
