Skip to content
CiderHQ
Search

Cider science

Choosing a sensory test

Which tasting test should I use to compare two ciders?

In short

Start from the question, not the test. "Can anyone tell these apart?" is a triangle or duo-trio test. "Is the new batch close enough to the old one?" is a similarity test, which is planned differently and needs more people. "Which is drier?" is a paired comparison on that one attribute. "What does each taste of?" is a descriptive profile by a trained panel. "Which do people like?" is a consumer test, and trained tasters are left out of it.

The question also decides the panel. The international standards for the triangle test give 24 to 30 assessors as typical for showing a difference and about 60 for showing similarity, and a consumer preference test should not have fewer than 60 people.

Decide the question, the test and the significance level before anyone tastes anything. Changing any of them after seeing the result makes the statistics meaningless.

From the question to the test

The difference tests ask assessors to find a sample, not to describe it, which is why they work with people who have no vocabulary for cider. The duo-trio test is easier to do than the triangle test but statistically less efficient, so it needs more people for the same sensitivity. The triangle test does badly with drinks whose flavour lingers, because each sample is tasted on top of the last. Tannic ciders, whose astringency builds from glass to glass, are the likely cider case.

The paired comparison is the only one of these that names the attribute. That makes it more sensitive to the named thing and blind to everything else. Whether it is one-sided ("is B drier?") or two-sided ("do they differ in dryness?") has to be decided before it is run.

The question a cider maker usually has, and the test designed to answer it.
The questionThe testWho tastesWhat it cannot tell you
Can these two be told apart at all?Triangle or duo-trio testScreened assessors who need no cider vocabularyHow big the difference is, or which one anyone prefers
Is the change small enough not to matter?Triangle or duo-trio run for similarity, with its own tableA larger panel, sized before the testThat the two are identical; only that a difference of the size chosen is unlikely
Which of the two has more of one named thing?Paired comparison (2-AFC), directionalAssessors who recognise the named attributeWhether the samples differ in anything else
What does each cider taste of, and how strongly?Descriptive profileA trained, calibrated panelWhether the cider is good, or whether anyone will buy it
Which do drinkers like better?Consumer hedonic testUntrained consumers from the people the cider is for; trained tasters excludedWhy they like it, or what to change

How many people

Small panels have a hard limit, not just a weaker result. With six assessors in a duo-trio test, even six correct out of six does not reach the 1% level. The discrimination test calculator says so when a panel cannot reach the chosen level.

Showing similarity takes more people because it has to show that a difference is unlikely, and absence of evidence is not that. The similarity tables in the standards are built on a second risk, the chance of missing a real difference, and on the proportion of people able to tell that the producer is prepared to accept. A difference-test result that is not significant is not a similarity result.

Panel sizes as the international standards give them. These are typical sizes, not minimums that guarantee a useful result.
TestTypical, to show a differenceTypical, to show similarityUsually not recommended below
Triangle test (ISO 4120)24 to 30About 6018
Duo-trio test (ISO 10399)32 to 36About 7224
Consumer hedonic test (ISO 11136)At least 60; 100 as the default minimum—60

Running it so the numbers mean something

Fizz is a variableCarbon dioxide changes how sweet, sour and aromatic a cider seems, and it escapes as soon as a bottle is opened. In the Virginia Tech cider lexicon study, samples were decanted each day into small screw-top bottles to limit the loss and served at 5 °C. Pour every sample in a test from the same kind of container, at the same temperature, at the same time.

Trained panels and consumer panels are not interchangeable

A trained panel measures. ISO 13299 says that an overall evaluation by a trained panel must not be a liking score, because trained assessors evaluate differently from the people a product is for. ISO 11136 excludes trained assessors from consumer tests for the same reason, and notes that in-house tasters tend to favour the products they recognise. A cidery's own staff are an in-house panel in exactly this sense.

A trained panel is also an instrument that has to be checked. ISO 11132 measures panel performance in three ways: whether the panel can tell products apart, whether the assessors agree with each other, and whether each assessor gives the same answer to the same sample twice. A panel that is never checked produces numbers of unknown quality.

A vocabulary built for cider

Descriptive work needs shared words, and cider now has a published set. A Virginia Tech study trained eight panellists for 13 hours, over eleven sessions, on reference standards, then profiled 42 commercial ciders from Virginia and New York. Of the 33 lexicon terms, 29 turned out to be the ones that mattered, and the accompanying flavour wheel was cut from more than 150 terms to 73.

The same work tested a label scale producers already used. It found the Merlyn Dryness Scale was not an effective way to describe how dry or sweet a cider tastes. How dry a cider tastes depends on more than the sugar a scale is built on.

Wine and food standards, applied to ciderThe ISO methods are general sensory-analysis standards written for any food or drink, not for cider. The panel sizes and procedures above are theirs. The lexicon, the flavour wheel and the dryness-scale finding are cider-specific, from the Virginia Tech work.

Also answered on this page

Questions this page covers, so you can tell at a glance whether it is the one you want.

Related

What people ask next

Questions readers ask about the things this page mentions. Each one goes to the section that answers it rather than to a page written to receive the question.

Sources

What this page rests on. Where a source is marked as registered rather than read, CiderHQ is recording that the body is authoritative on the subject without claiming to have worked through the document itself. See our evidence policy for what each state means.