Testing Shortened Versions of Smell Tests to Screen for Hyposmia in Parkinson's Disease

ABSTRACT Background Hyposmia is an early feature in neurodegenerative diseases, most notably Parkinson's disease (PD). Using abbreviated smell tests could provide a cost‐effective means for large‐scale hyposmia screening. It is unclear whether short smell tests can effectively detect hyposmia in patient populations. Objectives To test the ability of short smell combinations to “prescreen” for probable hyposmia in people with PD and target administration of more extensive tests, such as the University of Pennsylvania Smell Identification Test. Methods We assessed the screening performance of a short 4‐smell combination previously derived from use of the 40‐item University of Pennsylvania Smell Identification Test in healthy older people and its ability to detect hyposmia in a large cohort of PD patients. Results The novel 4‐smell combination included menthol, clove, onion, and orange and had a sensitivity of 87.1% (95% confidence interval, 84.9%–89.2%) and specificity of 69.7% (63.3%–75.5%) for detecting hyposmia in patients with PD. A different (also novel) 4‐item combination developed using a data‐driven approach in PD patients only achieved 81.3% (78.2%–84.4%) sensitivity for equivalent specificity. Conclusions A short 4‐smell combination derived from a healthy population demonstrated high sensitivity to detect those with hyposmia and PD.

Impaired olfaction (hyposmia) is an early feature of neurodegenerative diseases, most notably Parkinson's disease (PD). [1][2][3][4][5][6] The University of Pennsylvania Smell Identification Test (UPSIT), comprising 40 "scratch-and-sniff" microencapsulated odorant strips, is commonly used worldwide. 7 We previously identified short smell combinations derived from the 40-item UPSIT that could be cost-effective for large-scale hyposmia screening before targeted administration of the UPSIT. 8 That work was conducted in the healthy general population recruited to the PREDICT-PD cohort study and was compared with only a small sample of patients with PD. Here we tested the screening performance of several short smell combinations in the Tracking Parkinson's study that includes data from 1222 people with PD. 9

Validating a Short Smell Test in People with PD
The performance of 5 smells (menthol, clove, onion, gingerbread, and orange) achieved a balance between brevity and high performance for identifying people with hyposmia against the full 40-item UPSIT (sensitivity 94.1%) in the PREDICT-PD pilot cohort's healthy participants (n = 891 for discovery and 191 for validation, as described previously 8 ).
We assessed screening performance of a combination of 4 of these 5 smells, omitting gingerbread, in Tracking Parkinson's 1222 patients. This was because participants in PREDICT-PD used the 40-item US version of UPSIT, whereas Tracking Parkinson's cases completed the 40-item UK version of UPSIT. These 2 UPSIT versions are broadly similar, but 8 of the 40 smells differ to tailor smells for recognizability in certain populations. We prioritized cross-cultural smells (those appearing in both the US and UK UPSIT versions), which did not include gingerbread. A positive screen for hyposmia using this 4-item combination was defined when 1 or more smells was identified incorrectly. Positive and negative screens for hyposmia were compared with hyposmia defined by performance on the full 40-item test using the same age-specific and gender-specific thresholds described previously. 8 Deriving a Novel Set of Optimal Smells in the PD Cohort We next ran a similar but abridged version of the full analysis we reported previously. 8 We considered all possible combinations of 4, 5, and 6 smells from the full 40 UPSIT smells, testing multiple different score thresholds for defining hyposmia (ie, at least 1, 2, 3 etc. incorrectly identified smells to denote a positive hyposmia screen). These results were compared with the participants' scores on the full 40-item test in a "discovery" cohort comprising a randomly selected 90% of the participants from Tracking Parkinson's (n = 1100). The best-performing combination of smells (defined by the highest sum of sensitivity and specificity) at each hyposmia threshold was tested in an independent "validation" cohort comprising the remaining 10% of participants (n = 122). We considered different proportions of data for discovery/validation (including 50% for each, 75%/25%, 80%/20%, and 95%/5%), but none achieved notably greater validation screening performance than 90% for discovery and 10% for validation. Screening performance of the best smell combinations is expressed as the values derived from the independent validation set.

Additional Analyses
We next compared performance of the smell combinations used in the commercially available 4-item "Pocket Smell Test" to our novel combinations using the same methods. There are 2 commercially available 4-item "Pocket Smell Tests" (PST-A and PST-B); these are intended for use as a prescreen for identifying individuals who require a full UPSIT. The PST-A includes chocolate, strawberry, smoke, and leather, whereas PST-B includes grape, which is not included in the UK UPSIT and so was not further considered.
Finally, we assessed the ability of short smell combinations to distinguish people with PD (in Tracking Parkinson's) and those without (in PREDICT-PD). For this analysis, positive and negative screens using the same criteria described previously were compared with whether the individual had a diagnosis of PD (rather than hyposmia or not).

Validating a Short Smell Test in People with PD
Based on total UPSIT scores and age-specific and sex-specific thresholds, 80.9% of participants (988/1222) were classified as hyposmic in Tracking Parkinson's. At the time of olfactory testing, the PD patients had a mean disease duration of 1.89 years (range 0.36-4.50 years). Table 1 shows the screening performance of the menthol, clove, onion, and orange combination, where 1 or more incorrectly identified smell was used as a positive screen for hyposmia. A screen was considered negative if participants identified all 4 smells correctly. The corresponding screening performance values (and 95% confidence intervals) of the 4-item combination were the following:

Deriving a Novel Set of Optimal Smells in the PD Cohort
The optimum combinations of 4, 5, or 6 smells using different cutoffs to define hyposmia (ie, ≤1, ≤2, ≤3 etc. correctly identified smells) as identified in a "discovery" cohort of participants is shown in Figure 1. None of these exceeded the sensitivity of the 4-item combination used in stage 1 (Fig. 1), but some had higher PPV.

Discussion
An abbreviated combination of 4 smells from the UPSIT (menthol, clove, onion, and orange) retained high sensitivity for identifying individuals with hyposmia in the context of PD. Previous work demonstrated that these 4 smells, as well as gingerbread, have high screening performance for detecting hyposmia in a general population and hinted toward similar screening performance in people with PD, 8 an observation that is borne out in the current analyses. This combination of 4 smells also outperformed smells in the commercially available PST-A in differentiating PD cases from controls (on account of high prevalence of hyposmia in PD). Although it performed less well when compared with the 40-item UPSIT, these data suggest that abbreviated tests used as a screen in the prediagnostic phase of PD will not systematically miss a large proportion of those with PD-related olfactory dysfunction.
Our proposed application for an abbreviated smell test would be as part of a 2-step approach, whereby individuals complete an abbreviated test as a quick and cost-effective prescreen to identify individuals who might be pertinent to consider for full-smell testing. Maintaining high sensitivity would be most important for an abbreviated smell test used in this way to ensure that most people who require full testing are identified while allowing more cost-effective, targeted administration of the full UPSIT. For example, sending a full UPSIT ($26.95 per test) to identify hyposmia in a cohort of 10,000 healthy older people would cost $269,500. Postage costs, based on UK pricing, would amount to $4 return for each participant ($40,000 total). Using this 4-item prescreen ($3.95 per test), 31.0% in the PREDICT-PD study screen positive and would also be sent a full UPSIT (total cost in tests $122,940). The 2-step approach generates different postage costs as well. The lighter weight of a 4-item test costs $2 return in the UK ($20,000 for 10,000 participants), with 31.0% incurring a second return postage cost of $4 for the full UPSIT ($12,400 for 10,000 participants). Hence, the total cost for nondiscriminative UPSIT administration versus using this 4-item prescreen amounts to $309,500 versus $155,340, respectively. Given the 4-item combination's 87.1% sensitivity for detecting hyposmia here, this halving of cost would come at the expense of missing approximately 10% to 15% of people in whom full UPSIT testing would have identified hyposmia.
The attempt to derive a novel "PD-specific" abbreviated smell combination (stage 2) was notable for the much greater variability in smells featuring in winning combinations than was present using the PREDICT-PD cohort's general population. This perhaps reflects greater variability and more erratic trends in olfactory dysfunction in PD cases than controls. Indeed, this is borne out in the fact that the 4 smells identified as performing best in PREDICT-PD's general population outperformed the corresponding "winning" subset of 4 smells with hyposmia cutoff ≤3 derived from people with PD in every regard (sensitivity 87.1% vs. 81.3%, specificity 69.7% vs. 61.5%, PPV 92.3% vs. 88.6%, and NPV 56.2% vs. 40.7%, respectively). This adds further weight to the finding that the highperforming smells identified in PREDICT-PD's general population are a reliable marker for screening hyposmia.
Some limitations need to be noted. The high hyposmia prevalence (using UPSIT) in this cohort of people with PD (80.9%) is expected given the strong association between hyposmia and PD. In the PREDICT-PD general population, hyposmia prevalence using the same criteria was 16.2%. This warrants attention as high prevalence drives higher PPV and lower NPV. Indeed, as is clear in Figure 1, PPV was consistently higher than NPV for every smell combination in the Tracking Parkinson's participants. Great care is necessary if attempting to extrapolate PPV/NPV to other populations given their high context dependency. There are likely to be other sociodemographic differences between PREDICT-PD and Tracking Parkinson's participants in addition to disease status. These potential confounders could have influenced test performance, but the observation that the original 4 smells outperformed the data-driven approach suggests this was limited. The analysis also extrapolated the performance of 4-item smell tests from 4-smell combinations taken within a larger 40-item smell test. There is a chance that true performance with dedicated 4-item tests may differ from the 4-smell combinations identified in this work.
Our 4-smell combination retains a degree of cross-cultural relevance given that all 4 smells feature in both the UK and US versions of the UPSIT. However, testing here was only in individuals based in the United Kingdom, including a fairly restricted set of ethnicities. Further validation would be required including other ethnic minorities and in other countries to assess broader external validity. There also remain some unavoidable differences between the UK and US versions, most notably the use of different distractor options for some smells. In the analysis of screening performance for identifying PD cases versus controls, the cases (in Tracking Parkinson's) all completed the UK version of the UPSIT, whereas the controls (in PREDICT-PD) completed the US version. This could have impaired performance of controls versus cases given that it was a UK population completing a US test version. However, this would likely have led to the underestimation rather than the overestimation of screening performance owing to a reduced difference between cases and controls.
Prospective follow-up of these and similar cohorts will be able to provide additional information regarding possible clinical implications of having hyposmia in people already diagnosed with PD or whether disease duration is related to the incidence of hyposmia.
In conclusion, the abbreviated 4-smell combination of menthol, clove, onion, and orange retains high sensitivity to detect those with hyposmia in the context of PD. This subset of smells has cross-cultural relevance and outperformed attempts to derive a separate PD-specific combination of smells for hyposmia screening.