For those unfamiliar with scoping reviews, they’re a type of literature review that maps out the existing evidence on a topic and identifies where gaps exist. Unlike systematic reviews that follow strict PRISMA guidance and aim to answer very specific questions, scoping reviews cast a wider net to explore the breadth of research in an area. 

This particular review tackled something that many of us have probably taken for granted in our practice: the idea that a 2-point change on the Canadian Occupational Performance Measure represents a clinically significant difference. You’ve likely seen this mentioned in research articles, heard it in training, or even cited it yourself when writing reports. But where does this claim actually come from? McColl and colleagues set out to trace the origins of this assertion and determine whether there’s solid empirical evidence to support it. Their specific research question was “What magnitude of change score represents a clinically significant difference on the COPM?”

The context here is important. The COPM has been around since 1991 and is considered the Canadian standard for occupational therapy assessment. It’s widely recognised as responsive to change—meaning it can detect statistically significant differences over time. But statistical significance and clinical significance are two different things entirely. A change can be statistically significant with a large enough sample size, but that doesn’t necessarily mean it’s meaningful or noticeable to the client in their daily life.

For me, what makes this review particularly noteworthy is who conducted it. The first author, Mary Ann McColl, and the final author, Mary Law, were themselves involved in developing the COPM. Mary Law was one of the original creators. So, this isn’t a critique from researchers with competing tools or different theoretical perspectives, which is often what occurs in research literature. The tool’s own developers are saying ‘we need to be honest about what the evidence actually supports.’ 

Methods

• The scoping review followed Arksey and O’Malley’s five-step approach. 
• The search covered the articles published between 1991 to 2020 that included “COPM” or “Canadian Occupational Performance Measure” in the title, abstract, or keywords. 
• Studies had to have a minimum sample size of 25 per group and that it must report the performance and satisfaction scores.
• The final dataset was 100 papers. 
• For each of the papers, they extracted COPM change scores, categorised interventions, and reviewed the sources that papers cited when claiming clinical significance.
• One third of these were with children, indicating that the tool is being used with children themselves. The studies came primarily from Sweden, Canada, and the USA and the interventions took place across various settings including institutions, community settings, rehabilitation centres, and outpatient facilities.

The Results

The 2 point change evidence trail

Of the 100 papers reviewed, 53 mentioned a clinically significant difference, 44 of them provided a citation to support their claim. And when the researchers traced those citations back to their sources, they found something surprising: there was no empirical evidence for the 2-point threshold. One of the papers even cited an article that didn’t even use the COPM as their source for evidencing the 2 point change!

The most commonly cited source was, unsurprisingly, the COPM manual itself—referenced by 27 articles. But here’s the problem: the first edition from 1991 doesn’t mention clinical significance at all. The second and third editions note that an average change of 1 point on performance and 1.1 points on satisfaction indicated the ability for the COPM to detect change, no evidence was provided for 2 points being the specific threshold for clinical significance. The fourth and fifth editions mention the 2-point difference but offer no new empirical evidence to support it. The other articles referenced by the papers, also did not specifically mention the 2 point change. The scoping review authors essentially conclude that whilst the 2-point threshold has been widely repeated and accepted, there’s no research data underpinning it.

Intervention Types and Change Scores

Another interesting part within the review was that different types of interventions produced very different COPM change scores. These ranged from a mean change of 1.69 to 4.90 for performance and 2.01 and 4.60 for satisfaction depending on the intervention type. Task adaptation and support provision had the highest changes in mean performance scores, with skill development having the lowest. The average change across all 100 studies was 2.70 for performance and 3.11 for satisfaction.

The Two Studies with Actual Evidence

Two papers within the sample tried to determine what constitutes a clinically significant change empirically. Eyssen and colleagues found that clients perceived improvements in occupational performance at change scores of just 0.9 for performance and 1.9 for satisfaction, so lower than the 2-point threshold. Tuntland and colleagues in 2016 found the opposite: clients indicated that a “minimally important change” was associated with 3 points or more on performance and 3.2 points for satisfaction, so higher than the accepted threshold.

The practical takeaways are presented at the end of the page.

This is a systematic review. I covered two systematic reviews last week. For those who didn’t listen, systematic reviews are a rigorous, structured review of the literature that adheres to established PRISMA guidelines. Unlike scoping reviews that map the landscape of research, systematic reviews aim to answer very specific questions by synthesising all available evidence.

This review tackled the same issue that McColl and colleagues highlighted in their 2023 scoping review: the lack of empirical evidence for the widely accepted 2-point clinical significance threshold on the COPM. However, Yuine’s team took a different approach. Rather than examining intervention studies that used the COPM as an outcome measure, the purpose of this study was to investigate and report minimal important change (MIC) and minimal detectable change (MDC) within the COPM in various populations, while assessing the quality of the relevant studies.

• MIC (minimal important change) represents the smallest change that clients perceive as a meaningful improvement, it’s about what matters to the client.
• MDC (minimal detectable change) represents the smallest change that exceeds statistical measurement error. So it’s about what the statisticians can confidently say has changed beyond the margin of error.
• Ideally we would like our interventions to be meaningful to the client and showing significance on the statistical analysis.

Methods

The team followed the PRISMA guidelines for systematic reviews and assessed risk of bias using the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) Risk of Bias checklist. The search was run on 23 May 2023, it was unclear how far back the search went. They were very specific about ensuring the included papers reported methodologies for calculating MIC and MDC. After study screening, and full text review only five papers met their strict inclusion criteria. Four studies with calculated MIC values and one calculated MDC values. The first four were with adults and it was unclear if the fifth included children but the mean age was 47.3 years.  

Participants

From an initial pool of 229 studies, only five met their strict inclusion criteria. These five studies involved diverse populations: inpatients and outpatients with various conditions, community-dwelling older adults, people with ankylosing spondylitis, and subacute rehabilitation patients. Ages ranged broadly, and reassessment periods varied from 2 weeks to 3 months post-intervention. The COSMIN checklist revealed that whilst studies generally did well at comparing COPM with other outcome measures, they had weaknesses. Most didn’t adequately describe interventions in detail, didn’t analyse subgroups by important characteristics like disease type, and didn’t provide sufficient detail about the hypotheses they were testing regarding before-and-after comparisons.

The Results

MIC Values: Wide Variation Across Studies

The four MIC studies produced a striking range of values. For performance scores, MIC ranged from 0.20 to 3.20 points. For satisfaction scores, it ranged from 1.45 to 3.20 points. This variability was related to different client populations, reassessment periods, and calculation methods.

MDC Values: Measurement Error Thresholds

A single paper examined MDC. For standard personal interviews, they found MDC values of 1.47 points for performance and 1.80 for satisfaction. Interestingly, when COPM was administered by telephone or mail, performance and satisfaction scores increased. I haven’t gone back to review this study, but it’s worth checking out if you are using the COPM day to day or in research to unpick why the researchers thought this might be.

Conclusion

The authors in this systematic review concluded that  change of one point or less likely falls within measurement error—it’s not necessarily a real change, just the imprecision inherent in any measurement tool. A change of two to four points, depending on your specific client population, represents meaningful change that clients themselves would recognise as important.

• The first takeaway is that the 2 point change on the COPM being clinically significant is opinion based, rather than validated by research. As such, you cannot reasonably assume that a 2-point change automatically means you’ve achieved something clinically meaningful. These findings indicate that score changes are more nuanced than the simple 2-point rule and that you cannot simply apply a universal threshold.
• The reassessment period matters. This makes sense as if a client makes slow progress their reassessment would likely be different to someone making quick progress in the same time period. 
• There is a variance depending on the intervention type and population, you can use the articles to inform decision making related to specific interventions or clinical populations you are working with.
• As clinicians, rather than just seeing a 2 point change as the indicator that the goal is achieved, we could validate clinical significance through conversation with our clients. We can ask them directly questions like, “Does this feel like a meaningful change?” or “Is your child doing things now that matter to him that he couldn’t do before?” to help you to know if your interventions have made an impact for the family. 
• When writing reports or discussing outcomes we may need to be cautious about claiming clinical significance based solely on scores. It’s more accurate to say something like, “Sam’s COPM Performance score improved by 2.5 points, and his mum reports that he can now independently pack his school bag each morning, which was one of his priority goals.”
• For researchers, the take-home message is clear: we need to distinguish between statistical significance and clinical significance, and we can’t rely on a one-size-fits-all threshold. Different populations, different occupational performance problems, and different intervention approaches may require different levels of change to be considered clinically meaningful.