A recipe containing 109 ingredients is not only impractical, but likely unpalatable. Parallel to a recipe, less is often more when it comes to determining the final questions to include in a measure striving for a coveted ‘5-star’ rating. The challenge in refining the Sinclair Compassion Questionnaire (SCQ) was determining what number and combination of the 109 draft questions were worth keeping and which items needed to be discarded. But how does one determine which ingredients or questions are essential and which complicate, counteract, or make an overly cumbersome finished product?
As an initial step, we went to the most discerning of critics to taste-test our initial draft measure—Subject Matter Experts (SMEs) on the topic of compassion and patient advisors. While a less formidable forum than the tyrants of Hell’s Kitchen, our SMEs provided us with their critical first impressions before serving the SCQ to other beta testers. Our SMEs or ‘judges’ were tasked with deciding between alternate individual ingredients in the form of similarly worded questions, the readability of each question, while also providing expert feedback on the perfect combination of questions that were needed to create the perfect measure. For example, by choosing between items such as “my healthcare provider (HCP) showed genuine concern” or “my HCP showed genuine interest”, our expert consultants allowed us to refine the items comprising the draft measure to those that packed the most punch in terms of relevance, representativeness (i.e. mapping back to the domains of our Patient Compassion Model), flow, and clarity – a critical phase that allowed us to narrow down our item list by about half [1].
After the SMEs and patient advisors provided their 5-star rating, we then decided to beta test our culinary delight with a small group of diners to garner their feedback and recommendations. Accordingly,we piloted the draft 68-item SCQ with a diverse set of patients throughout various care settings including acute care, home care, long-term care, and hospice. As a result, the draft measure was further refined to 54 items, allowing us to analyze the performance of each of these ingredients at a microscopic level through exploratory and confirmatory factor analysis—modifying the questionnaire in a step wise manner by investigating the performance of individual questions and the effects of removing specific questions on the final product. As a result, we discovered that the optimal number of questions to be included in the world’s best compassion measure was 15 [2]. While reviews from individual patients on the measure are integral to getting things right, the measure scores also needed to be consistent over time at each return administration. If the same dish tastes different the second or third time around, trust in the recipe and even in the quality of the overall restaurant will diminish. In relation to the SCQ, each individual question of the SCQ was rigorously tested and approved for “internal consistency” and “test-retest” reliability—to ensure that it was accurate over time with the same patients—with the SCQ demonstrating both excellent test-retest and internal consistency.
Finally, we decided to do some ‘blind taste-testing’ not only between similar recipes, but also between recipes that were notably distinct. Our purpose in assessing the SCQ alongside other existing questionnaires measuring similar and dissimilar constructs was to demonstrate their convergent and divergent validity. Convergent validity compares apples-to-apples, so we utilized a similar scale (the Schwartz Centre Compassion Care Scale) to confirm that the SCQ was indeed measuring what we intended—compassion [2]. While one criterion for developing the world’s best apple pie is ensuring it tastes similar to apple pies created by others, it should also taste considerably different then chicken pot-pie, meat pie, or strawberry-rhubarb pie. This process with respect to measure development, is called divergent validity, which involved determining that the SCQ was sufficiently different than other measures that assess things such as patient satisfaction or symptom distress. In doing so, we once again confirmed that the SCQ specifically measures compassion, and compassion alone [2].
In conclusion, measure development is no trivial task—it takes time, expertise, scrutiny, end-user feedback, and rigorous scientific testing at each stage of the endeavor. This iterative process of developing the SCQ involved numerous counts of revisiting the scope and purpose of measurement, evidence informed decision-making, multiple checks and balances across each study stage, and rigorous psychometric testing. The result? The worlds most valid and reliable patient reported compassion measure for consumption by researchers and clinicians alike—compliments to the chef.
Check out the SCQ Video Tutorials to learn about the development, validation, and using the different versions of the SCQ in research, clinical practice and system analysis. https://www.compassionmeasure.com/gettingstarted
Works Cited:
[1] Sinclair, S., Jaggi, P., Hack, T. F., Russell,L., McClement, S. E., Cuthbertson, L., Selman, L. E., & Leget, C. (2020). Initial validation of a patient-reported measure of compassion: determining the content validity and clinical sensibility among patients living with a life-limiting and incurable illness. The Patient, 13(3),327–337. https://doi.org/10.1007/s40271-020-00409-8
[2] Sinclair S, Hack TF, MacInnis CC The COMPASS Research Team, et al. Development and validation of a patient-reported measure of compassion in healthcare: the Sinclair Compassion Questionnaire (SCQ) BMJ Open 2021;11:e045988. https://doi.org/10.1136/bmjopen-2020-045988
