Pip Griffiths, PhD
IQVIA
Written on behalf of the Psychometrics SIG Co-chairs and members
This year is my third and final year as chair of the Psychometrics Special Interest Group (SIG). Throughout my tenure I have tried to work with the other co-chairs to create inclusive projects for the SIG members to get involved with. This year was the most ambitious yet. We created 4 groups, each focussing on 4 different problems, in an effort to bring the best minds at ISOQOL together to come up with some solutions.
Of these groups, one was created to think about the analysis of intensive longitudinal data (ILD). ILD is the name given to data that is collected in voluminous amounts, and in relatively quick succession. For example, you could think about daily diary data, or data collected by a wearable device, such as a smart watch. The problem with this kind of data is that traditional psychometric methods for assessing the reliability and validity of a measure are not designed to handle. Traditional psychometric methods are designed for a single timepoint of data, or a best, a few timepoints, evenly spaced out over a broad stretch of time.
When faced with ILD data, researchers currently take some average of the data over a period of time to make a summary score. For example, this could mean taking a mean of daily diary scores over a week, or mean distance walked or step count over 14 days. The problem with this approach is that summary scores like these are estimates. This just means that there is a lot going on underneath the number that the number itself does not represent. If a patient has an average step count of 3000 steps in a 14-day period, we do not see the day-to-day variability around this number. Did this patient consistently walk 3000 steps each day? Did they have a few days of intense movement followed by days of bed rest? Was there a gradual increase in movement over the 14-day period? Were there some periods of the day where the patient was more active? The answer to all these questions is present in the collected data but is not present in the mean. Furthermore, the mean, with all this information hidden away underneath, is what is typically used as the value for psychometric assessment (and later assessment of treatment effect).
We came together to create a symposium to explore and explain how we could start assessing the psychometric properties of a measure that collects ILD, using the data in the way it is collected (rather than using a summary score). A team of SIG members divided between them different potential analysis methods that exist outside of our direct field and got to work adapting them for this purpose. We created a symposium focussing on three methods:
- time varying effects models (TVEM)
- a latent growth model using machine learning (ML) and regularised regression
- a continuous-time item response theory (IRT) model.
We have created this symposium to be a tutorial or introduction. We want to give the ISOQOL membership an initial look at what these methods can (and cannot) do, before we work together on more firm guidelines in the form of a white paper, or a tutorial manuscript.
Our hope is that researchers in the field, when presented with daily diary data or data from wearable devices will eventually move away from “taking the average” over a given time period. We want to give them the tools to start thinking about how to use the data in the way it was initially collected. This symposium is the first step in that direction, but there is more research needed to have full confidence in these methods. For example, future work needs to compare ILD methods to traditional methods through simulation work, stress-test the methods under adverse situations (like skew), assess the impact of missing data, test the interpretability of the results, and more.
If you have an interest in ILD and want to work together to explore one of these topics, please get in touch! You can reach out to the Psychometrics SIG on Teamwork. If you’re not already a member of the Psychometrics SIG, you can sign up on your ISOQOL member profile, and ISOQOL staff will add you to the Teamwork group.
Look forward to seeing you at ISOQOL in Prague!
The Psychometrics SIG Symposium will be held on Wednesday, 19 October from 4:30 pm – 6:00 pm.
This newsletter editorial represents the views of the author and does not necessarily reflect the views of ISOQOL.
How to Submit a Newsletter Editorial
Do you have something to share about health related quality of life and patient-centered outcomes? We want to hear from you!
Learn More
The International Society for Quality of Life Research (ISOQOL) is a global community of researchers, clinicians, health care professionals, industry professionals, consultants, and patient research partners advancing health related quality of life research (HRQL).
Together, we are creating a future in which patient perspective is integral to health research, care and policy.