The IMMERSE blog

Please follow our Twitter account @Immerse_Project to stay tuned.

The IMPROVE pilot study

Last week marked a significant milestone — I defended my PhD! In today’s blog, I want to focus on the final study of my thesis: the IMPORVE study.  

In a nutshell, the IMPROVE study was a trial run for testing ESM's usability in a clinical setting. We piloted a carefully crafted ESM protocol — a how-to guide for using clinical ESM software, which involved a nifty practitioner dashboard and a client app, complete with templates containing ready-to-use questionnaires, sampling plans, visual aids, and other additional features.

The good news? Both practitioners and clients gave a nod to the technical side — usability of the ESM software was considered sufficient to good. But here's where it gets interesting. Beyond the tech success, IMPROVE brought to light several challenges of integrating ESM into the day-to-day workings of clinical practice. In what follows, I want to address three of those challenges — for additional challenges, I refer to our publication.

A first challenge revolves around the limitations of generic templates, which consist of standardized ESM questions and sampling schemes. In clinical implementation, the significance of personalization is evident. The ability to tailor questionnaires and sampling schemes to individual needs is crucial for unlocking the full potential of insights derived from ESM. In the IMPROVE study, we used generic templates with the option for personalization enabled. However, the practical implementation of personalization faced hurdles. Practitioners encountered difficulties due to technical intricacies and time constraints inherent to the fast-paced clinical practice environment. These encountered difficulties highlight a gap between the ideal conceptualization of person-tailored approaches and pragmatic constraints such as limited therapeutic time. Addressing such constraints is crucial for realizing the full potential of ESM in clinical practice.

A second challenge relates to incorporating ESM data visualizations into clinical sessions. The findings illustrated that the simultaneous display of multiple visualizations could lead to practitioners and clients feeling overwhelmed. The key takeaway from this is the necessity for a more practical and user-friendly approach. One proposed solution involves the implementation of dynamic visualization interfaces, allowing the option to hide or display specific sets of visualizations at any given time. The hypothesis is that such an interface could improve user focus, streamline information delivery, and simplify the navigation of the collected data. However, the effectiveness of this approach remains speculative and requires empirical testing for validation.

A third and final challenge I will discuss as a part of this blog concerns factors influencing compliance, expressed as the percentage of completed ESM questionnaires as compared to provided ESM questionnaires. In the IMPROVE study, the variability in compliance rates ranged from below 30% to above 80% among different clients. Understanding the reasons behind such fluctuations is crucial, as the usability of ESM will largely depend on whether clients will be able to complete a sufficient number of ESM assessments. Unfortunately, our pilot study was too small to draw strong conclusions. Yet, one speculative explanation is that it may relate to client’s perceived benefit of collecting ESM data. Whether this factor is sufficiently explanatory, or whether other factors are involved, is a topic for future research.

Now, why did I address these challenges specifically? Well, within IMMERSE our work continues where IMPROVE ended. We are focusing on making personalization as easy and time-efficient as possible, we are working hard towards optimizing a user-interface for data visualizations, and above all are studying what makes it work. Hence, moving forward, I hope to return to this blog and write to you that we have overcome the challenges I outlined today. Hence, the journey is far from over, and even more reason to stay tuned!