Forsell, E., Mattsson, S., Hentati Isacsson, N., & Kaldo, V. (2025). Accuracy of therapists’ predictions of outcome in internet-delivered cognitive behavior therapy for depression and anxiety in routine psychiatric care. Journal of Consulting and Clinical Psychology, 93(3), 176–190. https://doi.org/10.1037/ccp0000943
Key Takeaways
- Aims: The study aimed to examine the predictive accuracy and confidence in predictions of ICBT-therapists in routine care settings. It also sought to compare therapist predictions to a statistical benchmark using patient symptom scores.
- Findings: Therapists’ predictions were more accurate than chance, but their balanced accuracy was lower than that of a statistical model. Therapists also demonstrated a tendency to be overly optimistic in their predictions.
- Implications: The findings suggest that statistical models are needed to facilitate clinically useful predictions of patient outcomes for decision support in clinical practice.
Rationale
Research indicates that a significant percentage of patients do not achieve satisfactory responses in psychological treatments, including ICBT.
Early identification of failing treatments could help prevent failures for at-risk patients.
However, previous research suggests that therapists are not good at predicting patient outcomes in traditional face-to-face psychotherapy.
It is unclear if this holds true for ICBT, where therapists have access to structured data on patient progression. This study aims to address this gap by examining the predictive accuracy of ICBT therapists.
Method
This was a prospective study in regular care.
ICBT therapists’ predictions of patient outcomes were made during the fourth week of treatment and compared to observed outcomes.
Sample:
897 patients: depression (n=373), social anxiety disorder (n=273), panic disorder (n=251), predominantly female (60-66%), average age between 31-37.
Procedure:
- Therapists predicted patient outcomes in week 4 of treatment.
- They reviewed weekly patient-reported symptom scores and treatment progression graphs.
- Predictions involved estimating symptom score changes and categorical outcomes (remission, response, deterioration).
- Predictions were compared against actual patient outcomes and a statistical model.
Measures:
- Montgomery-Åsberg Depression Rating Scale-Self-report version (MADRS-S): Used for MDD, scores range from 0 to 54.
- Leibowitz Social Anxiety Scale-Self-report version: Used for SAD, scores range from 0 to 144.
- Panic Disorder Symptom Scale-Self-Report (PDSS-SR): Used for PD, scores range from 0 to 28.
Statistical Measures:
- Balanced accuracy (classification predictions)
- Pearson correlation (continuous outcomes)
- Linear regression analyses
Results
- ICBT therapists’ predictions of outcomes were better than chance, but still below a benchmark based on a statistical model using weekly symptom data to predict outcome.
- ICBT therapists are optimistic in their predictions, as they predicted positive categorical outcomes more often and larger overall symptom reduction than occurred.
- Confidence in one’s predictions differs between therapists.
- Higher therapist confidence was weakly related to correctness in continuous outcome.
Insight
The study reveals that while ICBT therapists can predict treatment outcomes better than chance, their accuracy does not surpass that of a statistical model.
This suggests that relying solely on therapist predictions may not be optimal for identifying patients at risk of treatment failure.
The finding that therapists are overly optimistic aligns with previous research. It highlights the need for external decision support tools to enhance treatment effectiveness.
The study also found that therapist confidence in their predictions varies, but this confidence is not strongly associated with prediction accuracy.
This suggests that even when therapists are confident, their predictions may not necessarily be more accurate.
Further research is needed to explore factors that may improve prediction accuracy and to develop effective tools for personalized and precision care in mental health.
Clinical Implications
The findings imply that relying solely on therapist predictions may not be sufficient for identifying patients who need additional support or personalized interventions.
Implementing statistical prediction models in clinical practice could provide more accurate and useful predictions of patient outcomes.
This could aid in decision-making and improve treatment effectiveness for struggling patients.
Strengths
- Real-world clinical setting enhances ecological validity.
- Comprehensive statistical benchmarking provides meaningful interpretation.
- Large, representative patient sample.
Limitations
- Therapist adherence to prediction tasks varied.
- Predictions lacked consequences or incentives, possibly affecting accuracy.
- Therapist variability complicates interpretation.
Socratic Questions
- What ethical considerations arise from relying on statistical predictions versus clinical judgment alone?
- What factors might improve therapists’ predictive accuracy in future studies?
- How might integrating statistical tools alter therapist-patient interactions?
- Could therapist optimism, despite inaccuracy, positively influence patient outcomes?
- How would therapist training in statistical models impact clinical effectiveness?
Reference
Forsell, E., Mattsson, S., Hentati Isacsson, N., & Kaldo, V. (2025). Accuracy of therapists’ predictions of outcome in internet-delivered cognitive behavior therapy for depression and anxiety in routine psychiatric care. Journal of Consulting and Clinical Psychology, 93(3), 176–190. https://doi.org/10.1037/ccp0000943
