Setting

The University Clinical Psychology Service (SCUP) within the University of Padua, Italy, established in 2017, comprises various mental health services for students, employees and the public. The service remains open throughout the year, except during holidays, but the number of users typically decreases in June and July. The Psychological Assistance to Students – Psychological Consultation (SAP-CP), a component service of SCUP, provides free-of-charge diagnostic assessment and psychological consultation to students from the University of Padua who refer with psychological issues. These interventions may involve short- and medium-term individual or group sessions utilising cognitive-behavioural or psychodynamic approaches. A detailed description of SCUP and SAP-CP is included in the supplementary materials (description of the SAP-CP service). Requests to SAP-CP increased from 631 in 2021 to 656 in 2022 and 775 in 2023 (increase of 23% between 2021 to 2023). Average waiting times, i.e. between request for access and first contact, over the study period ranged from 2 to 6 weeks. For reference, the student population increased from 65 936 in the 2021/2022 academic year to 68 701 in 2022/2023, a relative increase of 4% (https://www.unipd.it/dati-statistici-iscritti).

Study design

We conducted a retrospective, single-group, ITS of RCD from the student mental health service at the University of Padua, SAP-CP, part of the integrated university mental health service, SCUP. The goal was to assess whether provision of an unguided self-help intervention while on the waiting list for clinical evaluation and face-to-face counselling or therapy could reduce depression and anxiety, and improve functioning, at the start of treatment. Specific reporting guidelines for ITS designs are under development (https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-observational-studies/#CARITS). Therefore, the study was reported according to the STROBE checklist for cohort studies. ^{Reference von Elm, Altman, Egger, Pocock, Gøtzsche and Vandenbroucke16}

Consent statement

Users contacting SAP-PC sign an informed consent form for personal data processing, agreeing to have their data processed for service evaluation and research. Because we used these pre-existing real-world data from individuals who had previously consented to the use of their data for research, no additional ethical board approval was sought.

Participants

Participants were recruited from the 797 Italian students (67% women, n = 534) who voluntarily requested access to SAP-CP psychological services between December 2019 and September 2023.

We excluded users who had spent fewer than 5 or more than 150 days on the waiting list, i.e. time elapsed between receiving the DWM booklet and completing the outcome measures. The threshold of 5 days was chosen to ensure that users had enough time to read through the booklet; the limit of 150 days was chosen to exclude participants with very long waiting times, during which symptoms might have remitted on their own. We excluded 47 participants (27 spent fewer than 5 and 20 spent more than 150 days on the waiting list). We also excluded one user who requested earlier access compared with others (in December 2019, whereas all others gained access from February 2021). With these exclusions, the sample included 749 Italian students (77% women, n = 499) who requested access to the service between February 2021 and September 2023, and who had completed the three routinely collected assessment scales between October 2021 and November 2023. The sample consisted of users who either received DWM (55%, n = 411) or did not (45%, n = 338) before completing the scales. A CONSORT-type flowchart of the selection of study participants is shown in Fig. 1. The number of participants completing the outcome measures in each month during the study period is presented in Supplementary Table S1 available at https://doi.org/10.1192/bjo.2025.10843.

Fig. 1 Flowchart showing the selection of study participants. DWM, Doing What Matters in Times of Stress.

Procedure

ITS

ITS is considered a reliable quasi-experimental design when it is not possible to implement RCTs. ^{Reference Penfold and Zhang17} It is suitable when a single unit (i.e. dependent variable) is being studied progressively on an ordered time series that has been interrupted once or multiple times due to the introduction of a generic intervention. ^{Reference Shadish, Cook and Campbell18–Reference Linden20} ITS is considered a powerful tool for evaluating the impact of social or public health interventions, ^{Reference Turner, Karahalios, Forbes, Taljaard, Grimshaw and Cheng21} and also when applied retrospectively on pre-existing data (e.g. ^{Reference Baker and Alonso22} ). The effect of the intervention is estimated accounting for the pre-intervention trend, testing potential step changes immediately after the intervention (i.e. level change or intercept change), and on its progression over time following the introduction of the intervention (i.e. slope change ^{Reference Ramsay, Matowe, Grilli, Grimshaw and Thomas23} ). This is computed with a so-called ‘segmented regression’ describing two regression lines, each with a different intercept and slope, for before and after the intervention. ^{Reference Penfold and Zhang17} The post-intervention line is then compared with the counterfactual scenario, representing what we expect would have happened if the intervention had never taken place.

Outcome measures

The Patient Health Questionnaire-9 (PHQ-9) is a self-report questionnaire, with items indexing the 9 diagnostic criteria for major depression, ^{Reference Kroenke, Spitzer and Williams24} and was translated into Italian. ^{Reference Rizzo, Piccinelli, Mazzi, Bellantuono and Tansella25} Scores range from 0 to 27, with each item rated on a 4-point Likert scale from 0 (‘not at all’) to 3 (‘almost every day’). Generally, a cut-off of 10 is used to indicate moderate depressive symptoms. ^{Reference Levis, Benedetti and Thombs26}

The Generalised Anxiety Disorder-7 (GAD-7) is a 7-item, self-report questionnaire initially developed to index the diagnostic criteria for generalised anxiety disorder (GAD), ^{Reference Spitzer, Kroenke, Williams and Löwe27} validated on an Italian sample. ^{Reference Shevlin, Butter, McBride, Murphy, Gibson-Miller and Hartman28} The items are valued on a 4-point Likert-type scale from 0 (‘not at all’) to 3 (‘nearly every day’), with scores ranging from 0 to 21. GAD-7 has been validated as a screener of anxiety not linked to a specific diagnosis in the general population in a nationally representative sample of German adults ^{Reference Löwe, Decker, Müller, Brähler, Schellberg and Herzog29} and one of Finnish adolescents, ^{Reference Tiirikainen, Haravuori, Ranta, Kaltiala-Heino and Marttunen30} as well as in various studies on community samples. A cut-off of 8 was shown as indicative of an anxiety disorder. ^{Reference Plummer, Manea, Trepel and McMillan31}

The Work and Social Adjustment Scale (WSAS) is a 5-item, self-report measure of functional impairment attributable to a specific problem. ^{Reference Mundt, Marks, Shear and Greist32} Scores range from 0 to 40, and every item can be rated from 0 (‘no impairment’) to 8 (’severe impairment’). Scores between 10 and 20 were associated with significant functional impairment but less severe psychopathology, while scores above 20 indicate moderately severe or worse functional impairment and psychopathology. ^{Reference Mundt, Marks, Shear and Greist32}

Typically, assessment scales were completed during the preliminary clinical interview at SAP-CP, either manually or digitally.

Intervention

Data analysis

As data preparation, questionnaires completed were manually inserted, while individual responses were downloaded in separate monthly data files from the data collection software, custom made by OpenView for SAP-CP. The data were then compiled into a single database, along with sociodemographic information and details about the duration of treatment. Descriptive analyses were conducted using the gtsummary R packageReference Sjoberg, Whiting, Curry, Lavery and Larmarange ³⁵ in RStudio (version 4.3.2, Posit PBC, Boston, Massachusetts, USA; see http://www.posit.co/) for macOS, to assess differences in outcome measures, sample and subsequent treatment characteristics between users who received the intervention (n = 411) and those who did not (n = 338). The analysis included the mean, standard deviation, median, range and statistical tests. Pearson’s chi-squared (χ ²) test analysed differences in gender, and the Wilcoxon rank-sum test was used to evaluate differences between groups regarding age, number of psychological sessions attended and days spent on the waiting list.

Single-group ITS analysis was performed on each of the three outcome measures for evaluation of the effect of DWM on depression, anxiety and functioning in users that progressed to psychological face-to-face intervention in SAP-CP between October 2021 and 23 June 2022 (before DWM was introduced), and between 24 June 2022 and 18 November 2023 (after DWM was introduced). Individual answers from October 2021 to November 2023 (n = 749) were aggregated at the level of the month, computing total score means and standard deviation for each time point. Three pairs of adjacent months (April and May 2022, July and August 2023, September and October 2023) were gathered in single time points, because of the small number of overall users accessing SAP-CP in each of these months (see Supplementary Table S1 for details). While the limited number of users in summer might be expected, the autumn 2023 fluctuation could be explained by the fact that the academic year had not yet begun. The limited number of users accessing the service in the spring of 2022 could have been due to the Easter holidays, during which many students were off campus.

Consequently, 8 pre-intervention and 14 post-intervention time points were considered in the final analysis. Single-group ITS was carried out for each scale with linear modelling using generalised least squares (‘gls’ function in the nlme R package ^{Reference Pinheiro and Bates36} ), which allows accounting for potential autocorrelation of data, ^{Reference Fox and Weisberg37} and also maximum likelihood estimation. For each scale, the uncorrelated model was first fitted, setting the following variables as main predictors: time (a numeric variable representing the order of time points), intervention (a dummy variable indicating whether the intervention has taken place) and post-intervention time (a numeric variable indicating the time elapsed since the beginning of the intervention). The risk of seasonality, non-stationary, autocorrelation (ACF) and partial autocorrelation (PACF) was controlled both graphically, with the use of ‘Acf’ and ‘Pacf’ functions on model residuals from the forecast R package, ^{Reference Hyndman, Athanasopoulos, Bergmeir, Caceres, Chhay and Kuroptev38} and inferentially through the Durbin–Watson test for autocorrelation ^{Reference Durbin and Watson39} and Augmented Dickey–Fuller Test for non-stationarity. ^{Reference Said and Dickey40} To improve model fit, a number of autocorrelated models with different autoregressive orders (p) and moving average (q) (i.e. ARMA) were compared using the ‘auto.arima’ function in the forecast R package. This function searches for a range of p and q values, after fixing the stationarity differencing index (d) by the Kwiatkowski–Phillips–Schmidt–Shin test. Finally, the function returns the model with the lowest Akaike information criterion score. If suggested by the function, the resulting generalised ARMA correlation structure was then specified for fitting the improved gls model, which was then compared with the counterfactual model of equivalent ARMA structure. The same procedure was performed for each of the three scales.

As a robustness check for the main ITS analysis, we tested differences between pre- and post-intervention trends in time by estimating the slope of each time series using the Kendall–Theil–Sen estimator (zyp R package ^{Reference Bronaugh and Schoeneberg41} ). This is a robust, non-parametric method that minimises the influence of outliers, useful for limited datasets and residuals not normally distributed. ^{Reference Sen42,Reference Wilcox43} To ensure reliable inference, we applied bootstrap resampling (100 000 iterations) to estimate confidence intervals for the slopes in the two periods (boot R package ^{Reference Canty and Ripley44} ). A one-sided Wilcoxon rank-sum test was performed on the bootstrapped slope distributions to test whether the pre-intervention slope was significantly greater than the post-intervention. Additionally, we visualised the distributions of the bootstrapped slopes to compare trends before and after the intervention (Fig. 2).

Fig. 2 Density distribution of bootstrapped slope estimates for Patient Health Questionnaire-9 (PHQ-9), Generalised Anxiety Disorder-7 (GAD-7) and Work and Social Adjustment Scale (WSAS) total scores (by horizontal facet) in the pre-intervention phase (blue) and in the post-intervention phase (orange).