Introduction
Drug utilization reviews to improve antibiotic prescribing in the inpatient setting is labor intensive; metric driven tools are mostly limited to reporting of location-specific usage rates to the National Healthcare Safety Network (NHSN). Reference Fridkin and Srinivasan1,Reference Davey, Marwick and Scott2 While these metrics are valuable for monitoring usage, they lack adequate risk adjustment and ability to attribute prescribing to specific providers. Reference Goodman, Pineles and Magder3 While provider-specific metrics for peer-comparison to motivate behavior change have been proven effective in the ambulatory setting, technical challenges make similar activities in the inpatient setting scarce. Reference Linder, Meeker and Fox4,Reference Allen, Dunn and Bush5
While patients, providers, and payors have been increasingly asking for physician benchmarks for quality improvement, effective metrics must be credible to providers to influence their behavior. Reference Burstin, Leatherman and Goldmann6,Reference Meddings, Reichert, Hofer and McMahon7 The reliability of quality measures considers variance as well as persistence. Reference Ranganathan, Hibbard and Rodday8,Reference Scholle, Roski and Adams9 To improve credibility of an Emory Healthcare pilot program benchmarking inpatient antibiotic prescribing among hospitalists, we assessed whether providers with high prescribing metrics repeatedly prescribe high without intervention. Reference Onwubiko, Mehta and Wiley10
Methods
Design and study population
We conducted a retrospective cohort study of antibiotic prescribing patterns using administrative data from three Atlanta hospitals from January–December 2023. Hospital A is a 582-bed comprehensive academic medical center; B and C are 537-bed and 373-bed specialized complex medical centers. All utilize dedicated hospital medicine services providing general medical care across multiple locations. Antibiotics administered to patients on dates billed by specific hospital medicine attendings were attributed to the billing attending, regardless of emergency medicine initiation or consultation involvement.
For each hospitalist, we used billing data to identify patient encounter dates and calculated billed patient-days (bPD). We extracted antibiotic prescribing data from electronic medication administration records (eMAR). For each bPD, matching transaction dates with NHSN-defined BSHO group antibacterial orders were captured as days of antibiotic therapy (DOT). Patient characteristics included demographics, age, microbiology results, ICD-10 antibiotic indications (pneumonia, COVID-19, sepsis, UTI), and comorbidities for Charlson score calculation. The methodology to derive the prescribing metric, an observed-to-expected ratio (OER), has been described elsewhere. Reference Onwubiko, Mehta and Wiley10 Data were aggregated into bi-monthly intervals for each provider to reduce data sparsity from variable work schedules and minimizing imprecision for providers with less frequent billing practices. OERs were generated for NHSN-defined antibiotic groups: broad-spectrum hospital-onset (BSHO, mostly anti-pseudomonal agents) and anti-methicillin-resistant Staphylococcus aureus (Anti-MRSA).
Analytic approach
Variance of BSHO and Anti-MRSA OERs were evaluated among providers at each hospital; roughly 25% of the providers had OERs >1.25 (25% prescribing > predicted). We employed two analytic approaches to evaluate persistence in prescribing behavior. First, we constructed transition matrices (Markov chain modeling) estimating immediate transition probabilities between any pair of sequential periods across three prescribing categories: low (OER <0.75), medium (OER 0.75–1.25), and high (OER >1.25), representing lowest quartile, interquartile range, and highest quartile, respectively. The immediate transition probabilities between states are represented by the values along each arrow in Figure 1. Steady-state probabilities were also calculated reflecting long-term probability (as system approaches infinite) the providers would remain in each prescribing state (values in each circle in Figure 1).
Second, we used log-binomial generalized estimating equations (GEE) quantifying associations between high prescribing in prior periods and subsequent periods. Models accounted for repeated measures and clustering at provider and facility levels. Separate models were estimated for BSHO and anti-MRSA agents, adjusting for sex and years since graduation as proxy for age and experience. Analyses used R Statistical Software (v4.2.0).
Results
Across the three hospitals, 70 hospitalists (32, 45% at A; 20, 29% at B; 18, 26% at C) contributed data to six bi-monthly OERs during the 12–month study period resulting in 420 bi-monthly observations; this created 350 transitions of reporting metrics from one period to the next period. Per period, providers cared for a mean 132 patients (range 63–306) over 329 patient-days (range 165–478). Providers prescribed BSHO antibiotics at a mean 110 DOT per 1 000 patient-days (range 50–200) resulting in a mean OER of 0.96 (range 0.60–1.56), with provider-level variance in O:E ratios ranging from 0.29 to 1.63 (mean: 0.83). For Anti-MRSA antibiotics, providers prescribed a mean 86 DOT per 1 000 patient-days (range 9–191) resulting in a mean OER of 1.0 (range 0.12–2.34), with provider-level variance in O:E ratios ranging from 0.10 to 1.49 (mean: 0.80).
Markov chain modeling revealed long-term probabilities for providers to prescribe in each OER category. Hospitalists spend approximately half the time in the medium state with remaining time split between high and low prescribing states (Figure 1, circles). Long-term probabilities were similar between antibiotic groups: providers most often had OERs in middle category (45% and 50% for BSHO and anti-MRSA, respectively), then lowest category (31% and 27%), then highest category (24% and 23%). For immediate transitions, most providers remained in the same category (Figure 1, curved arrow). These probabilities were similar between BSHO and anti-MRSA agents at roughly 50% (range 44%–51%). However, for BSHO agents the likelihood of remaining at a high OER between subsequent periods was 49% more than double the 24% for anti-MRSA agents (Figure 1, curved arrows). The probabilities to move progressively up from a lower OER category were similar between antibiotic groups: 7–12% from low OERs to high OERs and 44–45% from low OERs to medium OERs (Figure 1, solid arrows). The probabilities to move progressively down from a higher OER to a lower OER was also similar between antibiotic groups (Figure 1, dashed arrows).
Log-binomial GEE regression models quantified persistence of high prescribing behavior. High BSHO prescribers were three times more likely to prescribe high in the next period compared to other prescribers (adjusted relative risk 3.72, 95% CI: 2.45–5.65), controlling for years since graduation and sex (Table). This indicates strong temporal persistence for BSHO agents. In contrast, for anti-MRSA agents, this persistence was not observed (adjusted rate ratio 0.46, 95% CI: 0.23–0.94).
Discussion
Hospital medicine providers prescribing high amounts of BSHO antibiotic agents during one period are likely to continue prescribing high rates subsequently; however, this did not hold for anti-MRSA prescribing. This adds credibility to our BSHO OER as an internal performance metric. Specifically, persistence of “high” prescribing suggests chance alone does not adequately explain providers receiving high rates in any period. While the overall likelihood for any provider to prescribe high was only 24%, when in that category, there was roughly 50% chance of remaining there in the next period versus only 25% for anti-MRSA agents. We interpret these findings to mean general provider behavior influences BSHO prescribing and does not greatly vary over 4–month periods. Conversely, other factors outside provider behavior influence anti-MRSA prescribing as providers move in and out of high prescribing categories more often. Potential influencing factors include MRSA nasal screening swabs, which providers are encouraged to use for ruling out MRSA pneumonia; these results dramatically affect anti-MRSA de-escalation decisions. These data suggest providers prescribing high amounts of BSHO antibiotics will likely continue this pattern without intervention, creating opportunities for infectious disease pharmacist engagement or individual reflection on de-escalation. However, receiving high prescribing values doesn’t equate with inappropriate prescribing. We lack sustainable processes for appropriate diagnosis or treatment review. Additionally, patient factors affecting BSHO may not be fully captured by our OER calculations. Reference Onwubiko, Mehta and Wiley10
Despite limitations, this metric is informative for targeting providers with high BSHO usage, suggesting their practices don’t occur simply by chance. As inpatient quality efforts and performance feedback metrics gain strength, evaluating attributes of potential performance metrics, such as persistence, should be added to considerations of reliability, accuracy, and clinical credibility. Reference Meddings, Reichert, Hofer and McMahon7,Reference Scholle, Roski and Adams9
Figure 1. Markov Chain Plot, long-term probabilities (text within circles) of metric being high (OER > 1.25), medium (0.75 ≤ OER ≤ 1.25), or low (OER < 0.75), and immediate transition probabilities of metric change between subsequent periods progressing to higher state (solid arrow), lower state (dashed arrow), or remaining in same state (curved arrow), by antibiotic category.
Table 1. Association between current high prescribing (OE ratio >1.25) and subsequent high prescribing using log-binomial GEE regression models for broad-spectrum hospital-onset and anti-MRSA antibiotics
* Adjusted for sex and years since a provider graduated from medical school, accounting for clustering by facility and provider.
Acknowledgments
Not applicable.
Authors contributions
RPA and SF contributed to conception of the study; RPA, CM, and SF, contributed to design; SF, KAJ, RPA, SP, SS; to data acquisition and analysis and interpretation, RPA, LW, and SF to drafting and revision of manuscript. All authors read and approved of the final manuscript.
Financial support
This study was supported by the Emory Prevention Epicenter Program (PEACH) through the Centers for Disease Control and Prevention (CDC) [U54CK000601].
Competing interests
The authors declare that they have no competing interests.
Ethics approval and consent to participate
Study was approved with expedited review, with a complete waiver of HIPPA authorization and informed consent.
Consent for publication
Not applicable.
Disclaimer
The content is solely the responsibility of the authors and does not necessarily represent the official views of the Centers for Disease Control and Prevention.
Research transparency and reproducibility
The datasets generated and/or analyzed for the current study are to be available in the Harvard Dataverse (link: https://dataverse.harvard.edu/).
Open access
