Stroke imposes a major health care burden, but the adoption of effective interventions varies widely.1 Efforts to improve the quality of stroke management rely on rigorous outcomes data2 for avoiding misleading comparisons of hospitals. To identify potentially modifiable factors, analyses must account for casemix differences and random error.3 In particular, analyses must take stroke severity into consideration, as it is one of the strongest predictors of stroke mortality.2,4,5

Although the methodology is still evolving, standardised risk adjustment2 is highly relevant to health care consumers and policy makers. In a recent report of routinely collected hospital admissions data, significant variation in 30-day stroke mortality was found after adjusting for age, sex and comorbidities (including hypertension and diabetes), but there was no adjustment for stroke severity.6 The National Health Performance Authority (NHPA) has identified stroke as a condition for which inter-hospital differences in models of care (eg, patterns of patient transfers) and inconsistent recording of clinical information and procedures (eg, palliative care coding) may distort comparisons of mortality.7 Because hospital data must be complete, accurate and consistent, the NHPA is currently unable to support public reporting of inter-hospital disease mortality rates, as such comparisons could be unreliable.7 In contrast, the ability to reliably compare hospital performance with respect to patient outcomes has rapidly accelerated improvements in health care overseas.7

Our aim was to describe variance in 30-day stroke mortality between hospitals using risk-adjusted mortality rates (RAMRs), as part of our trialling a recently recommended new statistical method that includes stroke severity as a covariate.2

Methods

Study design

The Australian Stroke Clinical Registry (AuSCR) is a voluntary, prospective, clinical quality registry that captures standardised data for nationally agreed variables for all patients admitted to participating hospitals with acute stroke or transient ischaemic attack (TIA).8 AuSCR includes personal information (eg, name, address), clinical characteristics (eg, type of stroke), quality of care indicators (eg stroke unit treatment), and outcomes measured at discharge and at 90–180 days (eg, survival and quality of life).8 Stroke severity is captured using a simple, validated prognostic measure, the “ability to walk unaided at the time of hospital admission”.9 In the original statistical modelling by Counsell and colleagues,9 this criterion was associated with a relative risk for 30-day survival of 1.63 (95% confidence interval [CI], 1.15–2.31). In our earlier work, the strongest predictor of independence at time of hospital discharge was the ability to walk on admission (odds ratio [OR], 2.84; 95% CI, 2.18–3.71).10

Data from participating hospitals were obtained for the period from 15 June 2009 (six participating hospitals) until 31 December 2014 (40 participating hospitals). We included all stroke types (ischaemic, intracerebral haemorrhage, and undetermined) in our analyses, as well as demographic variables, as stroke mortality is higher at all ages for Indigenous than for non-Indigenous Australians,11 varies according to country of birth,12 and is greater for people of lower socio-economic status.13 Socio-economic status was assessed by matching patients’ addresses with the corresponding Index of Relative Socio-economic Advantage and Disadvantage (IRSAD) score,14 collated as quintiles. Whether the stroke was the first or subsequent stroke experienced by the patient was included as a covariate, as the risk of death is greater for recurrent events.15 Age was included as a continuous measure in all models. All episodes occurring within 30 days of admission were included. Harrell’s concordance statistic (C-statistic) was used to determine how well the variable “ability to walk on admission” predicted 30-day mortality in our models.

We excluded patients who experienced a stroke while in hospital for another condition or when transferred from another hospital, as the different patterns of care may distort mortality ratios.7,16 Data for stroke care in a paediatric hospital were also excluded because of the small sample size (fewer than 50 care episodes).

Mortality data

Survival status at 30 days was obtained by probabilistic matching of AuSCR registrant identifiers with the National Death Index (NDI) by the Australian Institute of Health and Welfare. AuSCR staff undertook the review of non-exact matches; for discordant dates, we used NDI data as the reference. Based on this linkage method, in-hospital death reporting in the AuSCR had 98.8% sensitivity and 99.6% specificity, compared with an in-hospital death determined with the NDI date of death.

Outcome measures and analyses

The primary outcome was the risk-adjusted mortality rate (RAMR) at 30 days after admission, using the method recommended by the American Heart (AHA) and Stroke Associations (ASA).2 To maximise the reliability of our estimates,17 analyses were conducted for individual hospitals that provided data on at least 200 episodes of care for stroke between 2009 and 2014. The stages in deriving each model were:

• entering the observed values;

• generating estimates from generalised linear latent and mixed models (GLLAMM) by maximum likelihood;

• generating expected probabilities;

• generating predicted probabilities;

• generating ratios predicted as expected; and

• generating the RAMR.

The RAMR for each hospital was calculated by dividing the overall RAMR by the risk-adjusted average hospital mortality, and then multiplying by the overall crude (unadjusted) proportion of deaths in the whole sample.

The results were compared in models with covariates corresponding to those available in hospital admissions data (the hospital admissions model) and after also including covariates corresponding to those available only in the AuSCR (the Registry model). A model adjusted only for age and sex was also estimated. The hospital admissions model was adjusted for age, sex, year of data, stroke subtype, IRSAD quintile, Indigenous status, and place of birth (Australia v elsewhere). The Registry model was adjusted for the same variables, as well as for stroke history and severity. Differences between the models in the ranking of individual hospitals were explored.

Data are provided on the calibration and discrimination of the models,2 using the likelihood ratio test, the Akaike information criterion (AIC), the Bayesian information criterion (BIC), and the C-statistic. A smaller AIC or BIC indicates a better fitting model; a C-statistic of 1 indicates a perfect fit model, while a C-statistic of 0.5 indicates that fit is no better than chance. Multilevel models were used, with one level defined as the hospital unit, to account for correlations between patients who were managed in the same hospital, and the other representing patients as individual units.

P < 0.001 (two-sided) was deemed statistically significant because of the large sample size. Analyses were performed in Stata 12.1 (StataCorp).

Identifying significant mortality variation and differences in mortality outcomes

According to standard practice, hospitals within two standard deviations (SDs) of the overall average RAMR were deemed to lie within normal variation, and those outside three SDs were deemed to vary significantly from the other hospitals in the sample.18 Funnel plots were used to investigate deviations from the average hospital mortality rate.19 The direction of the change was also explored by graphing the difference between the age- and sex-adjusted hospital admissions and Registry RAMR estimates.

Ethics approval

Appropriate ethics and governance approvals were obtained for all participating hospitals in AuSCR, and from the Human Research Ethics Committee of Monash University (reference, CF11/3537–2011001884). Ethics approval was obtained from the Australian Institute of Health and Welfare to conduct data linkage to the National Death Index (reference, EO 2013/2/16).

Results

Between 2009 and 2014, 26 302 episodes of care for 24 806 individual patients from 45 hospitals were recorded; 3151 patients (12%) died within 30 days of admission (excluding TIAs, 14%). The concordance of ability to walk on admission as an indicator of stroke severity with 30-day mortality was excellent (C-statistic, 0.97). Patients with intracerebral haemorrhage (29%) were more likely to die within 30 days than those with other stroke types (ischaemic, 12%; undetermined stroke, 14%; TIAs, < 1%) (online Appendix 1).

Data from hospitals reporting at least 200 episodes of stroke care

Eighteen hospitals located in metropolitan areas and ten in rural and regional areas, each with a stroke unit, provided data for at least 200 episodes of care. Hospitals in the eastern states contributed most data (Victoria, 40% of episodes; New South Wales, 17%; Queensland, 34%; Tasmania, 4%); in Western Australia (5% of episodes), only two hospitals participated. We excluded from our analysis 7509 patients who had a TIA or in-hospital stroke, or who were transferred from another hospital (online Appendix 1).

In total, 16 218 episodes of care were provided to 15 951 individual patients (median age, 77 years; women, 46%; ischaemic stroke, 79%). Compared with patients who were alive 30 days after admission, the proportion of women among those who died was greater; they were also older, fewer were able to walk on admission, and more had a history of stroke or TIA (Box 1). The characteristics of patients with stroke were similar across the 28 hospitals with respect to age, sex, and ability to walk on admission (online Appendix 2). The proportions of patients with severe strokes were similar for hospitals with more or fewer episodes of care (data not shown; P = 0.59).

Comparison of hospital 30-day mortality outcomes

The unadjusted (crude) mortality rates for the 28 hospitals with at least 200 episodes of care ranged between 7% and 23%. Excluding the 7509 patients who had a TIA or in-hospital stroke, the unadjusted mortality rates for hospitals ranged between 5% and 20%, and the age- and sex-adjusted mortality rates ranged between 8% and 20%. The RAMRs estimated by the hospital admissions model ranged between 9% and 20%, and those by the Registry model between 9% and 21% (Box 2). The overall RAMRs adjusted for different combinations of Indigenous status, country of birth and history of stroke are reported in online Appendix 3. According to the model fit statistics (BIC, AIC, likelihood ratio test, C-statistic), the Registry model had the best fit (Box 3, online Appendix 3). Correlations between the number of episodes contributed by a hospital and the differences between age- and sex-adjusted RAMRs and the Registry RAMR estimates (R² = 0.021) or hospital admissions RAMRs (R² = 0.001) were low. When the results of the hospital admissions and Registry models were compared, the variance ranged between 0% and 3%.

Although the ranges of estimates by the adjusted models were similar, the rank order of hospitals changed according to the initial crude estimate and simple age- and sex-adjusted models (Box 2; online Appendix 4). The change in ranks in the hospital admissions and full Registry models illustrates the possibility of a hospital attaining very different results. The models with the best fit were those that included stroke severity as a covariate (Box 3). Based on the funnel plot distribution, the estimated mortality for only two hospitals was more than three SDs from the mean, one with low mortality, and the other with borderline excess mortality relative to other hospitals (Box 4).

Quality of care and correlations with mortality rates

Adherence to processes of care was similar for all hospitals (online Appendix 2). Stroke unit admissions ranged from 99% for the hospital with lowest RAMR to 80% for the hospital with the highest RAMR (weak positive correlation between increased stroke unit access and lower RAMR: R² = 0.138). A negligible positive correlation was noted between increased prescription of antihypertensive drugs at discharge and lower RAMR (R² = 0.021).

Discussion

Assessing the quality of health care delivered by different health care providers is complicated by the variable quality of routinely collected hospital data.7 For burdensome conditions such as stroke, this problem is exacerbated by the inability to account for differences in stroke severity and by inaccuracies in the coding of diagnosis or cause of death.20 Clinical quality registries have emerged as important tools for resolving these problems, but support from government agencies is not as consistent in Australia as in comparable countries.

We have provided an important illustration of the value of a national clinical quality registry for stroke, using a new method for calculating mortality statistics. The models with the best fit for standardising mortality were those that included adjustment for stroke severity, a covariate routinely available only in AuSCR. The change in rank position according to different RAMRs was clearest for hospital 13, which was ranked number 6 in the full Registry model, but number 21 in the hospital admissions model. Rankings that are not based on models adequately adjusted for relevant risks can lead to interpretations that suggest that some hospitals provide substandard care, and thereby impugn their reputations and that of their clinicians. The funnel plot approach provides an alternative method for assessing performance, but the control limits associated with the assumption of a normal distribution of the data makes caution advisable, particularly if the data are skewed, as in our sample of only 28 hospitals. For the hospital with the lowest mortality in each of the models (Hospital 1), selection bias may have arisen because 99% of its patients were treated in a stroke unit (online Appendix 2), and other unmeasured factors may have also contributed to its better outcome.

Our findings differ from a previous investigation of hospital stroke mortality rates in NSW that applied more conventional modelling methods, without adjusting for stroke severity.6 Standardised 30-day mortality rates varied significantly, from 15% to 30%, and several hospitals were categorised as “poor performers”.6 Cases were sampled across different timeframes and with varying sample sizes, but there was a greater diversity of hospitals than in our study; for example, hospitals without stroke units were included.

Registry data that include disease severity risk-adjustment variables that supplement hospital data can be used to ensure that performance comparisons are more reliable. Given the growth in public reporting of hospital performance and the recognition of its potentially driving improvement of quality of care,21 it is essential that appropriate methods are employed. We estimated RAMRs using a new approach recommended by the AHA/ASA,2 replacing the observed number of deaths with a prediction of numbers of deaths estimated from the average number of deaths for hospitals in a risk-adjusted model. This reduces the influence of chance on the variation in RAMRs (predicted v expected). Our study is the first report on the application of this new approach, and our models predicted numbers of deaths within 0–9% of the actual number.

Our investigation has broader implications for Australia, in that it advances methods for hospital-level comparisons of risk-adjusted mortality, particularly on the basis of routinely collected registry data. We acknowledge, as a limitation of our study, that not all hospitals contribute data to AuSCR, and our findings may consequently not be generalisable to all hospitals. Further, our sample was restricted to hospitals reporting at least 200 episodes of care (10 084 episodes from 17 hospitals were therefore excluded from our analyses). Including all hospitals may have led to greater variance in our results, but our sample was broadly representative of the entire cohort (online Appendix 1). The overall crude 30-day death rate for eligible hospitals (with at least 200 episodes of care) was 15% (range, 7–23%), similar to reports from other countries (13–15%).4,17

Critical predictors of stroke mortality include age, sex, stroke severity, and comorbidities,18 and a further limitation of our study was the inability to adjust for comorbidities, but inconsistent reporting of International Classification of Diseases (ICD-10) coding of comorbidities is recognised.22 Future linking of AuSCR data with hospital admissions data will enable a greater range of variables to be explored. Several International stroke registries incorporate National Institutes of Health Stroke Scale (NIHSS) data that can be used for adjusting for stroke severity, but training is required to administer the NIHSS.23 The ability to collect NIHSS data was introduced in AuSCR in 2015, but the level of missing data currently undermines its usefulness, whereas “ability to walk on admission” information was available for 90% of episodes. In recent validation work,9 a model based on simple variables (including ability to walk) performed as well as one employing NIHSS and age data; the choice of measure should therefore be based on practical considerations.24 Because there were very few episodes of intracerebral haemorrhage, we included stroke type as a covariate rather than stratifying the dataset, as has been previously recommended by other authors.25

In conclusion, we highlight the importance of using appropriate risk adjustment variables and methods for comparing hospital outcomes for stroke, with particular emphasis on the need to account for stroke severity. Moreover, we have shown the value of clinical quality disease registry data for refining outcome performance measurement in health care. As this is an evolving field, further research into risk adjustment variables and comparison of mortality rates is encouraged.

Box 2 –
Comparison of ranking of hospitals according to 30-day mortality for stroke by the hospital admission and full registry models*

---

* The full data for this figure are included in online Appendix 4.

Box 4 –
Funnel plot of risk-adjusted mortality rates for hospitals (Registry model)*

---

SD = standard deviation. * The numbers for the hospitals indicate their rank according to crude mortality rates (lowest to highest). Registry model was adjusted for age, sex, stroke type, index of relative socio-economic advantage and disadvantage, Indigenous status, country of birth, year of admission, history of previous stroke, and stroke severity.

Equity of access is particularly concerning for minor stroke

Medical advances, such as stroke units, improved primary and secondary stroke prevention, and hyperacute treatments have revolutionised acute stroke management.1 The lessening of stroke severity as a result of such ground-breaking initiatives has, however, led to a larger proportion of individuals returning to community living following minor strokes2 (ie, with minimal motor deficits or no obvious sensory abnormality). In this article, we review current literature to identify the potential difficulties experienced following a minor stroke.

Individuals who survive a more severe stroke often undergo extensive multidisciplinary rehabilitation in an inpatient setting. By contrast, patients who have a minor stroke are likely to be discharged home early, often with limited referrals to services beyond their general practitioner.3 This is despite increasing evidence that survivors of minor stroke may have persisting stroke-related impairments that require rehabilitation.4 These “hidden” impairments may not become apparent until after discharge, when the patient attempts to resume their usual daily activities.2,4 Edwards and colleagues4 found that despite full independence with personal activities of daily living, 87% of patients who had a minor stroke reported residual difficulties with mobility, concentration, and participation in social activities and physically demanding leisure activities such as golf. These persisting subtle impairments may cause social and economic disruption for the individual and their family; however, due to difficulties identifying them in the hospital setting, it may result in poor coordination between primary and secondary care, especially if the patient is deemed fully independent at discharge. When the impairments are detected at a later stage, rehabilitation or support services may not be accessible, potentially rendering the patient “lost” in the health care system.

Equity of access is particularly concerning for minor stroke. In regional Australia, there may be no hospital or community rehabilitation services available,5 with patients at home dependent on the Medicare rebate for access to private allied health services within the current Chronic Disease Management (CDM) program.6 Women, who are more likely to be discharged to residential care, face further access challenges.7 Compounding this is evidence suggesting that all patients who may benefit from inpatient rehabilitation are not appropriately identified,1 which is concerning given the “hidden” nature of many impairments resulting from minor stroke.

A systematic review by Tellier and Rochette2 revealed that patients who have had a minor stroke often have impairments that span the domains of physical status, emotional health, cognition and social participation. The combined effect of these impairments may be an inability to fully resume valued activities, leading to reduced quality of life.2 Studies have shown that between one- and two-thirds of minor stroke survivors have compromised social participation outcomes.2,4 Edwards and colleagues4 found that 62% of patients who had a mild stroke had difficulty returning to employment or volunteer work, while 36% had reduced social activity 6 months after the stroke. Since about 30% of strokes occur in individuals under 65 years of age,8 these figures are particularly troubling. It is worth noting, however, that participants in the study by Edwards and colleagues4 had experienced a single ischaemic stroke and had a mean age of 64.74 years (range = 20–97 years). Therefore, as about only half of the participants4 in the study fell into the young stroke category, it is unknown how accurately these figures reflect the return to work status specifically of younger patients who had a minor stroke.

The 2014 National Stroke Foundation Rehabilitation audit9 found that less than 40% of patients who had a stroke received a psychological assessment before discharge. Formal neuropsychological assessment is expensive and not available in many areas and so inpatients rarely receive one, even if experiencing obvious impairments, such as aphasia or pronounced memory deficits. For people who have had a minor stroke, impairments are even less obvious and may manifest as a diverse range of milder cognitive problems, including attentional neglect or reduced processing speed. A neuropsychological assessment could identify these deficits and their impact on functioning and make recommendations for compensatory strategies or adjustments to reduce this impact.

Mental health problems, in particular depression, are prevalent regardless of stroke severity, with 25–29% of patients who have had a minor stroke reporting depression in the first year.10,11 Early and late onset post-stroke depression has been associated with disability and poor physical and mental health at 1 year,11 and with a reduced likelihood of driving a vehicle, participating in sports or recreational activities and interpersonal relationships at 6 months after the stroke.12 It is encouraging that improvement of depression within the first year after the stroke has been associated with better functional outcomes and quality of life.10 This highlights the need to regularly monitor patients after a minor stroke to identify and treat depression as soon as possible. Despite apparent good recovery, depression is a risk and some patients require referral to services, medication and psychological support in a coordinated manner.

As with most patients who have had a stroke, patients who have had a minor stroke are usually unable to drive for a period of time, relying instead on public transport, family members or unapproved driving for transport to medical appointments and other destinations. Research has found that one in four young survivors of stroke (aged 18–65 years) return to driving within 1 month after the stroke, despite recommendations to the contrary.13 Drivers who have had a minor stroke perform significantly worse on complex tasks, with greater cognitive load (eg, turning across oncoming traffic and bus following), and make twice the number of driving errors compared with control subjects.14 In addition to the detrimental influence of spatial, visual and cognitive impairments, the risk of seizure contributes to the moratorium on driving after a stroke. Premature return to driving may reflect poor compliance with advice, which is perceived as inconvenient and perhaps not fully explained to patients. Providing patients who have had a minor stroke with education about driving restrictions and alternative transport options and ongoing monitoring of driving fitness should be part of primary health care.

Patients who have had a minor stroke are also at risk of hospital re-admissions due to other medical conditions. For example, patients who have had a minor stroke have a heightened risk of experiencing a subsequent cardiovascular event.15 They may also have an array of concomitant medical conditions, including diabetes mellitus, atrial fibrillation and congestive cardiac failure,15 and may benefit from a coordinated approach to manage these comorbidities and prevent hospital re-admission.

Six months after a minor stroke, patients do significantly less high intensity physical activity compared with the activity done before the stroke, and despite the benefits of physical activity for future stroke prevention, they tend not to take up new high intensity activities.12 Indeed, Kono and colleagues16 found that higher levels of exercise in the form of daily step counts were associated with a reduced risk of new vascular events following minor stroke. Patients who have had a minor stroke and are living in the community may benefit from education about secondary stroke prevention. A GP-led multifaceted and target-based approach to secondary stroke prevention may be ideal for this population, especially given that a combination of medications (eg, aspirin, a statin and an antihypertensive agent), exercise and dietary modifications have been found to produce a cumulative relative risk reduction of stroke by 80%.17

Conclusion

In summary, minor stroke is a chronic health condition with long term impairment and disability.2 Residual impairments and comorbidities often require the involvement of multiple health care providers, the need for which may not always be evident at the time of stroke. Community-living patients who had a minor stroke may currently be managed through initiatives such as the CDM program. Access to CDM items can be problematic and, due to the mild nature of minor stroke, it is likely that these items will be overlooked. The five sessions per calendar year under the CDM program — which include a range of allied health services, such as speech pathology, occupational therapy, psychology and physiotherapy, with a Medicare rebate that may cover the total cost depending on whether the provider accepts the Medicare benefit as full payment for the service — are often inadequate for patients who have a more complex situation, but may be ideal in the population who have had a minor stroke and hence, a good use of existing resources. Therefore, we need to audit existing strategies in primary care to uncover which processes are working well, and which require attention. This is particularly pertinent given the creation of new government initiatives, including the National Disability Insurance Scheme (in which, however, patients who have had a minor stroke look unlikely to be eligible), and Primary Health Networks within the Health Care Home framework.

A GP-led approach that coordinates a range of primary and allied health professionals close to the home of patients who have had a minor stroke may be the ideal way to meet the needs of this population and prevent costly re-admissions to hospital, while simultaneously maximising quality of life. To ensure that community-dwelling patients who have had a minor stroke and have unmet needs are not missed, we need a coordinated, integrated primary health care response that detects and manages impairments and activity restrictions as they arise, along with medical comorbidity management and self-management support. At a minimum, we need to ensure that all patients who have had a minor stroke, regardless of their geographic location, have improved access to formal neuropsychological assessment, falls prevention, exercise programs and more extensive Medicare-based allied health funding if required. The key to this is auditing existing programs and investigating the relevance of new government initiatives as they arise for these patients, while also improving the communication between hospitals and primary health care services. Further study of the unmet needs and mechanisms for ensuring access for all patients who have had a stroke is also vital.