As the cornerstone of health care, primary care should be effective, efficient and responsive to the needs of patients, families and communities.1 The Australian primary care system currently has significant opportunities to co-create approaches to quality improvement (QI) and practice redesign in ways that could fundamentally improve health care in Australia. To ensure these efforts are successful, there is a need to build and sustain the ability of primary care practices to engage in QI activities in a systematic, continuous and effective way.2 Primary Health Networks (PHNs) are a central component of the government’s primary health care reforms and have a number of roles in improving the quality of primary care. They will work closely with general practices and other primary care services in planning and supporting primary care teams to adopt QI initiatives, with the overall aim of improving care delivery and health outcomes.3,4

The Australian Commission on Safety and Quality in Health Care has recently developed a national set of practice-level indicators of safety and quality for primary health care. These indicators are designed for voluntary inclusion in QI strategies at the local practice or service level and are intended for local use by organisations and individuals providing primary health care services.5 The Australian Safety and Quality Framework for Health Care was also developed, which sets out the actions needed to achieve safe and high-quality care for all Australians.6 To respond effectively to these, primary care practices will need to be equipped with both organisational development and change management approaches. In response, the Primary Care Practice Improvement Tool (PC-PIT), an organisational performance improvement tool, was co-created for Australian primary care.7

The co-creation methods involved combining results from a systematic literature review8 and pilot study9 with cyclical feedback from partners and end users (general practices), using a variety of engagement platforms (interviews, face-to-face meetings, formal presentations, workshops and webinar discussions).10 The systematic literature review identified 13 elements integral to high-quality practice performance, which was defined as “systems, structures and processes aimed to enable the delivery of good quality patient care” but which do not necessarily include clinical processes.11 During discussion sessions, stakeholders and end users identified a list of desired attributes for a performance improvement process; namely, that it should be simple, accessible online, enable a “whole-of-practice” approach, be an internal process facilitated by practice managers or nurses without the need for extensive external facilitation, have additional support resources, be no or low cost and fit with existing QI or practice support programs.

The tool resulting from this combination of processes, the PC-PIT, was initially piloted with six high-functioning practices.9 In this study, we trialled the refined PC-PIT approach nationwide with three objectives: (i) to document and describe the use of the online PC-PIT in practice; (ii) to validate the PC-PIT independent practice visit objective indicators; and (iii) to identify the perceived needs (eg, resources, professional development) to support practice managers as leaders of organisational improvement in general practice.

Methods

We conducted the national PC-PIT trial from March to December 2015, with volunteer general practices from a range of Australian primary care settings. Practices were invited to participate using newsletter information sheets distributed by partner and stakeholder organisations and through national webinars and conference workshops. We used a mixed-methods approach, collecting both quantitative and qualitative data. A full description of the trial protocol is available in Appendix 1. Ethics approval for the study was granted by the University of Queensland Behavioural and Social Sciences Ethical Review Committee (approval number 201000924).

The PC-PIT trial consisted of two parts. In Part 1, practice staff at all participating practices completed the online PC-PIT, with each practice staff member giving a subjective assessment of how they perceived their practice met (or did not meet) the best practice definition of each of the 13 PC-PIT elements, using a 1–5 Likert rating scale.

For Part 2, we selected a purposeful sample of the primary care services to represent a range of practice sizes, business models and geographic locations. Two external raters conducted an independent practice visit to each selected practice, during which they assessed the subjective practice assessment from Part 1 against objective indicators of the same 13 PC-PIT elements, as supported by documented practice evidence. Each rater completed a separate evidence assessment form and ranked a set of objective indicators for each PC-PIT element (using a 1–5 Likert scale) by reviewing the documented practice evidence; working with both practice managers and practice nurses to identify and cite the evidence. Box 1 provides a summary of the Likert ratings and how they are interpreted in the context of the PC-PIT elements.

Data were gathered from a variety of sources during the independent practice visits, including interviews with practice managers and practice nurses, background materials and documented evidence such as meeting minutes, policy and procedure manuals, and communications books. Well documented strategies to enhance trustworthiness and rigour were incorporated in the qualitative phase of the study design; namely, that the two raters cited the sources, which allowed the triangulation of information and the review and confirmation of findings.12,13 The in-depth, semi-structured interviews with practice managers and practice nurses explored their involvement in QI and also their perceptions of resources and support needed to facilitate their role in performance improvement. A proforma was used to guide the interview discussions and all interviews were recorded, either manually or using a digital recorder. Interviews were then transcribed, and participants were provided with the opportunity to review and edit their responses.

The staff ratings for the PC-PIT elements from Part 1 were aggregated to a median practice score for each element and then compared with the ratings from the objective indicators in Part 2 and displayed in two side-by-side spider diagrams. These data were presented to each practice in individual PC-PIT reports, thus presenting a profile of practice performance against each of the 13 elements, using staff perceptions compared with documented practice evidence. Appendix 2 provides two de-identified examples of completed PC-PIT reports, including spider diagrams, as provided to a higher- and a lower-scoring practice. Practice managers used their PC-PIT reports to lead staff discussions about the identification of specific improvements to be made, strategies to achieve them, a time frame, measures of success and the staff member(s) responsible. This was then formalised using the Plan, Do, Study, Act (PDSA) approach.

Data analysis

Quantitative data were entered into a Microsoft Excel 2013 spreadsheet, then imported into SPSS version 21.0 (SPSS); data were analysed using SPSS and Excel. A key outcome measure was the degree of concordance between Rater 1 and Rater 2 during the independent practice visits, measured using concordance and κ statistics. Standard integer weights were used, as described by Fleiss.14

A total of 32 in-depth, semi-structured interviews were held (with 19 practice managers and 13 practice nurses). Transcribed interviews were coded by one member of the research team (L C) using NVivo 10 (QSR International). Codes were reviewed for duplication and clarity. We used thematic analysis to identify and classify recurrent patterns and themes. Interviews focused on aspects such as the background training and current role of the practice manager and practice nurse, QI experience of the individual interviewees and the most recent QI undertaken in the practice.

Results

A total of 45 general practices participated in Part 1 of the PC-PIT trial. At the time of writing, complete datasets were available for 34 of the 45 practices. These represented a range of geographic locations (urban, regional and rural areas), although most were urban and regional practices. It also included a range of practice sizes (< 2, 2–< 5, 5–< 10, 10&plus; full-time-equivalent general practitioners) and represented a range of business models (privately owned, partnerships and corporate business models). Ten of the 34 practices described both undertaking internal QI activities, such as PDSA cycles, and involvement in externally run QI activities, such as Medicare Local programs, National Prescribing Service activities and the Australian Primary Care Collaboratives. The remaining practices described either internal QI activities or involvement in external improvement programs. One practice was newly established and had not undertaken any improvement activities within the past 12 months. The practice managers came from a variety of backgrounds, including business management, nursing and allied health. Appendix 3 details the characteristics of the 34 participating practices.

Of these trial practices, 20 were selected for the independent practice visits and qualitative interviews in Part 2, and complete datasets were available for 19 practices. One practice was excluded due to competing commitments of the practice manager, which made it impossible for the independent practice visit to be conducted by the raters during the study timeframe.

Assessment of practice performance against the PC-PIT

A total of 310 online PC-PIT forms were completed in Part 1 by practice staff, comprising 19 practice managers, 95 GPs, 56 practice nurses, 109 administration and reception staff, 25 allied health (including pharmacy) staff and six “other” staff (primarily in business development, finance or information management roles). Using the combined online PC-PIT element ratings, the independent practice visit ratings and interviews with practice managers and practice nurses, three specific practice types were identified among the 19 practices from Part 2, each with a distinct way of using the PC-PIT. Rather than being discrete, these three types represented key points along a continuum of organisational performance, from lower-scoring to higher-scoring practices.

First practice type

The three lowest-scoring practices appeared to have separate and uncoordinated clinical and practice management processes. This was evidenced by uncoordinated clinical governance and organisational management activities and the incomplete translation of clinical and management processes into formalised policies and protocols that were clearly known and understood by all staff (both clinical and administrative).

Second practice type

A further three practices had a primary focus on clinical governance, with organisational management as a supporting basis. In this model, practice managers had limited or no autonomy in relation to organisational changes within the practice. This was illustrated by a lack of cited documented evidence (and therefore lower scores) on the PC-PIT element of organisational management, including key indicators such as evidence of staff role descriptions, performance appraisals, internal QI activities and the use of information such as data reports, formal meetings and discussion to improve the internal function of the practice. The practice manager generally worked in a supporting role to the GP(s), but there was limited evidence of communication and coordination between clinical and organisational management.

Third practice type

The five highest-scoring practices recognised the equal importance of organisational and clinical management in supporting the ongoing operation of the practice as a whole, demonstrated by high ratings in both the independent practice visit and staff online PC-PIT forms. Documented evidence of meeting minutes and previous PDSA processes and outcomes showed that management processes were constantly reviewed in a combined approach by clinical and administrative–management staff and readjusted to facilitate patient care. These practices demonstrated close communication and shared decision making in relation to continuous QI, championed by an autonomous practice manager who worked closely with a defined clinical leader. They were also more likely to have a history of involvement in a range of external continuous QI programs.

The remaining eight practices fell along the continuum, with most toward the lower-scoring end. These practices were generally characterised by positive staff perceptions of the 13 PC-PIT elements but a lack of documented supporting evidence, particularly on the use of practice data in making ongoing improvements to their organisational processes and in reviewing and using performance results. Box 2 provides examples of the three practice types, the median PC-PIT element scores given by the staff and the raters, illustrative interview quotes and the evidence cited during the independent evidence assessments.

Agreement between raters: independent practice visits

Overall, there was complete agreement between the two raters in 11 of the 19 general practices. Rater 1 scored higher for 11 PC-PIT elements and lower for one. The mean difference was 0.10. Box 3 presents the agreement between raters and the κ statistic for each element. The element with the lowest κ (0.43) was team-based care. For this element, the two raters agreed in 11 of the 19 practices; Rater 1 scored higher than Rater 2 in seven practices and lower in one. If we excluded this element from the overall κ coefficient, the χ² test for homogeneity was 14.66 (P = 0.20).

To identify reasons for key discrepancies by practice and by element, the raters reviewed their evidence-based assessment forms and discussed possible reasons for the discrepancies. The discrepancy in Practice 10 was due to circumstances that required the raters to interview different informants and cite separate documentation in relation to the PC-PIT elements. Poor concordance between the ratings for the element of team-based care reflected a lack of formally documented policies and procedures available to practice managers, while additional undocumented information could be provided by practice nurses. Rater 1 scored this verbal (but undocumented) information, while Rater 2 did not (Box 3).

In terms of practices, Rater 1 scored more elements higher than Rater 2 in six practices, especially for Practice 10, where there was agreement on only one element (Appendix 4).

Resource and support needs of practice managers

During the in-depth interviews, four key themes were identified in relation to practice managers’ perceived resource and support needs (Appendix 5). Most practice managers were not familiar with internal organisational development tools other than those previously developed by the former Divisions of General Practice. Most of these tools were neither trialled nor validated in general practice settings. Only one practice manager described the use of a formalised approach to organisational development (Six Sigma), which was recently adapted for use in general practice but required extensive external facilitation to complete. Practice managers perceived the benefits of having additional supporting tools relating to elements in the PC-PIT, and also identified strategies such as online forums or email updates, based on the PC-PIT elements of high-performing practices, which might focus on sharing key problems and solutions for organisational performance improvement.

Discussion

This national trial of the PC-PIT determined that it can be a useful organisational performance tool in various general practice settings.

As health care delivery becomes more complex and technology-driven, the organisational context in which QI initiatives take place becomes increasingly recognised as a crucial determinant of their effectiveness.16,17 Contextual elements have been described as the “adaptive reserves” of a practice; that is, those features that represent a practice’s internal capability.18 They include features such as culture, leadership, collaboration and teamwork, data and information tools, improvement skills, incentives and time allocation, which general practices should address to support a context of continuing QI.19

The establishment of the PHNs and the release of the consultation paper for the review of the Performance and Accountability Framework indicators20 illustrate the integration of aspects of QI across the health reform strategy. Although as yet incomplete, the national Performance and Accountability Framework objectives relating to quality focus on outputs related to safety, responsiveness (based on measures of patient experience), capability and capacity.20,21 Following this, the proposed national PHN evaluation framework lists continuous QI activities, outputs and outcomes related to provision of practice support and the identification of high-priority practices (those practices requiring targeted support to build their capacity to engage in QI), such as accreditation and the use of data for practice improvement.

The need to develop and strengthen managers’ skills also involves the development of management processes for motivation, supervision control and action, and support at an organisational level.22 Practice managers may be responsible for large and fluctuating numbers of staff, high yearly financial turnovers and the ongoing facilitation and management of change; many do so with limited timely access to appropriate ongoing training and validated support resources. While there is an undeniable need to focus on the role of GPs in QI, it is also worth noting that the elements relating to organisational improvement are also the domain of practice of operational managers.

Although the independent practice visit was conducted by two external raters in this trial, we anticipate that this aspect will become part of the PC-PIT as a wholly internal assessment process. However, future testing of the PC-PIT will seek to further address the discrepancies relating to the involvement of different practice staff members (ie, practice managers v practice nurses) in the use of the evidence-based assessment forms. The objective indicators for the element of team-based care have also been refined and clarified to include additional meeting minutes and documentation accessible to either practice managers or practice nurses, to ensure that all available evidence is taken into account during the assessment.

Two of our partner organisations, the Royal Australian College of General Practitioners and the PHNs, identified a key benefit of the PC-PIT as the ability to identify the lower-scoring practices and more effectively engage them in organisational improvement activities, allowing for more targeted interventions that are relevant to the capacities of the individual practices. Thus, there are two aspects to the future sustainability of the PC-PIT: (i) embedding the PC-PIT approach into existing QI frameworks, and (ii) further research into the role of the PC-PIT in supporting performance improvement in primary health care. The PC-PIT approach will be further developed to include a suite of high-quality, validated and free-to-access resources that complement the use of the tool.23,24

Limitations

Although this trial was conducted with volunteer practices, every effort was made to ensure a range of geographic locations and practice sizes were incorporated in the inter-rater comparison. However, there was an over-representation of urban and regional practices. Many rural practices were unable to commit due to perceptions of the time required to complete improvement activities. However, this is an area that may be of interest to the newly established PHNs, given their formal role as facilitators and supporters of practice engagement in QI. We also acknowledge the lack of consumer involvement during the trial phase of the PC-PIT. Further work to refine and embed the PC-PIT in existing QI programs will seek to involve the Consumer Health Forum of Australia as a key partner in the process, with emphasis on the role of consumer feedback as an embedded feature of external validation.

In relation to the inter-rater comparison, the calculation of the aggregate value of κ over the 13 PC-PIT elements assumes that the κ values are independent, which is unlikely. The lack of independence, however, is unlikely to affect the aggregate value but might increase the standard error to a small degree.

Conclusion

With the refocus on the importance of organisational aspects of practice in relation to quality care delivery, the time is now right to focus on a standardised, internally led approach to improving practice performance, designed for the dynamic context of primary care. Work with our key partners and end users is ongoing, with the aim of further trialling and embedding the PC-PIT within existing QI initiatives.

There is growing awareness of the need to improve quality in health care, including in primary care.1–4 In Australia, this is witnessed by the National Primary Health Care Strategy, with its focus on the importance of quality as a foundation and driver of change.2 There is also an ongoing push for the development of new indicators for performance improvement, quality and safety benchmarking, and change management approaches and strategies for quality improvement (QI) in primary care.2

There are diverse terms and definitions used for QI,5 and varying QI strategies involving structured processes that include assessment, refinement, evaluation and adoption of processes used by individuals, teams, an organisation or a health system, with the aim to enhance some aspect of quality and achieve measurable improvements.6,7 These can include simple tools, such as flow charts and checklists; more complex multiple-method tools, such as re-engineering; and frameworks, such as the Plan, Do, Study, Act (PDSA) and audit cycles.8 These strategies have yielded modest change and are often not sustained over time.8,9 There is increasing evidence that QI initiatives that are locally owned and delivered, team-focused, formative and flexible and involve interorganisational collaboration and networking are more sustainable and yield better outcomes.10,11 The primary care practice team has a responsibility for QI as part of clinical and organisational governance, and team members are encouraged to collaboratively engage in QI activities in areas that will improve the safety or quality of patient health care.12–16 Primary care practices that embrace a QI culture and support QI initiatives are likely to have better health outcomes, better care delivery and better professional development.1,7,17,18

There is currently no single tool available to Australian general practices that combines traditional areas of clinical governance and less widely used aspects of organisational performance.19 In response, the Primary Care Practice Improvement Tool (PC-PIT)11 was co-created by a range of stakeholders using various engagement platforms, including ongoing cyclical feedback from partners and end users. The result is an organisational performance tool tailored to Australian primary care.11 The PC-PIT includes seven key elements integral to high-quality practice performance: patient-centred and community-focused care; leadership; governance; communication; change management; a culture of performance; and information and information technology. Results from the pilot study and the trial of the PC-PIT indicate that this tool offers an appropriate and acceptable approach to internal QI in general practice.11,20 The findings also showed that additional QI tools and resources are necessary to support the seven elements in the PC-PIT.20 Therefore, we aimed to undertake a systematic review of the international published and grey literature to identify existing primary care QI tools and resources that could support organisational improvement related to the seven elements in the PC-PIT. The identified tools and resources were then included in the next phase of study, which used a Delphi approach to assess the relevance and utility of these tools and resources for use in Australian primary care and to complement the PC-PIT.21

Methods

We undertook a systematic review of published and grey literature to identify existing online QI tools and resources to be included in a Delphi study assessing their relevance and utility in Australian general practice.21

Search strategy

In March 2014, we searched the electronic databases CINAHL, Embase and PubMed for articles published between January 2004 and December 2013, using the search strategy outlined in Table 1 of Appendix 1. We searched for articles where the search terms appeared in the title, abstract or subject headings, and limited results to those published in the English language. All searches were designed and conducted in collaboration with an experienced search librarian. We imposed no restrictions on the type or method of QI tool or resource and included any simple tools, multiple-method tools or frameworks that can be used by an individual in the practice, teams in the practice or the whole organisation to improve any aspect of organisational quality related to any of the seven elements in the PC-PIT.

In March–April 2014, we also conducted a comprehensive search of grey literature for documents dated between 1992 and 2012.22,23 This included an iterative manual search of the electronic database GreyNet International (http://www.greynet.org) and relevant government and non-government websites (Appendix 2). We consulted experts in primary care and QI to ensure key electronic databases, organisation websites and online repositories were included in the search. Searches were also conducted using Google Advanced Search (http://www.google.com/advanced_search) and repositories such as OpenGrey (http://www.opengrey.eu), WorldCat (http://www.worldcat.org) and OpenDOAR (http://www.opendoar.org).

For all relevant tools and resources identified through the grey literature search, we also searched in the research databases CINAHL, Embase and PubMed, as well as Google Scholar, for evidence of their use in practice. Search terms used in the grey literature search are shown in Table 2 of Appendix 1.

Finally, we reviewed the bibliographies of all identified relevant studies, reports, websites, databases, tools and resources to identify any additional QI tools and resources for inclusion. All additional tools and resources identified through this snowballing process underwent the screening and assessment process.

Selection of studies, tools and resources

All citations were imported into a bibliographic database (EndNote, version X7). To be included in the review, identified citations, tools and resources had to meet the following eligibility criteria: (1) purpose of the tool or resource is QI; (2) tool or resource is used in the primary care setting or has potential for use in primary care; (3) tool or resource addresses at least one of the seven elements integral to high-quality primary care practice; and (4) tool or resource is available and in the English language.

The initial screening process involved two reviewers (S U and T J) screening the titles and abstracts of published citations and any articles, reports, tools or resources identified through the grey literature, and categorising them as “relevant” or “not relevant” according to the review objective. The full texts of all tools and resources deemed relevant were sought and reviewed by two independent reviewers with expertise in primary care QI (S U and T J) to further assess their relevance according to the eligibility criteria.

There is no single well established assessment or scoring instrument suited for QI tools and resources that covers the broad range of tools and resources included in this review. Therefore, we developed a four-criteria appraisal framework from common sets of criteria proposed for assessing a range of QI tools, resources and initiatives, such as guidelines, instruments, programs and web-based resources (Box 1).24–30 All identified tools and resources that met the eligibility criteria were evaluated for their accessibility (ie, able to be accessed online and at no cost), relevance, utility and comprehensiveness using this four-criteria appraisal framework. Two reviewers (S U and L C) independently gave each tool or resource a score out of 8 using the criteria. Tools or resources with a score of 7–8 were rated as the “best” and passed on to the Delphi study21 for further assessment. Tools and resources rated less than 7 were rejected and not included in further assessment. The reviewers compared their ratings, and any discrepancies were resolved through discussion.

Data extraction and synthesis

We created a data extraction template using Microsoft Excel to assist in systematically extracting information about the tools and resources that met the eligibility criteria. A content analysis approach was used to explore each tool or resource to collate the following information: name of the tool or resource, year and country of development, author, name of the organisation that provided access to the tool or resource and its URL, accessibility information or problems, a brief overview of each tool or resource, the QI element(s) it addresses and any supporting evidence (published or unpublished data). If accessible, a copy of the tool or resource was downloaded into the bibliographic database. Any supporting evidence (studies, reports and any other data) on the use of the tool or resource in primary care was also added to the bibliographic database.

Results

The database search yielded 1900 citations after duplicate records were removed (Box 2). After reviewing the titles and abstracts for relevance to the review objective, the total was reduced to 249 articles. Of these, 140 did not meet eligibility criteria and were excluded, leaving 109 articles. Most excluded citations did not meet the eligibility criteria because the tools or resources were not used in primary care settings. From the 109 citations, 76 QI tools or resources were identified (Appendix 3).

The level of empirical evidence for each tool or resource varied substantially — some, such as the PDSA, had numerous studies supporting their use in primary care,31–35 whereas others, such as the Organisational Capability Questionnaire,36 had only been taken to pilot stage. Of the 76 tools and resources identified in the published literature, 37 were rejected because of accessibility problems. Of the remaining 39 tools and resources, 19 scored less than 7 on the four-criteria appraisal and were rejected due to problems related to utility (n = 10), relevance (n = 3) and comprehensiveness (n = 6). This left 20 that were classified as the best tools and resources (Appendix 4).

Through the grey literature search, we identified 186 tools or resources that met the eligibility criteria (Appendix 5). Of these, 12 were rejected because of accessibility problems. A further ten tools or resources were duplicates and also excluded. Of the remaining 164 tools and resources, 131 scored less than 7 on the four-criteria appraisal and were rejected due to problems related to comprehensiveness (n = 99), utility (n = 16) and relevance (n = 16). This left 33 tools or resources identified as the best from the grey literature (Appendix 4).

Of the total 53 best tools and resources identified through published and grey literature, 13 were from Australia and the remainder were from the United Kingdom (n = 14), United States (n = 14), Canada (n = 4), New Zealand (n = 4) and Europe (n = 4). There was significant overlap of the PC-PIT elements covered by the best tools and resources, with most tools relevant to two or more elements integral to high-performing practices. Of the 53 identified tools and resources, 34 predominantly addressed performance, 20 governance, 19 patient-centred care, 15 change management, nine leadership, nine communication, and six information and information technology (Appendix 4).

Discussion

In an effort to strengthen primary care practices, and thereby strengthen the broader health care system, many providers, delivery systems and other organisations are supporting the use of QI initiatives to improve the performance of practices.37 There are currently no published data regarding the available QI tools and resources for Australian primary care. In this review, we identified and synthesised existing primary care QI tools and resources from the international published and grey literature that are relevant to the seven elements integral to high-quality primary care practice,19 which are specifically covered by the PC-PIT.11 Our findings provide data on QI tools and resources that can be used to support QI initiatives in primary care, including complementing and optimising the value of the PC-PIT.

Given the complexity of health care, developing, implementing and assessing QI initiatives is a dynamic, evolving and challenging area.38 This review illustrates the wide range of primary care QI tools and resources that are available. There is substantial variability in the accessibility, comprehensiveness and utility of tools and resources for primary care, as well as the evidence for their use. Many tools and resources require extensive (and often costly) external facilitation, which adds further complexity and limitations to their application in general practice settings.

Variability in evidence

There is a gap in the published literature on QI tools and resources in primary care settings, and the available literature is of varying quality.39,40 This is partly due to the complexities involved in reviewing a heterogeneous set of interventions that are applied in a varying set of contexts.41 This lack of scientific literature has somewhat inhibited the acceptance of QI methods in health care.38 With new approaches, tools and resources being introduced at a rapid pace and disseminated through the World Wide Web, there is some debate about the most effective QI tools and resources for use in the health care setting.7 Although new studies are emerging,38 there is a need for more rigorous evaluations of different QI tools and resources in primary care settings.39,42

Comprehensiveness of tools and resources

There are many approaches and strategies that can be used to improve the quality of primary care practices. These improvement strategies are generally divided into two types: improvement focusing on clinical areas and improvement focusing on quality from a management perspective.6 Although the two may share common themes, they are often seen as discrete parallel activities. For example, the NPS MedicineWise Clinical e-Audits are used to facilitate clinical QI by assisting GPs to review their prescribing practices,43 while the Advanced Access and Efficiency Workbook for Primary Care focuses on improving the organisational quality of the practice to enable patients to see their doctor when they need to.44 Some tools and resources, such as Lean,45 Six Sigma46 and the Manchester Patient Safety Framework,47 are based on theoretical frameworks, whereas others, such as the Canning Data Extraction Tools48 or the eCHAT (electronic case-finding and help assessment tool), are more pragmatic.49 Some tools and resources, such as the PDSA, Six Sigma and Significant Event Analysis, are well known.6,50 Other less well recognised tools and resources range from the simple, such as the Organisational Capability Questionnaire,36 to the more comprehensive, incorporating a range of other supporting tools, such as the UK’s National Health Service (NHS) clinical engagement resources51 and the NHS Scotland Quality Improvement Hub.52

Due to the complexity of primary care practice and the dynamic process of QI, several QI tools and resources could be used in conjunction with each other, or one after another, to yield successful outcomes; for instance, beginning with root-cause analysis, then using either Six Sigma or PDSA to implement a change in processes.38 Another example is the use of tools and resources for improving chronic illness care, such as using the Primary Care Resources and Supports for Chronic Disease Self Management53 in conjunction with the Assessment of Chronic Illness Care,54 with the former focused on self-management support and the latter on improved patient and staff competency in self-management processes.

Accessibility and utility of tools and resources

It can be challenging to engage practices in QI initiatives because primary care clinicians and staff often feel intense time pressures; have competing priorities; lack a culture and leadership that support change; lack resources, capability and capacity; and may fear the perceived costs of undertaking QI.7,17,18,55 Therefore, ease of access and utility are important factors in optimising the acceptance and adoption of QI initiatives in primary care practices.7,18 In line with the literature, the main reasons tools and resources were rejected in this review were that they rated poorly with regard to their comprehensiveness (42%), accessibility (19%), utility (10%) (ie, too complicated, contained difficult language, too time-consuming or required extensive facilitation) and relevance to primary care (8%).

QI efforts need to be substantially more efficient and easy to access and must reduce the burden on practices to maximise their adoption in primary care settings.17 Recognising this need, some health care organisations provide comprehensive online libraries of quality and service improvement tools and resources that are readily accessible and free of charge.56–58 Nonetheless, it is often difficult for busy practitioners to navigate through multiple websites to obtain the right tools or resources for QI. Therefore, a better option would be a suite of QI tools and resources that is embedded into existing quality frameworks.

Support and incentives for quality improvement

Practices need to be supported and incentivised to adopt a QI culture and engage in continuous QI initiatives.7,18 Even the most determined practices are likely to require help in developing their QI capacity, such as skills to identify areas for improvement, knowledge and understanding of QI approaches, how to use data for QI, planning and making changes, and tracking performance over time.37 This demands the commitment of practice leadership and staff to dedicate time and resources to QI activities.37,38 Practices will also require external support, such as technical assistance, learning activities and tools and resources provided by organisations to assist practices undertaking QI initiatives.37

Public and private health care sectors around the world are now linking service quality with provider payment. Both the UK and the US provide financial incentives to some health care providers for adopting improved quality practices. Using a “pay for performance” system can drive and support practices to adopt QI initiatives to improve the quality of their practice and patient outcomes.59 In Australia, the Primary Health Care Advisory Group recently considered new payment mechanisms to better support the primary care system to drive safe and high-quality care.60

Limitations

Our review has several limitations. First, the exclusion of non-English-language literature may have omitted some relevant tools and resources. However, non-English tools and resources could not have been used in Australian primary care without being translated, which was not feasible within the scope of the study. Second, QI initiatives (including tools and resources) are poorly indexed in bibliographic databases.39 As such, we employed broad search strategies that used free text and Medical Subject Headings (MeSH) to optimise our search strategy. While we also included grey literature to capture tools and resources, an exhaustive search was not undertaken due to time constraints. Other studies have reported similar challenges.61,62 In response to this, we consulted with experts in the area to ensure that the key relevant electronic databases, organisation websites and online repositories were not missed in the search. Finally, the four-criteria appraisal framework and the method of rating the tools and resources was subjective and potentially biased, and we did not perform a sensitivity analysis against the robustness of the assumptions. Hence, caution is required when interpreting the classification and rating of each tool or resource. To address these limitations and increase reliability, the two reviewers who assigned the ratings discussed, checked and agreed on the scoring.

Conclusions

The necessity for QI initiatives permeates health care37,59,63 and presents opportunities to fundamentally improve health in Australia. Engaging primary care practices in QI and practice redesign activities allows them to work toward achieving improved quality, better health and improved patient and provider experiences, as well as reducing the ongoing costs of care.37,64 To ensure these efforts have a positive impact, there is a need to build and sustain the ability of primary care practices to engage in QI initiatives in a continuous and effective way. To foster QI capacity in Australian health care, we have identified tools and resources that can potentially be provided as part of a suite of tools and resources to support primary care practices in improving the quality of their practice, to achieve improved health outcomes. Following this review, a Delphi study was undertaken to evaluate the 53 best tools and resources to assess their relevance and utility in Australian general practice; the results are published elsewhere in this supplement.21

Box 2 –
Flow diagram outlining selection process for tools and resources