A goal of primary care research is to inform and facilitate beneficial change in health care delivery for both patients and clinicians in a way that is efficient for providers and economically sustainable for the health care system. Co-design or co-creation is a process whereby researchers and stakeholders jointly contribute to the ideation, planning, implementation and evaluation of new services and systems as a possible means to optimise the impact of research findings.1,2 Co-creation represents the highest form of stakeholder engagement, building on existing theories such as community-based participatory research,3 and emphasises the creation of value for both end users and researchers.

However, it is often infeasible to co-create with all stakeholders at all stages of a research project. Decisions must be made about which stakeholders to involve at different stages to achieve the greatest return on investment for researchers’ and stakeholders’ time and contributions, considering the context in which change is likely to occur.

In designing a controlled pilot study of a multifaceted intervention to reduce inappropriate polypharmacy in primary care involving 20 general practitioners and more than 150 patients in south-east Queensland, we identified both patients and prescribers as the most important stakeholders. For this project, however, we proposed that the critical gateway to effective co-creation in the first instance was the GP, with both the GP and patient to be involved in evaluating the pilot to inform future development. In this article, we describe the rationale for this approach, the process, challenge and value of co-creating with GPs in the planning and implementation of the intervention, and the anticipated value of involving both patients and GPs in the pilot’s evaluation.

Background

The impetus for this co-creation research project was Australian data suggesting high rates of potentially inappropriate medication use among older Australians, and its association with significant patient harm.4 The aim of the project was to design and pilot a multifaceted deprescribing intervention to minimise inappropriate polypharmacy in older people in primary care. Deprescribing is the systematic process of identifying and discontinuing the use of medicines where the actual or potential harms outweigh the benefits, giving due consideration to an individual patient’s care goals, current level of functioning, life expectancy, values and preferences.5 Box 1 shows an overview of the decision algorithm for use by the GPs in this pilot study. Deprescribing is an inherent principle of quality use of medicines and is part of good prescribing.7 The study presented here represents early-phase piloting and developmental work, as described by the United Kingdom Medical Research Council’s guidance for complex interventions.8

Rationale for co-creation with GPs

This deprescribing pilot study, as a quality prescribing initiative, aimed to effect change at the microsystem level (ie, the provider–patient interaction). Relevant stakeholders for co-design could have included patients (and family or carers if relevant), GPs, medical specialists, hospital providers, community pharmacists and other members of the primary health care team.

There were several reasons for prioritising GPs for co-design in this pilot. Recent literature exploring patients’ perspectives on deprescribing indicated that their GPs can be highly influential in encouraging patients to cease taking inappropriate medicines,9 confirming that GPs exert considerable influence on what is discussed during consultation time with patients.10 Deprescribing is a highly nuanced, individualised process that requires comprehensive, holistic review and follow-up of a patient. Accordingly, the GPs’ often long term and trusted relationship with, and tacit knowledge of, their patients place them in an ideal position to engage patients in the deprescribing process and participate in decisions about the continuation or discontinuation of long term therapy.11 Giving due consideration to the organisational structure and operation of primary care, and the asymmetry of medical knowledge between patients and GPs, the GP was considered to be best positioned to raise the issue of deprescribing, and collaborate with patients and other highly influential stakeholders such as specialists and health professionals, to facilitate the process as appropriate. We were also aware of research identifying patient attitudes and barriers to deprescribing initiatives, which we used to inform our study design.9,12 For all these reasons, we decided to pursue co-design with GPs in the first instance. Although co-design would not be undertaken with the patients at the front end, the pilot evaluation would include interviews with both patients and GPs aimed at ascertaining the impact and acceptability of the deprescribing process.

The challenge and reward of research co-creation in primary care

GPs are a busy and heterogeneous group of health professionals, whose practice settings vary markedly in terms of administrative and clinical support, resourcing and infrastructure, capacity and readiness for change.13 Engagement of GPs in co-creation research must occur at times and places that are optimally convenient for them, and must recognise the immense variability among GPs and their work contexts. The following describes examples of how GPs were involved in the planning, implementation and evaluation of the project in ways that recognise their busyness and diversity.

Planning

A collective approach to co-creating new systems or services is often preceded by information gathering and conceptual ideas.1 We undertook a systematic review of the literature of prescribers’ barriers and enablers to minimising the use of potentially inappropriate medications and used its findings to inform the first stage of co-creation.14 We also undertook focus group discussions with GPs from five large practices in the designated pilot study catchment area with two objectives: (i) to identify local barriers and enablers to deprescribing, as local context is highly influential in shaping clinician behaviour,8 and compare these with the findings of the review; and (ii) to determine the perceived applicability and utility of a purposively designed deprescribing framework in routine care, modified from one shown to be an effective tool for deprescribing in hospitals.15

The findings of the focus group discussions and the literature review together with liaison with a senior academic GP adviser experienced in the care of older multimorbid patients, were critical in understanding the context in which the intervention would be applied and consequently, in formulating its key components, including:

• conducting an interactive training workshop for GPs on deprescribing, based on the deprescribing framework;

• identifying patients at high risk of medication misadventure;

• scheduling extended deprescribing appointments with GPs for their high risk patients (with follow-up and referral at the GP’s discretion); and

• providing the option of referring patients to an accredited pharmacist trained in deprescribing to review a patient’s medicines to help GPs overcome some of their barriers to deprescribing, such as limited time.

GPs, with their patients, would be free to collaborate with or involve other health professionals throughout the deprescribing process, in recognition of the heterogeneity of GP work environments, care teams and with respect for the GPs’ professional autonomy.

Implementation

The next step was working with GPs recruited into the pilot study to co-design practical tools that would help meet study objectives.1 The following describes two examples, wherein a flexible, iterative research design was used that could respond to the needs of different general practices.

Example 1: Identifying patients at high risk of medication misadventure

The research team worked closely with the principals (or their delegates) of the practices recruited to the pilot to develop a standardised but customisable patient management software query that would help identify patients at high risk according to evidence-based criteria. This involved the use of the same medical software (Best Practice Software) at all but one site. A comparable query was performed using an external reporting software tool at the site that used different software. To account for variations in data quality across the sites, the software query was combined with a documented manual screening process so that each GP could generate a sample of consecutive eligible patients for study recruitment. The co-design of the software query minimised the extent of manual screening required by GPs and practice staff, but maintained a degree of standardisation of non-negotiable search criteria important to the research team across all of the sites.

Example 2: Co-designing a multipurpose tool for use during the deprescribing appointment

Focus group discussions with GPs emphasised that any tool for deprescribing should ideally be integrated into the medical software used in general practice. The research team aimed to minimise the burden of documentation for GPs as part of the project.

A tool or template, easily imported into the electronic consult notes, was developed and served three functions: collecting data for the research team; acting as a memory prompt for the GPs of the key steps of deprescribing; and providing a framework for documenting the deprescribing consultation. The draft tool was designed by the team, with the aid of a practising GP with research and software experience, after considering GPs’ work processes and the required sequential steps of the deprescribing framework.

This template (Box 2) was subsequently trialled in the medical software used by participating GPs who attended the preparatory deprescribing workshop. This resulted in additional decision support being added to the tool on the basis of suggestions made by GPs. All participants recognised that having a multipurpose tool integrated into the medical software increased the chance of it being used in routine care and capturing all information critical to project evaluation.

Evaluation

The future evaluation of the deprescribing project will involve a mixed-methods approach. The primary outcome measure is a change in the total number of regular medications taken by older people at high risk of medication misadventure. Secondary measures will include change in the number of medications taken as needed (prn), the number of medications to have doses changed, drug classes commonly deprescribed and process measures such as the proportion of patients attending their deprescribing appointment. A formal process for ascertaining and recording any instance of actual or potential patient harm arising from deprescribing has been implemented. Changes in patient attitudes towards deprescribing and quality of life will be assessed by pre- and post-questionnaire surveys. Interviews with GPs and patients will explore the acceptability and impact of the deprescribing process. The results of this pilot study will help determine if there is sufficient justification for seeking funding for a larger-scale trial that evaluates the deprescribing intervention used, or an amended version of it, across a wider spectrum of general practice settings.

Conclusion

Utilising a co-creation approach in research is both challenging and rewarding. The exact approach that is taken needs to be tailored according to the study purpose, views of key participants and context of change. The level of engagement between researchers and end users will vary at different stages of planning and implementation to ensure maximum value for both parties.

The Australian Government’s Partners in Recovery (PIR) program1 established a new form of mental health intervention aimed to better support people with severe and persistent mental illness with complex needs, and their carers and families. It aims to achieve this by getting multiple sectors, services and consumers to work in a more collaborative, coordinated and integrated way.1 Commissioning these services required an approach that engaged many stakeholders to generate a model that was widely supported and understood. Value co-creation offers a framework to describe this style of commissioning and has been applied to mental health commissioning internationally.2–6

Co-creation entails a new vision of value creation through a shift in thinking about the co-creators of value, the value networks, and the entire value of ecosystems.7,8 It involves redefining the way an organisation engages with individuals, partners and stakeholders by bringing them into a process of value creation and engaging them in enriched experiences throughout the journey, in order to design new products and services, transform management systems, and increase innovation, productivity and returns on investment.7–11 Co-creation requires focus and sustained efforts in making choices of where and how to co-create value with stakeholders and end users.9,10 Elsewhere in this Supplement, Janamian and colleagues12 describe value co-creation approaches and strategies to achieve value co-creation in primary care services research.

The aim of this article is to illustrate how the Brisbane North Primary Health Network (PHN) applied value co-creation approaches to the PIR program to co-design a solution with the partners and end users. Here, we focus on the co-creation processes; formal evaluation outcome data will be provided in detail elsewhere in future reports and publications. Only through the direct engagement and co-creation with stakeholders could the government’s program guidelines be translated into a real service that functioned effectively. The PIR program was established in two phases. Phase one is the collaborative work undertaken in the development of the funding submission in 2012–13; and phase two is program delivery from 1 July 2013 to the present.

Mental health commissioning in Brisbane North

Identifying and engaging stakeholders

Stakeholders encompassed not just providers of specialist mental health services, but also primary health care providers, emergency services, social services and consumers of these services and their families/carers. Involvement of stakeholders across multiple sectors and organisations produced new capacities and widened the scope and scale of interactions and experiences. This expansion of value creation in a “win more, win more” fashion leads to more transformational results as the scope of application expands — meaning that more and more people gain some impact or benefit from the co-created processes and outcomes.7–9 Brisbane North PHN (the PHN) engaged more than 100 organisations in the development phase, either by direct invitation or by public advertisement. The service model was developed through a series of three workshops in the last half of 2012. An additional workshop exclusively for consumers of mental health services and their families/carers identified the key outcomes that the model should deliver. A meeting of Aboriginal and Torres Strait Islander agencies was also convened.

During this phase, stakeholders reported satisfaction with the open and participative process, and they strongly embraced the opportunity to feed into development of the model. The outcomes of each workshop were documented and fed back to all participants, in an iterative process that co-created the service model. Champions for the PIR program and the PHN model emerged from these workshops and, through the process, 22 organisations expressed interest in forming a working collaborative. The PHN ultimately selected seven mental health specialist providers, the local hospital network (Metro North Hospital and Health Service), the Queensland Alliance for Mental Health (the state’s peak body for community mental health), and consumer and carer representatives to form the Consortium Management Committee (CMC). While the service model was co-created with all stakeholders’ participation, the budget was finalised by the CMC.

Co-creation experiences and platforms of engagement

A range of engagement platforms were established in 2013, each providing stakeholders opportunities for new co-creation experiences and outcomes of value.7–9 A key premise of co-creation is that by sharing experiences, the individuals involved will gain a greater understanding of what is happening on the other side of every interaction, enabling them to devise a new, better experience for both sides.10 Agency chief executive officers and senior managers meet with consumer and carer representatives as the CMC every 6 weeks. Service managers within agencies also meet 6-weekly and direct delivery staff meet in learning sets every month. A client information management software platform is used by all agencies, providing staff across eight separate agencies direct access to client information. An analysis tool enables real-time interrogation of both outcome and process data, and program-wide reports. This has facilitated the ongoing co-creation of quality improvements and the potential for integration with clinical data from primary care or public mental health services.

PIR regularly produces short videos, circulated on YouTube and other social media platforms, which update stakeholders on progress and showcase new initiatives. Information is available in a widely distributed electronic quarterly newsletter and through an interactive website (http://www.northbrisbane.pirinitiative.com.au). A separate website has been co-created by public and private health providers as a system navigation tool updated directly by providers (http://www.mymentalhealth.org.au). Additionally, an annual forum brings together the wide and diverse range of stakeholders initially involved in the program’s development phase. This provides both accountability for those delivering, and opportunities for prioritisation of new initiatives.

Importance of stakeholder value co-creation

Co-creation expands value creation for stakeholders in various ways: value as enacted through co-creative interactions; value as exemplified experiences; and value as emerging from diverse collaborations across multiple sectors and building on stakeholder’s existing capabilities.7 The PHN had limited expertise in working with people with severe mental illness, and an effective service model could not have been created without the genuine engagement of key stakeholders — community agencies, primary health care providers, public mental health services, consumers and carers. Through equal partnership, mutual relationships and sharing of the decision-making power, value was co-created jointly and reciprocally by all stakeholding individuals. A key value of the program was system improvement and, without contribution from the broad range of players operating in the local system, this goal would not have been achieved. In essence, the co-creation development process was part of achieving the co-created outcome.

During the delivery phase, PIR was managed by the CMC. The PHN adopted principles of collective impact13 and positioned itself as simply one member agency with the “backbone” responsibility for managing the formal partnerships, performance data and reporting accountabilities. PHN staff focused on creating a CMC environment that was high in trust where the PHN operated as a facilitator rather than a funder.

The high levels of trust and cooperation among members was evidenced in their response to performance data. For example, the data showed 3% of PIR clients in the region were of Aboriginal and Torres Strait Islander background, which was in line with overall population prevalence. However, given the over-representation of Aboriginal and Torres Strait Islander people in the target population, it was clear that PIR was not sufficiently accessible. Following discussion, the CMC agreed that an Aboriginal and Torres Strait Islander provider organisation needed to join the collaborative. The seven provider agencies agreed to give up a portion of their existing contracted funding and pool these resources to enable the PHN to formally partner with the Institute for Urban Indigenous Health, and for this agency to join the CMC. Subsequently, the proportion of Indigenous PIR clients increased. In this way, members of the CMC demonstrated that they were committed to making changes based on performance data, in an environment high in trust.

Co-creation partnership motivations

Dialogue, access, risk–benefits and transparency form the building blocks of value co-creation and require diligent application.14 The collective management of the program included data sharing and client consent across all agencies. This meant that providers could view client notes from other agencies, and all agencies could access client outcome data. As a result, in addition to the CMC meetings that were attended predominantly by executives, service managers from all agencies met to discuss differences in performance and approach. This has resulted in much stronger connections between agencies at multiple levels, and increased quality of service provision. The delivery system is more self-monitoring, with a convergence of practice, meaning consumers, referrers and other providers have a consistent experience of the program.

Stakeholder value co-creation opportunities

A deeper collaboration with stakeholders across multiple sectors increases the pool of resources, competencies and capabilities, accelerating value creation opportunities for all.7–9 Agencies collaborated on system improvement activities across a range of projects. Work on the primary–secondary health care interface was undertaken with public, private and community providers working together to create practical solutions in order to smooth the patient journey across these boundaries. More inter-sectoral work has occurred, and an advisory group including disability services, police and emergency services, and housing and homelessness agencies, was formed to better integrate responses. The CMC oversaw a program to incentivise innovative and collaborative activities, which resulted in work with community pharmacies, education of employers through the Queensland Chamber of Commerce and Industry and the production of new stepped-care models of housing and support. While some of this work has involved new initiatives, much is focused on changing existing interventions and referral practices as understanding and awareness of consumers’ needs increased.

Reported benefits of value co-creation

To evaluate the impact of the program, we recruited and trained a team of consumer evaluators who collectively designed the client data collection tool, approached clients directly (and not through their service provider) for interviews, facilitated qualitative workshops, and analysed the data collected.

Although formal results will be reported in future reports, to give a glimpse of the possible impact of the co-creation, early reports include that about 90% of the more than 1500 clients in the program reported experiencing a reduction in unmet need, and about 85% no longer reported problems with connecting to relevant services. Approximately four in ten clients had previously had no contact with the public mental health system and around one in ten PIR clients were of Aboriginal or Torres Strait Islander background. In parallel, our survey of providers found that PIR was thought by some to have contributed to improved coordination between clinical and community mental health providers.

The development of environments which are high in trust also has significant efficiency outcomes. Co-creation and collaborative management means organisations deliver according to their strengths while ensuring program consistency from a consumer perspective. Agencies were also prepared and willing to give up resources to alternative approaches if the evidence demonstrated their value.

Challenges using value co-creation

The value co-creation approach requires a significant commitment of time — not just by PHN staff but also many staff in CMC member agencies, consumers and broader stakeholders. At least initially, it would appear that simply delivering a program independently within an agency is less time-intensive. Formal evaluations are yet to be completed and reported; however, our experiences over 3 years of delivery suggest that investment in strong connections at multiple levels and through many organisations produces better quality for the client and ultimately saves time through a more integrated and collectively planned approach.

The major challenge is giving up long-held models of care that may work for individual agencies but are not effective from a systems perspective. This remains a work in progress, particularly for agencies that are large and complex and for professions that have been delivering care in a particular way for many decades.

Conclusion

The challenges of consistently delivering a program by working through eight separate agencies are considerable. The style and approach of the PHN as backbone to this initiative has a significant impact on the processes and outcomes. The use of value co-creation and collective impact has produced better outcomes for mental health consumers and their families, and has ensured that resources have been applied efficiently to create lasting system improvements.

Internationally, there has been a growing focus on the governance and performance of health systems and health professionals, and a redefining of the role of the health consumer. Crises relating to increasing aged populations, the increasing burden of chronic and complex disease, the continuing challenges of quality and safety, and escalating health care costs have resulted in pressure to achieve a more productive and sustainable health system that is responsive to the needs and cognisant of the rights of health consumers.1

In Australia, the focus on partnership approaches between researchers, health professionals and the community has been seen as a central part of this transformational change.2 This partnership must extend the role of the consumer beyond engagement and participations and enable a more substantial contribution in all aspects of health system and health service activity — as co-participants and co-creators of health services and health system sustainability.1,3 It is this form of consumer value co-creation that is critical to improving health services, enhancing the quality of care, increasing patient and health care provider satisfaction, and contributing to primary health care reform in Australia.

Consumer involvement in value co-creation

The inclusion of consumers in health care has been variously described as patient participation, patient-centredness, shared decision making and patient engagement.4,5 While consumers continue to be involved as active participants in managing their own health,6 the focus is now moving forward to include the potential for consumers to be involved in innovation and value co-creation in health care.4,6

Consumer value co-creation is not a new concept and has been described in business management literature and practice since the 1970s.1 Payne and Frow7 described two dimensions of co-creation, namely co-production and co-creation relating to value-in-use. Co-production is defined as the engagement of consumers in a specific product development; examples of this in health care encompass the development of health care services such as proposed models for chronic care.4,7 In comparison, value co-creation is defined as collaborative activities that enhance customer lifetime value.7 In this definition, value is co-created if and when a consumer is able to personalise his or her experience in using an organisation’s services or products and in undertaking tasks the organisation gives them.8 Thus, consumers determine value based on their individual experiences as end users of health care, and have control over value co-creation in health care. The health care team participates in this process as co-creators.9

There is growing literature on health value co-creation and the benefits of consumer value co-creation in the health care sector.10–13 These include increased efficiencies in health services,14 improved health outcomes,15 increased trust in the health care team, reduced health care costs to the patient and the health system, increased value and use of medical research, and increased patient satisfaction and compliance with treatment regimens.12,16,17 Value can be co-created for the individual, clinical practices, health care organisations and providers, and government.12

Engagement opportunities for value co-creation

There are many co-creation opportunities where the health consumer can engage with the health system, health professionals and other stakeholders to co-create value. Engagement platforms vary in type (such as cognitive, emotional and behavioural engagement); in level (from non-engaged to highly engaged individuals); and in duration (one-off, recurring and continuous engagement).18 It is through purposefully designed co-creative interactions — such as face-to-face discussions, online information sharing, collaborative product development, physical spaces for knowledge exchange and problem solving, site visits, open forums or joint ventures19 — that health consumers share their resources and experiences, build relationships and so co-create value.20 The building blocks of interaction for value co-creation include dialogue, access, risks–benefits and transparency.20 For active dialogue to occur, health care organisations or health care providers and consumers must become equal partners, focused on the issues of interest to both.

A framework for consumer value co-creation in primary health care

The use of value co-creation in health care involves embedding this approach across entire health systems — from the microsystem level (individual practices) to the mesosystem (health care organisations) and the entire macrosystem (overall health system and government policy).1 The system thus needs to allow for the consumers to inject their views, experiences and expectations through ongoing dialogue on varying aspects of treatment options, health services delivery, health policy and the overall health system.1

In relation to health care, numerous models and approaches to value co-creation have been proposed. Drawing on both the theory and practice of value co-creation, Nambisan and Nambisan6 describe four models of value co-creation in relation to consumer and health care organisations. These are:

• a partnership model for consumer co-creation in health care (ie, explicit knowledge acquired from consumers, such as innovative ideas, are combined with existing knowledge to develop new solutions or to improve existing services);

• an open-source model of consumer value co-creation in health care (consumer and community led activities with a focus on new knowledge co-creation);

• the support group model of consumer value co-creation in health care (consumer and community led forums for sharing specific knowledge and experiences related to disease and treatment); and

• the diffusion model of consumer value co-creation (knowledge-sharing activities initiated and led by health care organisations, focusing on those services or products offered by the organisation).6

In the following section, we provide examples of how these models have been used or are expected to be adopted across the health system.

Use of consumer value co-creation across different levels of the health system

Health care in Australia has only just begun to regard consumers as integral to value co-creation. Frameworks such as the maturity matrix developed by KPMG Global Healthcare21 reinforce the need for comprehensive and transformative approaches to realise the benefits of patient involvement in health care improvement. In addition, they encourage health care organisations and providers and government to move beyond simply involving consumers on boards and committees towards ongoing genuine consumer involvement across the journey from the inception and development phases to the implementation and evaluation phases of health care.

There is, however, an emerging trend of co-creation models taking place at the micro-, meso- and macrosystem levels in the Australian context. This demonstrates a growing recognition that working with consumers to establish what matters most to people about the health care they receive can powerfully shape health status and outcomes through improved service and system development.

The support group and diffusion models of co-creation have been applied at the microsystem level in hospital settings. In 2013, the Patient Based Care Challenge of the New South Wales Clinical Excellence Commission was implemented to spur system-wide change to promote patient-centred care. Designed in collaboration with a patient advisory committee, it has been implemented in 13 local health districts. Each district had tailored partnerships with patients, families and carers to address the health service priorities of their local community, which included involving them in the design of processes, new facilities and staff interview panels. Patient experience surveys and patient complaints data will inform an assessment of the impact that this initiative has had on hospital culture, systems and practices.22

At the mesosystem level, we can expect to see the partnership model of co-creation systemically applied. Guidelines require Primary Health Networks (PHNs) to involve consumer experience intrinsically in their core strategic and operational practices.23 It will be critical that PHNs appreciate the various models of consumer value co-creation and that steps are taken to support them to systematically adopt and embed good practice in consumer participation in all phases of primary care commissioning. A standards-driven approach to commissioning that is person-centred and outcomes-focused has enormous potential to exercise a lasting impact on the triple aims of health care: better population health, better experiences of care and more cost-effective health care delivery.

In education and research settings, we see the application of partnership and open-source models to initiatives that enable consumer input in generating ideas for local service and system improvements and provide suggestions for direction in health and medical research. For example, the University of South Australia’s International Centre for Allied Health Evidence offers a Professional Certificate in Health Consumer Engagement co-designed with the Health Consumers Alliance of South Australia. Health professionals and consumers worked together to inform the objectives, curriculum, delivery and assessment. Course participants are health consumer representatives and professionals who learn together, and plan and conduct a joint workplace improvement-oriented intervention.

At the macro level, broadly reflecting the partnership model, national policy mandates consumer representation and engagement as an important factor in policy development and governance of national agencies. Consumer representation is evident at the highest level in key federal reviews including the Primary Health Care Advisory Group,24 which examined new ways to coordinate and fund care for people with complex and chronic conditions, the Expert Panel for the Review of Pharmacy Remuneration and Regulation,25 and the Medicare Benefits Schedule Review Taskforce.26 Consumer involvement in the implementation of government responses to these reviews will be important to the integrity of policy translation and to upholding policy intent. A peak consumer body, the Consumers Health Forum of Australia, is a key part of the health architecture and facilitates consumer input into policy and decision making and builds the capacity of the consumer sector overall in public participation.

Conclusion

Existing and emerging practice demonstrates the role and value of consumers in shaping health care in Australia. Such practice is starting to become better accepted and embedded at various levels across the system and is no longer a peripheral issue in policy, research, program and service development, and care delivery. However, shifting from the paternalistic caregiver–care receiver dynamic continues to be challenging.27 Embracing the way we conceive of patients and thinking of them less as “users and choosers” and more as “makers and shapers” of services offers a way of achieving a more collaborative dynamic.28 In the Australian context, this requires further investment in consumer leadership, in building the capacity of consumers and all relevant stakeholders as co-creators, and in the active promotion of patient–clinician alliances. Only then can consumers define the destination, plan the journey and share the drive.

There is a growing focus on the central role of primary care in providing high-quality, accessible, and cost-effective care for international communities. However, maximising the reach and uptake of evidence into primary care policy and practice continues to be a major challenge and a major focus in health reform.1–4 Research translation must ideally be directed in a bottom-up fashion from the community, through effective and consultative partnerships at every stage.5 This demands a “refocus” on the way we conduct health services research,6,7 requiring that strategic partnerships with influential end users are formed early in the research process and nurtured throughout the entire development and implementation journey.

In an effort to maximise investment in research, funding bodies and governments are increasingly working with the academic sector to strengthen partnerships with industry, society and end users of the research to improve the productivity of the sector.8–11 In response, researchers have increasingly adopted approaches such as translational research,12 implementation research,13 and community-based participatory methods14 to facilitate uptake of research findings into health policy and practice.15,16 Although these terms emphasise the use of research in “real-world” settings, they ultimately remain researcher-driven approaches. Thus they may fail to engage the end users of the research (patients, clinicians, and other key stakeholders within health care systems) adequately, may not recognise stakeholder and end user values, and so, may limit effective translation into policy and practice.12 In many cases, the researcher may lack insight into the end users’ specific needs and values in the all-important development phase, making the research outcomes difficult to implement and often unsustainable in the real-world setting.16

Over the past decade, the business sector has witnessed the transformation of multi-million dollar enterprises through the adoption of the “value co-creation paradigm”.17 This paradigm includes important concepts, innovative approaches and key principles, within an overarching framework. Value co-creation is focused on creating value with, and for, multiple stakeholding individuals through regular interactions and value-creating processes.17 These, in turn, create opportunities for increasing innovation, productivity and co-created outcomes of value for the end users, the community at large and the economy.17 This approach is now gaining momentum outside the business world, where it has been used to increase innovation within the public sector to create new joint value.18 There is also a growing literature on health value co-creation and the benefits of consumer value co-creation in the health sector.19–22

In 2011, our Centre of Research Excellence (CRE) in Building Primary Care Quality, Performance and Sustainability was established through a partnership between the University of Queensland and some of Australia’s most influential primary care organisations.23 We used the value co-creation approach to encourage innovation, unify partner capability and maximise the impact of our complex program of research.17,24 In this article, we describe the underpinning principles of value co-creation, the benefits of using this approach, lessons learned and strategies to achieve value co-creation in primary care services research.

A new vision for value co-creation

Value co-creation introduces a new vision of value creation through a shift in thinking about potential co-creators of value, the value networks, and the entire value of ecosystems.25 Value co-creation also requires a redefining of the way an organisation or group engages individuals internally (employees, teams, departments) and externally (suppliers, partners, and other stakeholders) through a process of value creation and engagement in enriched experiences to design new products and services, transform management systems, increase innovation, productivity and returns on investment.24 Strategy is the art of creating value26–28 and, in a co-creative research enterprise, the central focus of the strategy becomes making choices about where and how to co-create value with stakeholders and end users of the research (Box 1).

Co-creation expands the creation of value in three ways: (i) value as enacted through co-creative interactions; (ii) value as exemplified experiences; and, (iii) value as emerging from strategic stakeholder engagement through diverse collaborations across multiple sectors and evolvable ecosystems of capabilities (Box 2).17 Key steps and principles that capture the essence of implementing value co-creation are described below. In Box 3, we provide a case study that illustrates the application of the six key steps to the development of a quality improvement tool for primary care, one of the centrepieces of our CRE program.

Value co-creation steps

1. Strategically engaging stakeholders in creating value together

The first step in any co-creative initiative is to identify key stakeholders and end users who can take part in the co-creation efforts. Value co-creation demands that organisations and individuals reach out to different groups, enterprises or sectors so they can co-create new knowledge together, thus generating more value.17,24 A deeper collaboration with multiple influential stakeholders increases the pool of resources, competencies and capabilities, which accelerates value creation opportunities for all.17,29 With this in mind, the research team must identify and invite research stakeholders, including influential clinical organisations, policy makers, funders and other end users to take part in the co-creation journey from inception.

2. Identifying and supporting champions

Over time, influential co-creation champions need to be identified and supported within all involved enterprises, ecosystems and stakeholder organisations to increase the chance of success of the co-creative initiative.17,29 Change is challenging, and using new approaches to innovation can be confronting to those accustomed to following traditional processes. As the aim of value co-creation is to embed collaborative, ongoing, successful relationships between stakeholders, so champions of the new approach are essential to lever and maintain cultural change in both research, funding, policy and end user groups.

3. Purposefully designing co-creative interactions for stakeholder engagements

The value co-creation approach positions the source of value within the co-creation experience, which is actualised through the organisation–stakeholder or end user co-creative interactions. These interactions must be purposefully designed by strategically aligning people, processes and interfaces to co-create the value propositions and outcomes together17,29 while being mindful of stakeholders’ individual domains of experience and with the aim of building effective connections. Engagement platforms can be formal face-to-face meetings, online interactions (website, webinars, and blogs), private or public community forums, site visits or open forums. These platforms must also enable new experiences by ensuring opportunities for end users to co-create their own unique outcomes (products, services, and experiences).17,29

4. Expanding the circle of stakeholders and joint value creation opportunities

A co-creative enterprise must be cognisant of maximising the ecosystems of capabilities in the social, business, civic, and natural communities in which stakeholders exist to produce new co-creative capacities of value creation.17 It is important for the enterprise to assess value in order to verify that those involved have generated new value. Further to this, it is imperative to recognise that value is subjective and varies as a function of the co-creation experiences of stakeholders, both their experiences in interacting through the platform and the experiences of the co-created outcomes.17,29 There are many different kinds of value — economic or societal gains, organisational improvement, personal achievements, satisfied stakeholders just to name a few.17

5

Engaging stakeholders in private, public, and social sectors to expand benefit for all

It is important for an enterprise to be mindful that building ecosystems of capabilities together with other private, public, and social sector enterprises will expand the wealth–welfare–wellbeing all around. By strategically engaging stakeholders across the numerous industries and domains, co-creation enables more resourceful and sustainable growth opportunities which, in turn, benefit the stakeholders in the ecosystems, the economy and society as a whole.17,26

6

Deepening the impact and enable the viral spread of “win more–win more” value creation

In the co-creation paradigm, we work with others to continuously multiply and join the capabilities that exist across a variety of industries and business domains; across the private, public and social sectors. Succeeding together in growing value of many kinds, fuels a “win more–win more” environment, in which the number and scope of interested co-creation partners expands and the capability to spread and build on value co-creation approaches increases.17,26

Co-created outcomes of value

Embedding value co-creation across the CRE research program led to co-created outcomes of significant value which are described briefly below.

Robust and enduring partnerships

The approach encouraged the research team to build genuine multi-stakeholder relationships with influential partner primary care organisations, national and international researchers, clinicians, government, consumers, and relevant stakeholders to undertake collaborative research. From inception, the approach brought together influential primary care peak bodies including the Royal Australian College of General Practitioners; the Australian Association of Practice Managers; the Australian Primary Health Care Nurses Association; the Department of Health; Australian General Practice Accreditation Limited; the Improvement Foundation; Chronic Illness Alliance; and the Australian Commission on Safety and Quality in Health Care; and the Medicare Locals (now Primary Health Networks [PHNs]). These partner organisations collectively represent 40 000 primary care clinicians and managers delivering more than 100 million patient consultations per year, and organisations responsible for establishing policy, and the quality and safety benchmarks for Australian primary care.

Co-created research outcomes of value to end users

The involvement and input from the strategic partners throughout the entire research journey, that is, from the development of the research program to the implementation and evaluation phases, ensured that the co-created research outcomes were highly relevant and easily transferrable to the real world. One example of a co-created research journey is provided in Box 3, showing how value co-creation resulted in development of the Primary Care Practice Improvement Tool (PC-PIT), an on-the-ground quality improvement tool now embedded in practices Australia-wide. Another approach delivered a best-practice governance framework30 now in use in integrated care arrangements between PHNs and local hospital and health services.

Establishment of the International Implementation Research Network in Primary Care

Through co-creative networking and a focus on building value in service innovation internationally, the CRE and stakeholders began the International Implementation Research Network in Primary Care (IIRNPC). This collaboration of researchers and stakeholders shared a vision of advancing the field of implementation research with a specific focus on care transformation policy and practice.31 The IIRNPC, through multiple channels and interactions (such as e-newsletters, online forums, website interactions, a shared repository of publications and resources, and annual face-to-face meetings), stayed engaged, shared implementation research, developed expertise on implementation and translational research methods, and developed international collaborative primary care research.

Formation of the International Primary Health Reform Conference

The CRE’s approach in working with end users to co-create value in primary care research has expanded its national and international relationships and networks with influential partners. The CRE has been fortunate to co-host three successful International Primary Health Care Reform Conferences, bringing together primary care organisations, policy makers, consumers, researchers and clinicians, to continue the reform platform for primary care globally, and to further develop our international network for future growth.32

Strategies to achieve value co-creation in primary care services research

Leading theorists predict that value co-creation will become a primary source of an organisation’s competitive advantage.33 In our experience, value co-creation has drawn together strategic primary care organisations responsible for over 100 million consultations annually. As the case study (Box 3) illustrates, the key benefits of value co-creation lie in the development of relationships with stakeholders through meaningful engagement (co-creative interactions). It focuses on improving the experiences of stakeholders through the provision of strategic value creation opportunities. We propose that applying the principles of value co-creation in a way that strategically and genuinely considers all influential stakeholders from different ecosystems, maximises the productivity, creativity, utility and impact of the evidenced-based research outcomes and their translation to primary care policy and practice. To use this approach in primary care services research will require a commitment from the research enterprise to the value co-creation principles. It also requires creative research management processes and solutions and an effective leadership and governance framework.

The use and implementation of value co-creation will vary from one case to the next. Yet, there are key principles and strategies which are essential to successful value co-creation that leaders should consider as they follow the six steps and apply this approach in primary care services research. In Box 4, we provide these based on our experience of applying value co-creation. Underpinning these principles are four key lessons we learned, summarised below.

Partnership development requires an investment of time, effort and resources

The regular engagements required for research stakeholders to build effective long term relationships and co-create value takes effort, time and resources.19 Considering that most research programs or projects have limited time frames and funding, researchers must strategically factor in extended time, resources and budget to allow for successful value co-creation. In addition, funding bodies and university organisations continue to hold traditional views of research which require clearly articulated aims, set methodological processes and proposed outcomes that are largely inflexible and do not allow for the dynamic and evolving nature of value co-creation.

Management systems need to be “co-creative”

Creative research management solutions are needed to ensure that the benefits of a true co-creation approach can be achieved. This includes effective leadership and governance to manage multi-stakeholder relationships, and the design of appropriate platforms of engagement to yield useful co-created research outcomes. Traditional research teams do not typically have these skills, and may consider recruiting appropriate staff or providing training in the principles and application of the co-creation paradigm.

From the health system perspective, co-creative leadership entails the ability to embed a co-creation culture within the research enterprise, understanding the process of co-creation through its key building blocks (dialogue, access, risk assessment and transparency),34 embracing co-creative engagement, and designing co-creative management systems and processes with internal and external stakeholders.17 The research enterprise, as the organising agency, sets the overall strategic direction and defines the boundaries between what can and cannot be created.28 The governance structures, including the identification of the roles and responsibilities of people and projects, must be co-created by all involved. Research committees and boards must have representation from all stakeholders and end users of the research, and the management system should allow stakeholders to participate as co-creators throughout the entire enterprise (ie, co-creating value identification, strategy formulation and execution, fine tuning processes, involvement in problem solving, performance and risk management activities, and implementation).17

Openness and understanding is needed to work across different sectors and cultures

Working across the research–policy interface and care delivery systems can pose challenges related to differing cultures, priorities, preferred styles of communication, and time frames.23 The value co-creation approach must allow time for getting to understand stakeholder needs and values (which are subjective and can evolve in the course of the research journey) and thus ensure that the research team is in a position to better meet them (eg, designing more appropriate platforms of engagement, adjusting methods or deliverables, and producing co-created outcomes in a timely manner).

Flexibility, fairness and transparency is essential in value co-creation

It is widely accepted that value co-creation involves the free exchange of knowledge, information and contribution in different forms from external stakeholders. Therefore, issues of fairness, ethics, and intellectual property need to be considered, and in some senses reinvented, as part of an ongoing co-creation process. Once again, it is linked to a deeper understanding of stakeholder motives for involvement and engagement which can range from intrinsic (altruistic, free-worker) to extrinsic (economic rewards);35 and their perspectives of fairness, ethics, and values.36 From our experience, and supported by the literature,36 transparency and constant dialogue are the best solutions to ensure that these differing perspectives align. Clarifying expectations between stakeholders reduces potential perceptions of “free riding”, exploitation, or the perception that intellectual property is being used for the exclusive benefits of other participants or organisations. It includes mutually constructing rules of engagement, encouraging stakeholders to voice their needs, discuss their obligations, give their consent and identify perceived benefits as these evolve throughout the journey.

Conclusion

The value co-creation paradigm provides an effective framework to join researchers and the broader community in better creating end user value.23 It is an effective, sustainable approach to building and expanding long term research partnerships across multiple sectors, and increasing research productivity and the benefits of co-created outcomes for stakeholders, the health care economy and society as a whole.

As the example of the PC-PIT shows, the essence of the value co-creation is the development of long term, trusting relationships with and between key stakeholders and end users of the research, ultimately resulting in greater uptake of research outcomes into both policy and practice. This facilitates a shift in focus from traditional research processes and outcomes toward stakeholder or end user articulated processes and co-created outcomes. The “human experience”, forms the central focus of the co-creation strategy, leading to a cycle of ongoing, valuable and sustained relationships between researchers and end users.

Box 2 –
Value co-creation in primary care services research*

---

Adapted from Ramaswamy and Ozcan, 2014, figures 1–3, page 29.17

Box 4 –
Summary of the key principles and strategies for achieving value co-creation in primary care health services research17,23,24,26,28,34

Harnessing our collective capability could enable effective, ongoing reform

For over 20 years, the composition, delivery and importance of primary care have been the subject of quickening international reform. In 1993, the United Kingdom government announced a “primary care-led” National Health Service, underpinned by primary care commissioning and general practitioner fundholding, both vehicles to allow local primary care clinicians to influence government purchasing of health services for their community.1 In 2001, New Zealand introduced its primary health organisations, delivering comprehensive primary health services to enrolled populations either directly or through provider members.2 In 2007, the United States described the patient-centered medical home3 — an approach now linked with provider accountability, care analytics, and a focus on population health and the Affordable Care Act, often referred to as ObamaCare. This was signed into US law in March 2010 and aims to expand the affordability, quality and availability of private and public health insurance through consumer protections, regulations, insurance exchanges and other reform.

In Australia, primary care reform has been more timid, with general practice accreditation to support quality improvement, Divisions of General Practice and Medicare Locals tasked with specific local health system improvements, and an open-ended Medicare system expanded to support team care.4

International health care delivery is now challenged and enabled by the changing epidemiology of health and disease (eg, the recent emergence of the Zika virus), the exciting but confronting reality of disruptive technology (technology that displaces or challenges an established technology), rapidly increasing health care costs, and societies that are increasingly experiencing and celebrating the power of the individual. This suggests future health care systems that will be digitally accessible, appropriately responsive, mindful of costs and benefits, and appreciative of the centrality of patient choice and engagement. Our national health reviews of primary care,5 Medicare,6 private health insurance7 and eHealth8 have recently been completed, and 2016 will usher in their policy impact. The need for change is now undeniable. A value co-creation process, where stakeholders and end users share, combine and leverage each other’s resources and abilities from design to implementation, would provide the collaborative, ongoing impetus required for success.9,10

History has taught us that health care reform is not a finite deliverable. It is, ideally, an ongoing, relationship-based team pursuit of excellence in the community delivery and utilisation of health care — a descriptor not often ascribed to current arrangements. Value, like beauty, is often in the eye of the beholder. Communities value affordability, accessibility, personalisation and quality; service deliverers seek quality, patient satisfaction, professional reward and remuneration; and governments merit quality, safety and cost–benefit return.

Understanding and engaging stakeholders, collectively, in creating value is immensely challenging and time consuming, requiring skilled facilitation and focus. It also requires new platforms of engagement, which allow appropriate input, shared vision and governance arrangements that acknowledge, and positively leverage, inevitable reform tension. Alliance contracting — one such platform used in Canterbury, NZ — has delivered significant and ongoing benefits in health and social care. This involves collaboration between community and acute care providers in a region to address complex health and social care problems by taking a “whole-of-system approach to planning and decision making based on what is best for the patient and health system”.11 As this model becomes a NZ-wide approach to regional co-creation, the future focus will be on achieving agreed outcomes that benefit whole communities, rather than the traditional siloed planning, budgeting and delivery. This approach to value creation could provide a useful reform template for Australia.

Consumers of care will be important co-creators of new care models. With about half of health care service and spend estimated to be related to unhealthy lifestyle choice, consumer buy-in has never been so important in delivering care that makes a difference.

While much easier said than done, the experiences of consumers via individuals, organisations or representative groups should be respected and valued at all phases of reform design, testing, implementation and review. Encouraging, supporting and incentivising patients’ engagement in their own care is also fundamental.

Leaders and champions will be essential for success. It is easy to frighten communities regarding the unknown; the more challenging option is to harness change to confront, address and improve the realities of the future. The future for health, as with education and banking, involves the reality of disruptive change that will enable new approaches that build capacity and access, resulting in a shift in traditional methodology. A rapidly reduced national gross domestic product is a current powerful imperative for primary care’s chief funder to require a focus on achieving more with less. A co-creation approach, involving providers and end users as active partners in addressing and responding to this reform driver, would be highly beneficial. Initiatives such as Choosing Wisely12 and the Medicare Benefits Schedule Review6 could benefit from this methodology. But critical to the success of ongoing reform are leaders and champions who understand the reality and complexity of clinical care, can communicate clearly the change drivers, see the opportunity as well as potential difficulty, and lead their organisations and communities as important co-creators in future health care delivery.

Important new stakeholders and influences will arise from public, private and social sectors, as building the value and outputs of care for health consumers becomes the central target. Technology partners are increasingly important, as are those who assist in measuring the impact of health endeavours. Data collection, sharing, meaningful review and quality improvement are just beginning at a systems level, yet partnerships which embrace and enhance these can grow significant value.

Growing experience and confidence with the techniques and benefits of value co-creation will enable a “win more, win more” result, as partners in successful co-creation ventures enlist a growing circle of interested parties, including volunteer groups, local government, the health-focused business community, and social care organisations.

Peering into the primary health care system of the future may be frightening for some, but it can also be exciting for many. Technological advances will allow rural, aged, time-poor and disabled Australians to step beyond the tyranny of distance and engage more actively in their health. Point-of-care testing and community care models will reduce the cost and inconvenience of care. The “burning platform” of the health care budget may even unlock collective innovation and a willingness to change. Health care reform is not linear — there will be unexpected outcomes (both good and bad) and many learning opportunities. Most of all, there may be the opportunity to collectively unite over time to extract maximum value from our health care system for all of us who are so dependent on it.

Using a value co-creation approach to build closer integration between researchers and the “business” of health care can deliver effective health care reform

Australia’s outgoing Chief Scientist Ian Chubb issued a researcher “call to arms” in November 2015, when he identified Australia as the Organisation for Economic Co-operation and Development (OECD) country in which the research and business communities were least likely to engage with each other: “[given] the rhetorical tailwinds blowing over decades … you would have to conclude that we have a very big anchor (our culture) or no sails (our collective will)”.1 This Supplement supports the importance of this link and argues that the mismatch in a nation that does well in research but poorly on translation can be overcome — but only with our collective focus, new approaches to engagement, and a willingness to capitalise on a governmental direction now firmly rooted in innovation.

In 2013, the Strategic Review of Health and Medical Research (the McKeon Review) made 21 recommendations for improving Australia’s research quality and productivity.2 Among them was the bringing together of hospital and community care networks, universities and research organisations “to embed research within the health care system” and “facilitate best-practice translation of research into healthcare practice”.2

Nearly 3 years later, we are yet to see real evidence of more productive researcher and end user partnerships. Unlike in the United Kingdom, where evidence of the impact of community research is a key requirement for funding success, our national research metrics continue to be heavily weighted to international publication performance and nationally competitive grant processes, which disadvantage non-academic chief investigators. Similarly, health services rarely invite relevant academics or end users to bring their evidence or experience to inform and “grow value” around significant service innovation.

Concurrently, the broader community is witnessing significant change in the way value is conceived and created. This includes a shift from the value-chain model, where value is added by different suppliers in a sequential process, to the constellation model, where value is co-produced by different actors in a non-linear set of interactions — as exemplified by the value co-creation concept.3 Value co-creation occurs when organisations, stakeholders, and end users share, combine and renew each other’s resources and abilities throughout the entire journey from design and production to implementation and continuous development.4,5

Value is subjective and varies as a function of the co-creation experiences of consumers and stakeholders.5 Therefore, value goes beyond optimising health outcomes, and can be achieved in different forms — for example, as organisational improvement, as personal achievements and experiences, via satisfied consumers and stakeholders, or by economic or societal gains.6

The value co-creation approach has been successfully demonstrated within many industries, including tourism and commerce (think Expedia, eBay and Amazon), where design processes have been “disrupted” in order to better understand end user expectations, facilitate meaningful dialogue and improve value for all.5 The application of value co-creation in health care involves a paradigm shift across the entire system — incentivising researchers, clinicians, policy makers, consumers, health care organisations and other stakeholders to jointly explore and create better value in health policy, system design and service delivery.7,8

Key stakeholders in health care include government departments, whose job it is to provide frank, impartial, evidence-based advice to government on policies that will improve community outcomes. Consultation and “real” engagement between the policy and research sectors is critical to maximise value, and much current reform policy is concerned with initiatives that emphasise cooperation between different sectors, such as the primary and secondary care sectors, the public and private sectors, and Commonwealth and state funded sectors. Approaches such as integrated care initiatives seek to link different sectors together in a seamless manner. An example is the “beacon” outpatient substitution model, which brings the strengths of hospital and community care delivery together via partnership between general practitioners with special interests and consultants to increase care quality and reduce cost for patients with complex chronic disease.9,10

In undertaking this work, governments bring together key policy levers around funding, legislation and regulation to encourage and drive improvement. All of these rely on healthy cooperation and engagement between different sectors and ideally should operate in an environment of co-creation, including where appropriate, the relevant research community.

Better use of existing health data is also an increasingly recognised area for value creation. Although impressive efforts have already been made, in the form of reports and peer-reviewed articles from the National Health Performance Authority (NHPA), the Australian Commission on Safety and Quality in Health Care (ACSQHC) and the New South Wales Bureau of Health Information, there remains much untapped potential for these data to be used to build value in health service improvement at all levels. Innovative partnerships between consumers, health professionals, policy makers, researchers and data organisations have demonstrated how new knowledge, using techniques to mine big data, reveals where investments to improve care in Australia will yield the highest return. The NHPA has publicly named postcode areas where improvements in immunisation rates would almost certainly prevent sickness, hospitalisation or premature death;11 local areas where avoidable hospitalisations for the treatment of chronic health conditions are at least 10 times higher than in other similar communities;12 and hospitals with the highest rates of serious yet preventable infections.13 The ACSQHC recently identified local communities where rates of antibiotic dispensing were over 11 times higher than in other similar areas, and local areas where 33 000 knee arthroscopies are performed despite evidence the procedure is of limited value for people with osteoarthritis.14 End user partnerships also offer insights into the information needs of consumers of health as they navigate complex systems. To create value, however, information mined from big data must also create a call to action and inform audiences about where the biggest opportunities lie, allowing them collectively to take informed action to change health systems and services.

In 2014, in the MJA Supplement “Building a culture of co-creation in research” (https://www.mja.com.au/journal/2014/201/3/supplement), we shared our experience of using the value co-creation approach with influential research partners and stakeholders. Now, in this Supplement, we explore the use and impact of value co-creation as a driver for health care reform, at a time when systems are reaching to improve value within resource constraints: reporting on the further study of the Primary Care Practice Improvement Tool (PC-PIT) in practice settings, and resulting development of a resource suite to support quality improvement initiatives, and providing perspectives on the evolution of the role of consumer in value co-creation, as well as the experience of using a value co-creation approach in mental health commissioning and polypharmacy reduction research design.

With Australia’s health system under ongoing pressure, due to increasing health complexity, an ageing population and rising rates of chronic disease, we need to maximise user value within tightening fiscal frameworks. A system that integrates and leverages existing opportunities and technology, increases involvement of consumers and all stakeholders, and creates opportunities for increasing innovation, productivity and co-created outcomes of value for end users is key.

What we now need is a clear national direction that locks in a vision for value co-creation at all levels of the health system — establishing ongoing, collaborative design and review as a key role for health services, end users and the research community. Such integration between the researchers and “businesses” of health care might do much to lift our current unenviable OECD ranking, and unlock the innovation for which our nation has been traditionally renowned.

There is international interest in the best methods for improving quality of care in primary health care settings.1,2 As a result, governments and health care organisations carry out large-scale programs, including national quality strategies and accreditation, in an effort to improve the quality of services, enhance patient experience and health outcomes and reduce the cost of care.3–6 Quality improvement (QI) involves “a structured process that includes assessment, refinement, evaluation and adoption of processes by an organization and its providers to achieve measurable improvements in outcomes to meet or exceed expectations”.7

In Australia, a system-wide approach for QI has been driven by the endorsement of national frameworks and policies, including the Quality Framework for Australian General Practice (2007),8 the Australian Safety and Quality Framework for Health Care (2010),9 the National Primary Health Care Strategic Framework (2013)10 and, most recently, the Primary Health Network Quality Partnership Framework (2015).11 Most QI activity in general practice has been motivated by the accreditation process based on the national standards of the Royal Australian College of General Practitioners (RACGP)5 and the opportunity to participate in externally facilitated programs, such as the Australian Primary Care Collaboratives.6 These programs have addressed issues such as chronic disease management, access and e-health and, while these efforts have had a positive impact on improving health care and building practice capacity in these areas, only 2116 out of about 7035 practices have been involved in some aspect.12,13

While the external drivers such as frameworks and accreditation for QI are important, internal factors such as organisational infrastructure, strong team leadership and a culture of QI enable practices to improve their performance and the outcomes of their patients.7,14 However, specific areas that practices choose to deal with through ongoing QI efforts, and the methods they use to do so, are likely to vary based on each practice’s concerns, circumstances, capacity and resources.3 Evidence suggests general practice teams need to “own” the quality agenda, take leadership and be actively engaged as partners in QI.4 QI that is internally led at a practice level, with support from regional networks, is likely to be more effective.15

There are numerous QI tools and resources available for clinical practices and practitioners, with various applications for improving different aspects of quality.16–20 However, the accessibility, utility and quality of these resources are variable.1 There is also increasing evidence suggesting that QI initiatives that take on a whole-of-practice approach, such as the Primary Care Practice Improvement Tool (PC-PIT), Six Sigma and the Clinical Microsystem Assessment Tool, are more effective as they engage all practice staff to improve varying aspects of organisational and clinical practice, while recognising practice context and capacity.21

There has been limited exploration of general practitioners’ and practice managers’ perceptions of, and preferences for, QI tools. Available evidence focuses predominantly on GPs and suggests that the length, format, accessibility, content, relevance, reliability (scientific evidence), skills required for use, perceived ease of implementation and perceived benefits and support are factors that affect their choice of tools and resources for use in practice.22–25

This work follows on from the pilot study and trial of the PC-PIT approach to improve organisation performance in primary care.21,26 The PC-PIT includes seven key elements integral to high-quality practice performance: patient-centred and community-focused care; leadership; governance; communication; change management; a culture of performance; and information and information technology.21,26 During the trial, practice managers identified the need for additional resources to support the PC-PIT and provide a “one-stop shop” for organisational performance improvement.

After conducting a systematic literature review to identify online QI tools and resources to support organisational improvement related to the seven elements in the PC-PIT,27 we conducted this Delphi study to assess their relevance and utility and select a final suite of tools and resources to both complement the PC-PIT and support QI activities in Australian general practice.26

Methods

This study was conducted from November 2014 to August 2015. Ethics approval was granted by the University of Queensland Behavioural and Social Sciences Ethical Review Committee (approval number 201000924).

An Expert Advisory Panel evaluated the 53 tools and resources identified in the systematic review (Appendix 1)27 using a modified Delphi technique.28,29 In contrast to the “pure” Delphi process, which provides collated feedback from reviewers back to all reviewers over a series of rounds, in this study de-identified collated feedback was provided to all reviewers during the final (third) round only. This modification was necessitated by the workload and time constraints of the panel members. The modified Delphi process was chosen for its efficient use of time and resources, as well as its ability to minimise the impact of group interaction and influence.30,31 It is also a method for providing valuable expert information where knowledge is incomplete.30

Evaluation process

A five-step approach was used in the evaluation (Box 1).

Step 1. Establish Expert Advisory Panel

A panel of six GPs and six practice managers (as end users of QI tools and resources) who each had more than 5 years’ experience in QI activities in general practice was purposively recruited from practices that had participated in the PC-PIT pilot study in 2013. Of the panel of 12, five were male and seven female, and their practice settings were evenly divided between metropolitan and regional areas. The main task of the panel was to assess, using a standard assessment form, the relevance and utility of selected tools identified through the international literature review.27

Step 2. Develop and pilot assessment form

The pre-tested assessment form (Appendix 2) was based on five domains commonly used for assessing quality of interventions, health information and websites: (i) target audience; (ii) relevance to the PC-PIT; (iii) usability; (iv) strengths; and (v) limitations (pertaining to utility).32–34 A mix of tick-box categories and open-ended questions and statements (eg, “Please comment on the strengths, limitations and your overall perception of the utility of using this tool in your practice”) elicited qualitative feedback from the reviewers. The final section of the form asked reviewers to make an overall recommendation (“I do not endorse this tool”, “I am unsure about recommending this tool” or “I recommend this tool to be used to complement the PC-PIT”) and provide a written justification for the chosen recommendation. A score from 0 to 10 for each tool (where 0 indicated poor utility and 10, high utility) provided additional information about tool recommendations.

Step 3. Data collection

In Rounds 1 and 2, the review process was undertaken using the standard assessment form and scoring system (Appendix 2).

Round 1 review: QI tools and resources were randomly divided into six groups of about nine tools each. Two reviewers (a GP and a practice manager) were allocated a group of tools to review. The reviewers categorised the tools and resources as recommended, not recommended or unsure, and provided reasons for their decision.

Round 2 review: Tools and resources that had received an unsure recommendation from both reviewers, or divergent recommendations (ie, one reviewer had recommended the tool, but the second reviewer did not recommend it or was unsure), in Round 1 were sent out for Round 2 review by different pairs of reviewers using the same assessment form.

Final review: The final review included the tools and resources previously recommended by both the practice manager and GP in Rounds 1 and 2, plus the three highest scoring (those that scored 29/40) divergent Round 2 tools and resources that were relevant to the Australian context, as indicated in the reviewer justification comments. All 12 members of the Expert Advisory Panel were invited to participate in the final review, and nine (five practice managers and four GPs) were able to do so. Each was sent a spreadsheet (Microsoft Excel) that included online links to the tools and resources, with de-identified comments and scores from the Round 1 and 2 reviews. Using the modified Delphi technique, reviewers were instructed to consider the comments and scores from their peers and make a decision to accept or reject each tool or resource for inclusion in the final suite, along with providing a brief reason for their decision. This process provided reviewers with a final opportunity to revise their judgements in light of the collated information from previous rounds.

Step 4. Data analysis

Quantitative data were entered into a spreadsheet and imported into SPSS version 21.0 (SPSS). Data were analysed using SPSS and Microsoft Excel 2013. One of the research team (L C) completed an additional accuracy check of about 10% of coded and entered data. Quantitative results are reported using descriptive statistics. We explored free-text responses using NVivo 10 (QSR International) and used thematic analysis to identify common themes relating to the strengths and limitations of each of the tools and resources.

Step 5. Select suite of recommended tools to support PC-PIT

Tools and resources were selected for inclusion in the final suite if they received five or more recommendations from the expert panel of nine in the final review round.

Results

Of the original 53 tools and resources provided to the Expert Advisory Panel, 28 were included in the final review round. Of these, 21 were selected for inclusion in the final suite. Box 2 shows the number of tools and resources accepted or rejected in each round.

Box 3 shows the mean and range of scores for recommended, rejected and divergent tools and resources in Rounds 1 and 2. Overall, results suggest good concordance in ratings between practice managers and GPs, and clear delineation between recommended and rejected tools and resources. All reviewers provided a recommendation, but not all provided a final score.

Of the final suite of 21 tools and resources (Box 4), five were Australia-specific, six were from the United Kingdom and seven were from North America. Nearly all the tools and resources addressed two or more elements of the PC-PIT, with the most common elements being leadership, change management and patient-centred and community-focused care. Most tools and resources took a whole-of-practice approach and involved most practice staff.

As reviewers perceived the tools and resources to be relevant to different target audiences and PC-PIT elements, only consensus results are reported (ie, where both reviewers in Round 1 indicated the same audience or PC-PIT element for a recommended tool, these are recorded in Box 4; likewise where three or four reviewers indicated the same audience or element in Round 2).

Qualitative results

Key results are summarised below (full details are presented in Appendix 3). Three key themes were common to the recommended tools and resources; namely, that they were: (i) easily used (high utility), (ii) useful to the practice (high value) and (iii) complemented and supported elements of the PC-PIT. Tools and resources were more likely to be scored highly if they had been successfully used on previous occasions by the reviewer(s) or had the perceived potential to be modified or adapted, or if the reviewer indicated an intention to use the tool or resource in the future.

A broad range of limitations with the recommended tools and resources were also noted. GP comments centred mainly on poor utility (too complex or too general), whereas both GPs and practice managers focused on potential implementation challenges, including the need for further resourcing, strong leadership and buy-in from other members of the practice.

The six unanimously rejected tools and resources from Rounds 1 and 2 were perceived to have poor utility (hard to follow, too sophisticated, too generalised, too time- and resource-intensive or too wordy); be relevant mainly to hospitals or other non-primary care organisations; be already out of date; require external facilitation; or to be duplicated by or of no additional value to the PC-PIT.

There were 35 tools and resources in Round 1 and 22 in Round 2 to which practice managers and GPs gave divergent or unsure judgements. Most of these tools complemented the PC-PIT, but there were mixed comments on their utility and usefulness, and perceptions of their lack of relevance to the broader Australian context or the context of general practice. If reviewers perceived that the tool replicated the PC-PIT or existing accreditation resources, these were also noted as limitations.

Overall, the tools and resources accepted in the final suite were perceived to have high utility and relevance, which outweighed any limitations.

Tools and resources excluded from the final suite were rejected primarily because they did not fit the Australian context or were too complex. All excluded tools and resources were perceived to have some strengths, but more limitations in relation to utility when compared with the final set of tools and resources.

Discussion

Addressing practice systems is a recognised component of achieving QI in primary care.4,22,35 Our study indicates that potential end users have a preference for tools and resources that cover multiple aspects of practice function. However, issues remain regarding how to promote and sustain QI and the need for ongoing practice support, training and potential financial incentives to undertake and implement QI activities.17

As primary care organisations, health authorities and research networks work to develop QI tools, resources and approaches, our study highlights the need to consider both the capacity of health professionals and practice managers to undertake and drive QI initiatives and their preferences for tools and resources that are most applicable to the context of their practice, including factors that enable (eg, strong leadership, buy-in from others, high relevance) and hinder (eg, implementation cost) tool use.3,16,17,36,37

The final suite of tools and resources included three that were unanimously recommended by the Expert Advisory Panel: Event Analysis: the Seven Steps, the UK National Health Service clinical engagement resources and the Plan, Do, Study, Act (PDSA) cycle. These types of tools and resources are familiar to practices; the PDSA is particularly widely used by both the Australian Primary Care Collaboratives and RACGP QI programs.37,38 Health care professionals demonstrated a clear preference for resources they perceived to be of high utility — those that are simple to understand, easy to use and require no additional training.25 Tools and resources that can be used by all staff and involve all domains of practice operation are considered of highest value, particularly in identifying areas in need of change and in facilitating and monitoring the change process.36 Tested and proven tools and resources (ie, those with high credibility) and those perceived to be easily adapted to suit practice context were judged as most acceptable in our study. Indeed, the credibility of a tool or resource has been linked to greater adherence and implementation, particularly in the use of clinical guidelines.25

However, access to well designed, adaptable tools and resources is only part of the QI process. The acceptability of tools and resources is moderated by considerations relating to implementation and perceptions of the degree of benefit to be gained (perceived value). In this study, reviewers weighed up the costs of implementation (eg, time commitment, the need to pay for external facilitators) against the level of benefit that could potentially be gained. Rejected QI tools and resources were those perceived to be too time-consuming or too complex and thus beyond practice capacity.17,39,40

There has been limited research on appropriate end user-selected tools and resources relevant to general practice. Several of the tools and resources, although not designed for the Australian context, were highly regarded by the Expert Advisory Panel. There is an opportunity to further explore how these tools and resources could be adapted for the Australian primary care context and so provide additional valuable resources to support organisational performance in general practice. Whatever tools and resources are used, QI is a dynamic process and one that often requires the use of more than one QI tool or resource. It requires fostering and sustaining a QI culture, strong team leadership and the implementation of QI at the grassroots level to ensure buy-in, uptake and, ultimately, better quality care.4

Our study had some limitations. While the Delphi technique is a well recognised review method, the judgements of the selected Expert Advisory Panel may not be representative of all end users, and tool or resource acceptability may vary according to the specific interests of individuals. Perceptions of utility may also change with exposure to and increasing familiarity with specific QI tools and resources. However, we endeavoured, through the panel selection process, to engage people with a high level of experience in the field and from a diversity of practices. Our modification of the Delphi technique also limited the number of rounds of review for each of the tools and resources. Ideally, all panel members would have reviewed all tools and resources in each round, but this was not possible due to time constraints and reviewer workloads. We also acknowledge that new QI tools and resources are constantly becoming available and will not have been included in this evaluation, while existing tools and resources can become outdated or difficult to access. It is also likely that several of the excluded tools and resources could be useful for specific tasks in practice, despite their identified limitations.

In conclusion, the final suite of tools and resources to support and enhance the use of the PC-PIT offers one approach to improving the quality of primary care in Australia. Finding ways to integrate and sustain the currency of this resource suite will need the future support of existing peak professional partners, such as the RACGP, the Primary Health Network and the Australian Association of Practice Management. Further work should explore the feasibility of the use of this suite and the potential to modify useful international tools and resources to suit the Australian context.

Box 2 –
Evaluation process for assessing quality improvement tools and resources

---

* Highest scored tools were those that scored 29/40.