Indonesia has been shifting from ensuring access to health services towards improving service quality. Accreditation has been used as quality assurance (QA) mechanism, first in hospitals and subsequently in primary health care facilities, including Puskesmas (community health centres). QA provides measures of whether services meet quality targets, but quality improvement (QI) is needed to make change and achieve improvements. QI is a cyclical process with cycles of problem identification, solution testing and observation. We investigated the factors which influenced the process of QI based on experience of maternal health QI teams in three Puskesmas in Cianjur district, West Java province, Indonesia.
Qualitative data were collected using 28 in-depth interviews at two points of time: pre- (April 2016) and post- QI intervention (April 2017), involving national, provincial, district and Puskesmas managers; and Puskesmas QI team members. Thematic analysis of transcripts was conducted.
We found four main factors contributed to the process of QI: 1) leadership, including awareness and attitude of leader(s) towards QI, involvement of leader(s) in the QI process and decision-making in budget allocation for QI; 2) staff enthusiasm and multidisciplinary collaboration; 3) a culture where QI is integrated in existing responsibilities; and 4) the ongoing Puskesmas accreditation process, which increased the value of QI to the organisation.
Making QI a success in the decentralised Indonesian system requires action at four levels. At individual level, leadership attributes can create an internal quality environment and drive organisational cultural change. At team level, staff enthusiasm and collaboration can be triggered through engaging and tasking everyone in the QI process and having a shared vision of what quality should look like. At organisational level, QI should be integrated in planned activities, ensuring financial and human resources. Lastly, QI can be encouraged when it is implemented by the wider health system as part of national accreditation programmes.
Citation: Limato R, Tumbelaka P, Ahmed R, Nasir S, Syafruddin D, Ormel H, et al. (2019) What factors do make quality improvement work in primary health care? Experiences of maternal health quality improvement teams in three Puskesmas in Indonesia. PLoS ONE 14(12): e0226804. https://doi.org/10.1371/journal.pone.0226804
Editor: Kamal Gholipour, Tabriz University of Medical Sciences, ISLAMIC REPUBLIC OF IRAN
Received: May 20, 2018; Accepted: December 5, 2019; Published: December 20, 2019
Copyright: © 2019 Limato et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The REACHOUT Consortium is funded by the European Union FP7 grant (number 306090). The funders had no roles in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Efforts to improve healthcare in Indonesia have shifted from focusing on access to healthcare to improving the quality of healthcare. This is reflected in the Indonesian Ministry of Health (MoH) national strategic plans for 2010–2014 and 2015–2019 [1–4]. Specifically, these plans refer to quality related to human resources for health, drugs and medical supplies, across several objectives and over time [2,3].
Despite improvements in the uptake of skilled birth attendance and facility delivery, maternal health issues still remain a public health concern in Indonesia. The country struggled to decrease the Maternal Mortality Rate (MMR) from 359/100,000 live births in 2012 to the Millennium Development Goal [5,6] of 102/100,000 live births in 2015, reaching just 305/100,000 . Many causes of maternal death can be prevented or managed with evidence-based healthcare services that are available in Indonesia. Thus, these circumstances reflect that quality of the healthcare services might be sub-optimal or unreliable .
One MoH strategy to assess and ensure health service quality is hospital accreditation [8,9]. Since 2014, accreditation has been expanded towards primary care facilities . The main provider of primary health services is the Puskesmas (community health centre), a category of public health facilities located at the sub-district level that deliver curative and preventive services and health promotion to the communities they serve .
However, Varkey  argues that solely focusing on quality assurance and quality control through accreditation will not significantly improve health outcomes. Continuous quality improvement (QI) approaches should be incorporated in the existing quality assurance methods to improve quality of care . QI in health care is defined as a rigorous systematic process that focuses on continuous efforts and activities to achieve measurable improvement of health outcomes, system performance and professional development, with the ultimate goal to improve community health [12–15].
Most of the QI conducted in healthcare in low- and middle-income countries (LMICs) has focused on hospitals and higher-level health facilities. However, there are several studies available on QI aiming to improve maternal healthcare at the community level, particularly in Sub-Saharan Africa and Middle East. In Northern Ghana, QI teams at health post, health centre and hospital level were found to have a positive effect on key maternal and child health outcomes, including increased skilled delivery . In South Africa, QI teams that included community health workers and their supervisors contributed to improved maternal health knowledge and feeding practices among women in the community . Studies in Ethiopia  and Tanzania  report about community members being part of QI teams, resulting in increases in postnatal care attendance and general care seeking and healthy behaviours respectively. Iranian study [20,21] shows that involvement of pregnant women in the QI process give them positive pregnancy experience and is effective to improve the standard care of delivery. A few studies reported on facilitators of QI processes. In rural Tanzania, health workers mentioned new methods for monitoring progress and mentoring visits as motivating and supporting in improving maternal and newborn care . An evaluation of the maternal and newborn health partnership in Ethiopia found that community-level QI members and health workers at health centre and district level reported that the district’s culture, in relation to supervision and learning, and leadership improved, and that their capacity in QI was built [18,23].
Between April 2016 and April 2017, the plan-do-study-act (PDSA) cycle method [24–26]–with plan, do, study and act stages in each cycle–was introduced in three Puskesmas in Cianjur district, West Java province. ‘Plan’ stage includes developing change idea, and determining the success indicators, how the data will be collected, who are the subjects, and the predicted outcome. ‘Do’ stage comprises performing and documenting the intervention, its results, and the challenges. ‘Study’ stage is when the QI teams analyse the results, compare results with the predicted outcome, and make reflections on the lessons learn. Finally, the ‘act’ stage is where QI teams adapt the change idea based on the analysed results and make preparation to plan the next cycle [25,26].
The intervention focused on the process to improve care delivery of maternal health by the village midwives. QI trainings were delivered, mainly, to the supervisors of the village midwives. Over the whole implementation period, the QI process was followed, to investigate the factors that influence (successes and failures of) the QI process in the three Puskesmas, which had never engaged with the concept of QI before.
In this paper we present factors that influenced the process of QI at Puskesmas level in Indonesia based on experience of maternal health QI teams in three Puskesmas. This study aimed to contribute to improving health service quality in the primary health care system in Indonesia.
Materials and methods
REACHOUT QI intervention
A QI intervention was implemented in the district health office (DHO) and three selected Puskesmas (hereafter referred to as Puskesmas A, B, and C), based on the recommendation from the DHO. The maternal health topic was chosen due to persisting maternal deaths in Cianjur. The Puskesmas have a similar geographical context, community culture and a high annual number of maternal deaths in comparison to the other Puskesmas in the district [27–29]. The QI intervention was initiated by the REACHOUT consortium, which focuses on improving the equity, effectiveness and efficiency of community-level health services in Indonesia and is working in five other countries.
The QI intervention activities included two training workshops and follow-up on the QI progress by the Indonesian REACHOUT team (three researchers those are the first, second and fourth author) who previously completed QI training delivered by a health development professional from LSTM.
The first training workshop was conducted in April 2016, attended by two DHO officers and three Puskesmas teams (comprised of the head of Puskesmas and the midwife coordinator). Over the course of this two-day workshop, the REACHOUT team introduced the basic QI concepts and how QI differs from quality assurance. The topic on how to implement a PDSA cycle was presented in a step-by-step participatory and interactive manner. After presentation of every step, the three Puskesmas teams sat together to progressively develop their own QI project and action plan. Readings related to QI implementation were provided prior to the workshop. The expected outputs were that the attendees would conduct a similar workshop in their own institutions for their staff, form a QI team and initiate the QI implementation process, as set out in the action plans.
Following the workshop, the REACHOUT team followed up on the progress of each institution. In the first three months, the progress of three of four institutions was slow. The DHO had not started the implementation due to budget constraints, and it would not start the QI until the 2017 financial year started. Puskesmas A only formed a QI team and Puskesmas C had developed an action plan but had not started the implementation of any of the activities noted in the plan. Meanwhile, the QI team of Puskesmas B had completed one PDSA cycle. All institutions conducted a dissemination meeting about QI intervention but did not fully replicate the workshop.
Therefore, we conducted a second workshop in August 2016, which was held in each Puskesmas. This time we included all Puskesmas staff and the established QI team as participants to introduce them with QI concept and PDSA cycle. After the workshop, each Puskesmas developed an action plan with specific activities for QI in maternal health, which are summarised in Table 1. The three Puskesmas had different levels of success in the outcomes of the QI intervention, where two measured improvements in different aspects of quality of maternal care and one did not.
Data collection and analysis
Qualitative data were collected at two time points: April 2016 (baseline) and April 2017 (endline) through in-depth interviews (IDIs) with purposively selected participants. During the baseline data collection, a total of eight IDIs were done, including participants from the DHO (maternal and child health staff) and Puskesmas (the head of Puskesmas and midwife coordinator) from the three Puskesmas who were going to be involved in the QI intervention. For endline data collection, we interviewed 20 respondents. National and provincial maternal health stakeholders and members of the Puskesmas QI teams were added as study participants. The number and characteristics of these participants and the selection criteria are presented in Table 2.
The REACHOUT team developed various types of interview guides for IDIs according to participant characteristics that were translated into Indonesian and back translated for checking content-consistency (S1–S6 Files). The baseline instruments aimed to explore the perception of quality; QI and quality services delivered by village midwives; efforts, facilitators of and barriers to past efforts to improve quality; and the systems used to measure quality. The endline instruments sought to gain understanding on the changes in perception of quality and QI; the QI process including QI team formulation, how the QI team performed the PDSA cycle, the facilitators and barriers of QI process and sustaining the QI intervention; and the significant changes that occurred as a result of the QI process. In this paper, we present the results related to factors that influenced the QI process, and therefore most data come from the IDIs conducted during endline.
All IDIs were conducted in the Indonesian language by trained researchers from the REACHOUT team and the interview length was between 60 to 90 minutes. Prior to each interview, researchers explained the purpose and main topics of the interview. We asked and obtained written consent to perform and tape-record the interview from all participants. IDIs were conducted to assure that participants could be interviewed in their workplaces, allowing sensitive areas to be probed and avoiding issues of hierarchy affecting group discussion [30–32]. All interviews were digitally recorded, transcribed verbatim, and translated to English by independent translators. Research assistants checked the quality of the translations.
The coding process used open coding  in NVivo (v.11™) software, combined with a pre-defined framework of factors that could influence the process of QI as based on the topic guides. Emerging themes were discussed, and the coding was refined based on research team consensus. The coded transcripts were further analysed, “charted” and summarised in narratives for each theme and sub-theme.
The baseline interviews provided information related to perceptions about QI, efforts to perform QI and written QI policies. The eight interviewed participants had similar perceptions about QI (cyclic process which aims to improve quality) and their understanding about QI was focused on QA (a process to maintain the fulfilment of a desirable quality).
“Regarding maternal health, we have specific SOP (Standard Operating Procedure) for the midwives. Everything we do (in health services) must be in line with the SOP. That is how we improve the quality services.” (Midwife coordinator, Puskesmas C, baseline)
When asked about written policies related to QI, all participants referred to the head of Puskesmas’ mandate letter to conduct activities required for Puskesmas accreditation.
In the discussions with the QI teams in three Puskesmas during the second workshop, it was revealed that despite SOPs being available for midwives, there were no process indicators and activities to assess whether the SOP is followed accurately. The quality assessment was based on the final outcome, measured by the number of maternal deaths. The factors that influence the QI process were never systematically assessed.
From the endline interviews, we identified four main factors that influence the QI process from the perception of QI implementers in the Puskesmas: leadership, human resources, quality culture and Puskesmas accreditation.
We found that leadership was perceived as the most prominent factor that influenced the QI process. Both leaders and QI team members identified three leadership elements to be instrumental: 1) awareness and attitude of the leaders towards QI; 2) direct involvement of the leaders in the QI process; and 3) decision making on the budget allocation for QI.
The data revealed that the awareness of leaders on the importance of and the benefits offered by QI was crucial. The head of Puskesmas A and B, through the QI workshop, obtained understanding on the importance of QI to support their ongoing programmes.
“The QI workshop made us aware that it is important to have a strategic way of thinking and strategic planning, and then following it up with the interventions that we are capable of doing.” (Head of Puskesmas B, female, endline)
Most QI team members confirmed this notion as reflected in the quote below.
“The success of QI implementation depends on the leader. When the leader is passionate, it motivates me to do the process.” (QI team Puskesmas B, female, endline)
A lack of awareness of the leader resulted in implementing the QI intervention (in Puskesmas C) solely because it was appointed by a higher level of authority, in this case the DHO.
“We are involved in this QI process because we were appointed by the DHO. I don’t understand why our Puskesmas is always appointed if there is a new programme … At the beginning, we felt obliged to do this process.” (Head of Puskesmas C, male, endline)
“The head of Puskesmas does not have initiative to implement the QI programme.” (QI team Puskesmas C, male, endline)
We also found that when the leader was directly involved in QI implementation, it led to the success of the QI. The close monitoring and follow up, especially when the QI team encountered problems, helped the QI team to implement the QI.
“I helped the QI team to solve the problems they faced, especially in decision making on which issue must be prioritised.” (Head of Puskesmas B, female, endline)
The statements from the head of Puskesmas was confirmed by some of the QI team members:
“The main facilitator of the QI implementation is the support and involvement of the Puskesmas head.” (QI team Puskesmas A, female, endline)
The third leadership related issue was the decision making on budget allocation for QI activities. QI implementation requires money, and in the Puskesmas, the leader is the one who has the mandate for budgeting. Therefore, the QI activities could only be implemented once the leader approved the budget needed for related activities. Puskesmas A showed an example of how the leader supported the QI activities through budget allocation for, among others, providing village midwives with Haemoglobin digital test sticks as part of the QI intervention and this was confirmed by the QI team members.
With regard to human resources, we found two factors that influenced the QI process. First, the enthusiasm of people in the institution; and second, the collaboration between all staff to conduct the QI.
As activities in the QI process required teamwork to conduct data collection and analysis and implementation of selected interventions, the enthusiasm of Puskesmas staff to do this was seen as a contributing factor by most QI team members. Both QI team members from Puskesmas A and B stated that the enthusiasm of their colleagues made them achieve their improvement goal.
“One supporting factor that makes the quality improvement was achieved is the enthusiasm of my colleagues to do the field work.” (QI team Puskesmas B, female, endline)
What made the team members enthusiastic was that they were involved in the whole QI process starting with conceiving of the idea of change. This made them feel to be an ‘owner’ throughout the process. This is reflected in the case study (Box 1) and the following quote.
“We did the whole QI process as a team. For example, when we prioritised which problem that we want to initially intervene, besides the QI team, we also engaged village midwives, Head of Puskesmas and the nutrition QI team.” (QI team Puskesmas A, female, endline)
Box 1. Case study Puskesmas A
In Puskesmas A, the QI principles were performed. This was visible from quality-oriented leadership, involving everyone in the QI process and joint decision-making. At start, the head of Puskesmas engaged all Puskesmas staff in the QI workshop to let them get exposed to the QI concept and process. This quality-oriented leadership was also shown by including everyone in the conception of the QI work. All Puskesmas staff from different programs and sections, namely maternal and child health, nutrition, health promotion, environmental health and the Puskesmas quality team which consisted of physicians, nurses, dentist and pharmacist were gathered to discuss maternal health issues. Everyone was given opportunity to apply their knowledge and ideas in the PDSA framework. First, they identified and prioritised which maternal health problem to tackle based on the urgency, cost and feasibility. This was followed by collective decision-making on the selected interventions and task division to perform interventions. Last, an action plan formatted in a Gantt-chart was produced as a guideline for the team to track their activities. All Puskesmas staff felt enthusiastic and embraced the QI, since they were confident on the quality of the data they gathered and contributed in the decision-making process. This was shown by, for example, a physician taking a role as a resource person in a refreshing workshop for village midwives.
The second human resource factor, which was related to the first, was the collective action of all actors in the Puskesmas to work together as a team. Oftentimes, each division in the Puskesmas works in isolation to achieve their team’s goal. The QI team in Puskesmas A and C experienced a different teamwork dynamic which influenced their QI process. QI team A felt that the teamwork between all staff in the Puskesmas tangibly improved the results as stated below.
“The coordination, especially between nutrition team and maternal and child health team, is very good. This condition makes the problem identification faster than before.” (QI team Puskesmas A, female, endline)
On the contrary, the QI team of Puskesmas C faced challenges of uncooperative behaviour of other Puskesmas staff who were not in the maternal and child health (MCH) division to execute the QI activities, mostly because of ‘ego-programming’ (finding one’s own division the most important).
“Only few of my colleagues were responsive to do this process together. The rest was indifferent … The initial collaboration between the MCH team and other teams like the dentist, the general practitioner and other divisions (developed during the QI workshop) was discontinued. I think it was because we tackled the midwife problem, so the others felt that this QI implementation was solely the responsibility of the MCH team and not their responsibility.” (QI team Puskesmas C, female, endline)
A quality culture can be defined as shared values, attitudes and behaviour of everybody in the organisation on working towards continuous QI. This was revealed by both leaders and QI team members as another factor that influenced the QI process. Both leaders and QI team members who treated QI as part of the daily job were more successful in the QI process in comparison to ones who did not. Most participants from Puskesmas C considered the QI intervention as an isolated program rather part of their daily work. This is reflected in these following statements.
“The barrier to do QI is that we have limited time because we have another task to do, and this programme is colliding with other programmes.” (QI team Puskesmas C, female, endline)
“Because of the demand to do our daily routine activities, we do not use the QI checklist in every ANC service.” (QI team Puskesmas C, female, endline)
This was in contrast with the situation as perceived by study participants in Puskesmas A and B.
“The QI process doesn’t add to my workload because it is part of my daily job.” (QI team Puskesmas A, female, endline)
“The observations to the village midwives indeed required extra time, but I and my team were happy to do so. We enjoyed the process and the results.” (QI team Puskesmas B, female, endline)
During the one-year QI implementation, all three Puskesmas underwent accreditation. This process pushed the Puskesmas to develop several strategies to ensure services qualities, for example the development of written standard operating procedures for health providers, the formation of a quality team and the provision of a feedback scheme from the patients to the providers through the availability of patient satisfaction questionnaires. All informants at managerial positions and a few QI team members mentioned that accreditation and QI influenced each other.
“When I noticed the big-three assessment points in accreditation: management, individual health and community health, all of them have the quality element. I can use the knowledge obtained from the QI workshop to improve the strategies to succeed accreditation… Prior to QI, we discussed some quality strategies, but they were like a splattered puzzle. However, after we learned about QI, we were able to put all the puzzle pieces together.” (Head of Puskesmas A, female, endline)
The synergy between accreditation and QI was evident also when the accreditation assessor praised the maternal health section of Puskesmas B for their quality initiatives.
“The QI process had synergies with the accreditation. Recently, the assessor team for accreditation expressed appreciation for the maternal health team of Puskesmas B as the best team for Puskesmas accreditation in Cianjur. I think their assessment was objective because they were from an independent body.” (DHO officer, female, endline)
This study aimed to investigate factors that influenced the process of QI based on experience of maternal health QI teams in three Puskesmas which had never engaged with QI before. We found that the existence of leadership traits that support QI, individual enthusiasm, collaboration of everyone involved in the organisation, a teamwork approach, organisational quality culture and Puskesmas accreditation contributed to the success of the QI process in the Puskesmas. This became even clearer when assessing the differences between the QI interventions in Puskesmas A and B as compared to Puskesmas C. In Puskesmas C, the QI process lagged behind and less of the above described facilitators of QI were reported. As can be seen in Table 1, the implementation of the QI in this Puskesmas led to less positive, more mixed results with regard to knowledge and skills of community midwives as compared to Puskesmas A and B.
Following the findings of this study, the best approach to succeed in QI would be a multilevel approach. There are four levels where changes should occur: the level of the individual, team or group, organisation and the system where the organisation is embedded . At the individual level, as found in studies elsewhere [35–39], leadership is seen as the foundation for successful QI. A leader is the one who can provide a clear vision and guidance of the organisational direction , create an internal quality management environment [40–42] and drive organisational cultural change . One key element to ensure leader embarks in the QI process is quality literacy, which can be defined as competencies for QI–changing knowledge and values towards quality and competent behaviour to implement QI, that are obtained through quality education . However, this study found that quality education can change knowledge but not necessarily leads to behaviour change. Three heads of Puskesmas (the organisations’ leader) attended the same QI workshop, but only two of them had a changed behaviour towards quality. One viewed QI as a burden to the existing workload. Therefore, leadership should be there at different levels in the organisation, in other words, leadership of (official) leader of the organisations might not be enough for a QI process to succeed.
Literature suggests that besides quality literacy, the availability of a quality policy and incentives for QI are needed to foster quality-oriented leadership. A quality policy underpins the planning and strategies for QI, since it creates an environment where quality of care can flourish, gives space to build capacity to improve quality , and sets frameworks for quality outcomes measurement and financial reforms towards QI . Thus, it provides a leader a platform to perform QI, including in developing QI strategies and budget allocation. Another strategy to engage leader in QI is to provide incentives for QI through linking the compensated incentives to the quality metrics . In this study, the differences in awareness and attitude between the leaders were related to personal traits rather than quality literacy, and the availability of quality policy and incentives.
Another leadership element that supports QI is the engagement of the leader in QI work. Other studies found that interaction between leader and staff when planning QI strategies was related to better quality outcomes, including patient health outcomes [48,49]. This study confirmed these findings as most QI team members in Puskesmas A and B perceived that the involvement of their leader in the QI process was crucial. The other facilitator of QI work in Puskesmas was found to be budget availability. Many primary health services operate on thin margins, and even though QI can reduce costs, at the introduction it often requires additional funds . In the rather hierarchical health system of Indonesia, the budget allocation for Puskesmas operations depends on the decision of the leader–the head of Puskesmas. Therefore, the engagement of the leader in QI work is crucial in QI budget allocation. While maternal health is one of the priority programmes in Indonesia, and therefore the allocation of budget for QI in maternal health might be easier as compared to other programmes, leader and decision-makers need to be engaged in QI processes in general to ensure that necessary budget adjustments or relocations are made.
At the team or group level, two interrelated factors emerged as facilitators of QI implementation: enthusiasm of and collaboration between QI team members. There is no magic bullet to make people feel enthusiastic about QI work. However, it can be developed through setting short milestone achievements using the PDSA framework, in which the results, for example on intermediate outcomes related to maternal health, can provide information of what works and what does not work. This can make people feel the efforts pursuing a goal are worthwhile and wanting to do more . Enthusiasm is also encouraged when the QI interventions meet the goal of the organisation , in the case of the Puskesmas who participated in this QI intervention, improvement of maternal health at community level. Besides, an organisational culture that supports personal contribution  and engagement of multidisciplinary actors in a team [22,54,55] provides an environment where improvement efforts can blossom.
At the organisation level, in this study, the existence of a quality culture emerged as a facilitator of the QI process. The changes in actions towards quality are rooted in the culture of the organisation . The complexity of the quality culture lies in its development that requires changes towards quality at the two previous levels: the individual and the team level. A quality culture cannot be mandated but it has to be built through leadership and ownership [43,57–59]. Leadership sets the direction and articulates the vision of the organisation, and this trickles down through the organisation internal system. Quality-oriented leadership stimulates team members to adopt the same perspective and behaviour  and to implement quality-standard work performance . At the same time, another crucial ingredient to create a quality culture is to give a sense of ownership to the staff through involving them in decision making , thereby stimulating leadership at different levels in the organisation. Findings in this study were in line with these notions as a quality culture was there in two Puskesmas that had quality-oriented leaders and enthusiastic staff, including midwives, who were involved in decision making.
At the system level, accreditation was mentioned as one of several strategies that facilitated QI. A study in Indonesia showed that Puskesmas accreditation had a significant positive correlation with quality services for patients, in relation to reliability of the providers and responsiveness to the patients’ needs . Accreditation focuses on process and procedures, availability of written standards and compliance with standards, based on which recommendations are made by the surveyor [9,63,64]. It is administered by an independent body and based on voluntary participation of the accredited institution , although few can afford to not participate as it is a prerequisite for being included in the universal health coverage insurance scheme. However, to improve quality, it also takes internal reviews  and constantly seeking changes to ensure the standards and external review process are relevant . Therefore, using a QI approach within a model that is compatible with accreditation leads to success, as also confirmed by our study participants. Further, to ensure the sustainability of QI, its embedment into the national and local policies and improvement strategies is paramount [15,55].
There are three limitations of this study. First, short data collection period (one year) limited the assessment of the end outcomes (number of maternal deaths). The participating Puskesmas only conducted at most two PDSA cycles over the one-year period in which the study was conducted. While a further assessment on the number of maternal deaths could not be conducted, intermediate outcomes were monitored, as presented in Table 1 under the ‘study’ stage of the PDSA cycle. Second, while all Puskesmas used anonymous scorecard filled by patients after they received maternal health services, quality measurement from health care users’ point of view was not presented in this paper. We have presented the QI process as performed by the organisation, including care delivery team and health care providers. Adding the health care users’ point of view on the process of QI in maternal health could have added valuable information, as presented in other studies [20,21,67]. Studies in Ethiopia and Tanzania, where community members were part of QI teams, have shown positive maternal health-related outcomes [18,19]. Third, the introduction of the intervention and the study that followed the process were both led by the same team. This could have influenced answers of study participants. However, given the fact that study participants reflected upon both barriers and facilitators in the QI process, we trust the validity of the study results.
This study reviewed the factors influencing the process of QI in primary health care in Indonesia based on the experiences of maternal health QI teams in three Puskesmas. The process of QI in the decentralised Indonesian system is influenced by factors at four levels. At the individual level, leadership attributes create an internal quality environment and drive organisational cultural change. At the team level, staff enthusiasm and collaboration are built through engaging and tasking everyone in the QI process and having a shared vision of what quality should look like. Factors at the organisational level include integrating QI in the organisation structure and activities, securing financial resources and investing in human resources. Lastly, the existing accreditation system encouraged Puskesmas to implement and sustain QI approaches through external buy-in. QI should be implemented by the wider health system as part of national efforts to improve quality services at primary health care level.
S2 File. Interview guide QI baseline Indonesian.
S3 File. Interview guide QI team endline English.
S4 File. Interview guide QI team endline Indonesian.
S5 File. Interview guide QI stakeholders English.
We thank Vicki Doyle who introduced the QI approach to the REACHOUT team. We are grateful for the involvement and support from the DHO of Cianjur and the participating Puskesmas in this study. We appreciate the inputs from Sally Theobald and Rosalind McCollum during the early stage of the manuscript development.
- 1. Bappenas. Indonesia Health Sector Review: Consolidated Report. 2014. Available: https://catalogue.nla.gov.au/Record/6933972
- 2. Kementerian Kesehatan Republik Indonesia. Rencana Strategis Kementerian Kesehatan 2010–2014. HK.03.01/60/I/2010 Indonesia; 2010.
- 3. Kementerian Kesehatan Republik Indonesia. Rencana Strategis Kementerian Kesehatan 2015–2019. HK.02.02/Menkes/52/2015 Indonesia; 2015. 351.077 Ind r
- 4. Mahendradhata Y, Trisnantoro L, Listyadewi S, Soewondo P, Marthias T, Harimurti P, et al. The Republic of Indonesia Health System Review. Hort K, Patcharanarumol W, editors. Health Systems in Transition. New Delhi, India: SEARO Library, World Health Organization, Regional Office for South-East Asia, Indraprastha Estate, Mahatma Gandhi Marg; 2017. Available: http://apps.who.int/iris/bitstream/10665/254716/1/9789290225164-eng.pdf
- 5. Kementerian Kesehatan Republik Indonesia. Profil Kesehatan Indonesia 2015. Jakarta, Indonesia; 2016. 351.077 Ind
- 6. UNICEF Indonesia. Maternal and child health. 2012. Available: https://www.unicef.org/indonesia/A5-_E_Issue_Brief_Maternal_REV.pdf
- 7. Campbell OMR, Graham W. Strategies for reducing maternal mortality: getting on with what works. Lancet. 2006;368: 1284–1299. pmid:17027735
- 8. WHO. Quality and Accreditation in Health Care Services: A Global Review. Geneva, Switzeland; 2003. Available: http://www.who.int/hrh/documents/en/quality_accreditation.pdf
- 9. Poerwani SK, Sopacua E. Akreditasi Sebagai Upaya Peningkatan Mutu Pelayanan Rumah Sakit. Bul Penelit Sist Kesehat. 2006;9: 125–133.
- 10. Menteri Kesehatan Republik Indonesia. Peraturan Menteri Kesehatan Republik Indonesia Nomor 46 Tahun 2015 Tentang Akreditasi Puskesmas, Klinik Pratama, Tempat Praktik Mandiri Dokter, dan Prakter Mandiri Dokter Gigi. Kemenkes, No. 46/2015 Indonesia; 2015 pp. 1–289. https://doi.org/10.1073/pnas.0703993104
- 11. Menteri Kesehatan Republik Indonesia. Peraturan Menteri Kesehatan Republik Indonesia Nomor 75 Tahun 2014 tentang Pusat Kesehatan Masyarakat. No. 75/2014 Indonesia; 2014 pp. 1–24. https://doi.org/10.1017/CBO9781107415324.004
- 12. Varkey P, Reller MK, Resar RK. Basics of Quality Improvement in Health Care. Mayo Clin Proc. 2007;82: 735–739. pmid:17550754
- 13. Batalden PB, Davidoff F. What is “Quality Improvement” and How Can it Transform Healthcare? Qual Saf Heal Care. 2007;16: 2–3. pmid:17301192
- 14. Riley WJ, Moran JW, Corso LC, Beitsch LM, Bialek R, Cofsky A. Defining Quality Improvement in Public Health. J Public Heal Manag Pract. 2010;16: 5–7.
- 15. Leatherman S, Ferris T, Berwick D, Omaswa F, Crisp N. The Role of Quality Improvement in Strengthening Health Systems in Developing Countries. Int J Qual Heal Care. 2010;22: 237–243. pmid:20543209
- 16. Singh K, Brodish P, Speizer I, Barker P, Amenga-Etego I, Dasoberi I, et al. Can a Quality Improvement Project Impact Maternal and Child Health Outcomes at Scale in Northern Ghana? Heal Res Policy Syst. 2016;14: 45. pmid:27306769
- 17. Horwood C, Butler L, Barker P, Phakathi S, Haskins L, Grant M, et al. A Continuous Quality Improvement Intervention to Improve the Effectiveness of Community Health Workers Providing Care to Mothers and Children: A Cluster Randomised Controlled Trial in South Africa. Hum Resour Health. 2017;15: 39. pmid:28610590
- 18. Tesfaye S, Barry D, Gobezayehu AG, Frew AH, Stover KE, Tessema H, et al. Improving Coverage of Postnatal Care in Rural Ethiopia Using a Community-Based, Collaborative Quality Improvement Approach. J Midwifery Women’s Heal. 2014;59: 55–64. pmid:24588916
- 19. Tranced T. Implementation of community-level quality improvement in southeastern Tanzania: a mixed methods process evaluation of what worked, what didn’t, and why? London School of Hygine & Tropical Medicine. 2015. Available: http://researchonline.lshtm.ac.uk/2528880/2/2015_ITD_PhD_Tancred_T.pdf
- 20. Gholipour K, Tabrizi J, Asghari-Jafarabadi M, Iezadi S, Farshbaf N, Rahbar-Farzam F, et al. Customer’s self-audit to improve the technical quality of maternity care in Tabriz: a community trial. East Mediterr Heal J. 2016;22: 309–317. pmid:27553397
- 21. Gholipour K, Tabrizi JS, Jafarabadi MA, Iezadi S, Mardi A. Effects of customer self-Audit on the quality of maternity care in Tabriz: A cluster-randomized controlled trial. PLoS One. 2018;13: 1–19. pmid:30307957
- 22. Baker U, Petro A, Marchant T, Peterson S, Manzi F, Bergström A, et al. Health workers’ experiences of collaborative quality improvement for maternal and newborn care in rural Tanzanian health facilities: A process evaluation using the integrated’Promoting Action on Research Implementation in Health Services’ framework. PLoS One. 2018;13: 1–21. pmid:30566511
- 23. Stover KE, Tesfaye S, Frew AH, Mohammed H, Barry D, Alamineh L, et al. Building District-Level Capacity for Continuous Improvement in Maternal and Newborn Health. J Midwifery Women’s Heal. 2014;59: 91–100. pmid:24588921
- 24. Langley GJ, Moen RD, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey-Bass; 2009.
- 25. Health Quality Ontario. Quality Improvement Guide. Queen’s Printer for Ontario. 2012. pp. 1–64. Available: www.hqontorio.ca
- 26. Health Quality Ontario. Quality improvement primers: Quality improvement science. Canada; 2013. pp. 1–15. Available: http://www.hqontario.ca/Portals/0/Documents/qi/qi-science-primer-en.pdf
- 27. Seksi KIA. Rekapitulasi laporan kematian ibu menurut kecamatan dan Puskesmas kabupaten cianjur bulan Januari—Desember 2013. Cianjur; 2014.
- 28. Seksi KIA. Rekapitulasi laporan kematian ibu menurut kecamatan dan Puskesmas kabupaten Cianjur bulan Januari—Desember 2014. Cianjur; 2015.
- 29. Seksi KIA. Rekapitulasi laporan kematian ibu menurut kecamatan dan Puskesmas kabupaten Cianjur bulan Januari—Desember 2015. Cianjur; 2016.
- 30. Pope C, Mays N. Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. BMJ. 1995;311: 42–45. pmid:7613329
- 31. Mack N, Woodsong C, McQueen KM, Guest G, Namey E. Qualitative Research Methods: A data collector’s field guide. Family Health International. North Carolina, USA: Family Health International; 2011.
- 32. Spencer L, Ritchie J, O’Connor W, Morrell G, Ormston R. Analysis in Practice. Secomd. In: Ritchie J, Lewis J, Nicholls CM, Ormston R, editors. Qualitative research practice: A guide for social science students and researchers. Secomd. London, UK: SAGE Publication Ltd.; 2014.
- 33. Charmaz K. Constructing grounded theory. Second. London, UK: SAGE Publoications Ltd; 2014.
- 34. Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom and the United States: A framework for change. Milbank Q. 2001;79: 281–315. pmid:11439467
- 35. Athamneh KA. Assessment of infrastructural determinants for quality improvement programs for healthcare organizations in Qatar. Qscience Connect. 2014;26. Available: http://www.qscience.com/doi/10.5339/connect.2014.26
- 36. Somatunga LC, Sridharan S, Refai MACM, Malavige KK, Gamini LPS. Factors influencing continuous quality improvement programme in government hospitals of Sri Lanka. Int J Sci Technol. 2015;4: 118–123. Available: http://www.ijstr.org/final-print/june2015/Factors-Influencing-Continuous-Quality-Improvement-Programme-In-Government-Hospitals-Of-Sri-Lanka.pdf
- 37. Phung VH, Essam N, Asghar Z, Spaight A, Siriwardena AN. Exploration of Contextual Factors in a Successful Quality Improvement Collaborative in English Ambulance Services: Cross-sectional Survey. J Eval Clin Pract. 2016;22: 77–85. pmid:26303398
- 38. Law MP, Graham R, Ross K. A Primer on Quality in Public Health: What’s Needed to Advance CQI in Ontario Public Health? 2013.
- 39. Siriwardena AN. Improving primary care quality now and into the next decade. Qual Prim Care. 2010;18: 357–358. pmid:21294979
- 40. Paulová I, Kva MMĹ. Leadership–the Key Element in Improving Quality Management. Qual Innov Prosper. 2011;XV: 27–36.
- 41. Cianfrani CA, West JE (Jack). Cracking the case of ISO 9001:2008 for service: A simple guide to implementing quality management to service. Second. Milwaukee, Winsconsin: ASQ Quality Press; 2010.
- 42. Kliger J. Sustaining and spreading quality improvement. In: AHRC (Agency for Healthcare Research and Quality [Internet]. 2014 [cited 28 Nov 2017]. Available: https://innovations.ahrq.gov/perspectives/sustaining-and-spreading-quality-improvement
- 43. Bonias MD, Bartram T, Leggat S. Benefits and barriers of clinical engagement: A qualitative study of perceptions of medical practitioners and human resource managers in three Victorian public hospitals. ANZAM. 2011. pp. 1–17. Available: http://www.anzam.org/wp-content/uploads/pdf-manager/459_ANZAM2011-118.PDF
- 44. Ehlers U. Quality literacy—Competences for quality development in education and e-learning. REM-Research Educ Media. 2011;3: 2195–218. ISSN:2037-0849
- 45. Department of Health Republic of South Africa. A policy on quality in health care for South Africa. Dep Heal Repub South Africa. Pretoria; 2007. Available: http://www.cohsasa.co.za/sites/cohsasa.co.za/files/publication_pdfs/qhc.pdf
- 46. Simpson LA, Fairbrother G. The role of policy in quality improvement. Acad Pediatr. 2013;13: S16–S20. pmid:24268079
- 47. Joshi MS, Hines SC. Getting the board on board: Engaging hospital boards in quality and patient safety. J Qual Patient Saf. 2006;32: 179–187. Available: http://www.jointcommissionjournal.com/article/S1553-7250(06)32023-5/pdf
- 48. Vaughn T, Koepke M, Kroch E, Lehrman W, Sinha S, Levey S. Engagement of leadership in quality improvement initiatives: Executive quality improvement survey results. J Patient Saf. 2006;2: 2–9. Available: https://www.researchgate.net/publication/232114090_Engagement_of_Leadership_in_Quality_Improvement_Initiatives_Executive_Quality_Improvement_Survey_Results
- 49. Versteeg MH, Laurant MGH, Franx GC, Jacobs AJ, Wensing MJP. Factors associated with the impact of quality improvement collaboratives in mental healthcare: an exploratory study. Implement Sci. 2012;7: 1. pmid:22230594
- 50. Geonnotti K, Taylor EF, Peikes D, Schottenfeld L, Burak H, McNellis R, et al. Engaging primary care practices in quality improvement: Strategies for practice facilitators. Rockville; 2015. 15-0015-EF
- 51. Williams DM. How do people sustain their enthusiasm for QI? In: Institute for Healthcare Improvement [Internet]. 2017 [cited 23 Nov 2017]. Available: http://www.ihi.org/education/IHIOpenSchool/resources/Pages/AudioandVideo/WilliamsSustainEnthusiasm.aspx
- 52. Pettigrew A, Ferlie E, McKee L. Shaping strategic change-the case of the nhs in the 1980s. Public Money Manag. 1992;12: 27–31.
- 53. Dixon-Woods M, McNicol S, Martin G. Ten challenges in improving quality in healthcare: lessons from the Health Foundation’s programme evaluations and relevant literature. BMJ Qual Saf. 2012;21: 876–884. pmid:22543475
- 54. White M, Wells JSG, Butterworth T. The impact of a large-scale quality improvement programme on work engagement: Preliminary results from a national cross-sectional-survey of the “Productive Ward.” Int J Nurs Stud. 2014;51: 1634–1643. pmid:24890896
- 55. Parand A, Benn J, Burnett S, Pinto A, Vincent C. Strategies for sustaining a quality improvement collaborative and its patient safety gains. Int J Qual Heal Care. 2012;24: 380–390. pmid:22669328
- 56. ASQ. Culture of Quality: Accelerating growth and performance in the enterprise. New York; Available: http://eaq.ee/sites/default/files/lisad/culture-of-quality.pdf
- 57. Scott T, Mannion R, Davies HTO, Marshall MN. Implementing culture change in health care: Theory and practice. Int J Qual Heal Care. 2003;15: 111–118. pmid:12705704
- 58. Gordon G. The Roles of leadership and ownership in building an effective quality culture. Qual High Educ. 2002;8: 97–106.
- 59. Srinivasan A, Kurey B. Creating a culture of quality. In: Harvard Businness Review [Internet]. 2014 [cited 21 Nov 2017]. Available: https://hbr.org/2014/04/creating-a-culture-of-quality
- 60. Scoville R, Little K, Rakover J, Luther K, Mate K. Sustaining Improvement. Cambridge, MA; 2016. Available: http://drspweb.ihi.org/resources/Pages/IHIWhitePapers/Sustaining-Improvement.aspx
- 61. O’Regan N, Sims M, Ghobadian A. High performance: ownership and decision‐making in SMEs. Manag Decis. 2005;43: 382–396. Available: http://www.emeraldinsight.com/doi/pdfplus/10.1108/00251740510589760
- 62. Utami Yeni Tri Tamtomo D, Sulaeman ES. Patient Characteristics, Financing Type, Accreditation Status, and Quality of Health Services at Community Health Center, Surakarta. J Heal Policy Manag. 2017;2: 79–90. Available: http://www.thejhpm.com/index.php?journal=thejhpm&page=article&op=view&path%5B%5D=27
- 63. Shaw C. External Assessment of Health Care. BMJ. 2001;322: 851–4. pmid:11290644
- 64. Shaw C. The External Assessment of Health Services. World Hosp Heal Serv Off J Int Hosp Fed. 2004;40: 24–27, 50, 51. Available: http://www.who.int/management/ExternalAssessmentofHealthServices.pdf
- 65. Scrivens E. Putting continuous quality improvement into accreditation: improving approaches to quality assessment. Qual Health Care. 1997;6: 212–8. pmid:10177038
- 66. Hamm MSHS. The fundamentals of accreditation. Washington, DC: American Society of Association Executives; 1997.
- 67. Berwick DM, James B, Coye MJ. Connections Between Quality Measurement and Improvement. Med Care. 2003;41: I30–I38. Available: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=12544814 pmid:12544814