Conceived and designed the experiments: TS BF ARL CAPEDP Study Group. Performed the experiments: TS. Analyzed the data: EL AE ESV EP ARL. Contributed reagents/materials/analysis tools: AG RD ST FT. Wrote the paper: TS EL TG ARL.
The authors have declared that no competing interests exist.
Implementation fidelity is a key issue in home-visiting programs as it determines a program’s effectiveness in accomplishing its original goals. This paper seeks to evaluate fidelity in a 27-month program addressing maternal and child health which took place in France between 2006 and 2011.
To evaluate implementation fidelity, home visit case notes were analyzed using thematic qualitative and computer-assisted linguistic analyses.
During the prenatal period, home visitors focused on the social components of the program. Visitors discussed the physical changes in pregnancy, and psychological and social environment issues. Discussing immigration, unstable employment and financial related issues, family relationships and dynamics and maternity services, while not expected, were found in case notes. Conversely, health during pregnancy, early child development and postpartum mood changes were not identified as topics within the prenatal case notes. During the postnatal period, most components of the intervention were addressed: home visitors observed the mother’s adaptation to the baby; routine themes such as psychological needs and medical-social networks were evaluated; information on the importance of social support and on adapting the home environment was given; home visitors counseled on parental authority, and addressed mothers’ self-esteem issues; finally, they helped to find child care, when necessary. Some themes were not addressed or partially addressed: health education, child development, home environment, mother’s education plans and personal routine, partner support and play with the child. Other themes were not expected, but found in the case notes: social issues, mother-family relationship, relation with services, couple issues, quality of maternal behavior and child’s language development.
In this program, home visitors experienced difficulties addressing some of the objectives because they gave precedence to the families“ urgent needs. This research stresses the importance of training home visitors to adapt the intervention to the social, psychological and health needs of families.
Home-visiting programs have become one of the most popular early childhood interventions. These programs serve more than 500,000 families in the United States
One of the main purposes of home-visiting programs is to reduce the impact of social stress on the mental health of vulnerable families
Although home-visiting programs share common features, visiting clients’ homes remains a method for delivering a service rather than a service in itself
Program implementation, i.e. applying a program protocol in practice, is currently an important yet very recent
In a qualitative study (tape recorded nurses’ case notes) of the challenges experienced by professionals working in the Nurse Family Partnership (NFP) program, Kitzman, Cole, Yoos, & Olds
In a longitudinal mixed methods study (families and home visitor case studies, focus groups, videotaping and interviews), Hebbeler & Gerlach-Downie
The following study seeks to evaluate the extent to which the manualised program guidelines were reflected in home visitor case notes in an early childhood intervention, the CAPEDP Project [Compétences Parentales et Attachement dans la Petite Enfance: Diminution des Risques Liés aux Troubles de Santé Mentale et Promotion de la Résilience] and to identify case note themes that did not figure in the program.
The CAPEDP Project took place in Paris, France, from 2006 to 2011. The project was developed to consolidate perinatal and early childhood mental health promotion services in Paris and its suburbs, by offering home visit support to families presenting demographic characteristics associated with a higher incidence of subsequent maternal postpartum depression and infant mental health problems: mothers had to present one or more of the following inclusion criteria to participate in the program: (1) having less than 12 years of schooling, (2) intending to raise the child without the father (3) being eligible for health care free of charge, due to lack of personal resources or income. The program aimed to reduce the incidence of maternal postpartum depression and infant mental health problems as well as to promote parenting skills, infant-mother attachment security and social and professional integration. A total of 440 pregnant, primiparous women under the age of 26 were recruited in maternity wards between 2006 and 2009. Median age of participants was 22 years; 28.3% of the sample were mothers intending to raise their child alone; 74.4% had less than 12 years education and 45.7% were eligible for free health care. 52.3% were born outside France
The visits were conducted by a team of nine psychologists. All were female and from 23 to 34 years of age when recruited. All home visitors received specific training in the CAPEDP service implementation protocol, which was backed up with a detailed training manual built around four periods in the baby’s life: prenatal period, 0 to 6 months, 6 to 15 months and 15 to 24 months. The program and its manual were largely based on the work of Weatherstone
To evaluate to what extent the priorities of the home visitors as reflected in the home visit case notes reflect the intended processes of the intervention, we qualitatively analyzed the home visitor case notes. All home visit case notes were written by the home visitors themselves. It allowed us to evaluate both the frequency of themes that were declared as discussed during the home visit as well as the home visitor’s subjective perception of the visit.
The present qualitative study constitutes the first stage in evaluating the CAPEDP trial. Results from this trial will be published in the coming years, and discussed in regards to the services that were actually delivered to the participating families.
In the routine stage of the research (inclusion started in June 2007 and ended in January 2008), we randomly selected 10 to 12 families from the case load of each of the nine home visitors to participate in the current study. A total of 105 families were randomly selected. For the duration of two years (until June 2009), home visitors were asked to indicate, after each visit, the duration of the visit, the place where the visit was conducted (at home, in a public place, at hospital…) and the people present during the visit for each of the selected 105 families. Home visitors were then asked to write a brief report on the current family situation, the topics discussed during the visit and the relationship with the family. The 105 families received a total of 2,457 home visits from 2006 to 2010 and the home visitors collected a total of 1,058 case notes from 2007 to 2009.
A total of 1,058 case notes were collected from 105 families, which represent 26.2% of the 4,034 home visits that took place during the 5-years program.
Each of these 105 families received an average of 23.4 home visits (1–63; SD = 14.5) from the 7th month of pregnancy until the end of the program. This average number of home visits represents 53.2% of the number of home visits that the CAPEDP project had programmed per family. For these 105 families, on average 10.1 case notes (1–48, SD = 10.9) were written from the 7th month of pregnancy until the end of the program.
Following the program intervention manual, case notes were divided into four chronological child age categories: prenatal, 0–6 months, 6–15 months and 15–24 months.
Prenatal | 0–6 months | 6–15 months | 15–24 months | TOTAL | |
N case notes | 289 | 369 | 262 | 138 |
|
Mean Length of the visit (minutes) | 70.9 | 69.6 | 71.2 | 65.1 |
|
% visits outside the residence | 18.3 | 16.5 | 8.0 | 7.2 |
|
% visits with the father | 12.8 | 15.2 | 11.1 | 13.0 |
|
% visits with another person | 18.3 | 18.4 | 19.8 | 15.2 |
|
The demographic characteristics of the 105 families for whom case notes were collected were compared to those of all participating families. No significant differences between these two groups were found in terms of maternal age, percentage of single mothers, history of immigration, maternal and paternal education level, income, unplanned pregnancy, perceived health, and maternal attachment.
Case notes were first classified into four child age categories and then randomly organized using Microsoft® Excel’s = RAND() function. The first 50 case notes from each of the four categories were extracted for thematic qualitative analysis. All 1,058 case notes from the 105 included families were used for the linguistic analysis.
The thematic analysis of the case notes was conducted using principles of grounded theory
A computer-assisted linguistic analysis using ALCESTE software [Analyse des Lexèmes Coocurrents dans un Ensemble de Segments de Textes, or Analysis of Co-occurring Lexemes in a Set of Text Segments
Categories from the thematic analysis and the computer-assisted analysis were then compared to the CAPEDP manual for each intervention period. The purpose of these comparisons was to investigate to what extent the themes identified in case notes corresponded to instructions given by the project manual. For each child age period, we identified themes present in the case notes but not in the manual, and the themes absent from the case notes but targeted explicitly in the manual for that period. This information illustrates the extent to which the CAPEDP intervention was faithful to its initial model and guidelines.
Category | Program Manual Objectives | Thematicanalysis | ALCESTEanalysis | Additional (+) oromitted (−) thematic | |
Observe | Material needs | X | X | X | |
Discuss | Physical and psychological changes during Pregnancy | X | X | X | |
Expectations of the baby | X | X | X | ||
Presence of social support | X | X | – | ||
Immigration, financial and employment issues | – | X | X | + | |
Relationship with family | – | X | – | + | |
Relations with maternity services | – | X | – | + | |
Inform | Delivery | X | X | X | |
Postpartum mood changes | X | – | – | − | |
Support from partner | X | X | – | ||
Importance of breastfeeding | X | X | – | ||
Fetal development | X | X | X | ||
First developmental stages | X | – | – | − | |
Counsel | Health during pregnancy | X | – | – | − |
Do | Negotiate the objectives of the home visit | X | X | X |
Category | Program manual objectives | Thematicanalysis | ALCESTEanalysis | Additional (+) oromitted (−) thematic | |
Observe | Psychological needs | X | X | X | |
Knowledge of child needs | X | X | – | ||
Medical and Social network | X | X | – | ||
Quality of parenting | – | X | – | + | |
Discuss | Couple needs | X | X | – | |
Job/Education-related needs | X | – | X | ||
Inform | Health education | X | – | – | − |
Feeding and sleep | X | X | X | ||
Importance of partner support | X | X | – | ||
Adaptation of home environment | X | – | – | − | |
Baby’s early development | X | – | X | ||
Counsel | Parent-child interactions | X | X | X | |
Promote social support | X | X | – | ||
Administrative needs | – | X | X | + | |
Self-esteem | – | X | – | + | |
Do | Negotiate the objectives of the home visit | X | X | X |
Category | Program manual objectives | Thematicanalysis | ALCESTEanalysis | Additional (+) or omitted (-) thematic | |
Observe | Psychological needs | X | X | – | |
Mother and Child Health Needs | X | X | X | ||
Attachment quality | X | X | X | ||
Medical and Social network | X | X | X | ||
Discuss | Couple needs | X | X | – | |
Educational plans | X | – | – | − | |
Inform | Importance of partner support | X | – | – | − |
Adaptation of home environment | X | – | X | ||
Developmental stages | X | X | X | ||
Counsel | Promote self-esteem | X | X | – | |
Elaboration of personal goals | X | X | X | ||
Parents-child interactions | X | X | X | ||
How to set limits | X | X | X | ||
Promote social support | X | X | – | ||
Administrative problems | – | X | X | + | |
Do | Help to find child care | X | X | – | |
Negotiate the objectives of the home visit | X | X | X |
Category | Program manual objectives | Thematic analysis | ALCESTE analysis | Additional (+) or omitted (−) thematic | |
Observe | Health needs | X | X | – | |
Mental health needs | X | X | X | ||
Medical Social network | X | X | X | ||
Language development concerns | – | X | – | + | |
Discuss | Representations of parental authority | X | X | X | |
Importance of social support | X | X | – | ||
Feedback on intervention | – | X | X | + | |
Social and professional situation | – | X | X | + | |
2nd pregnancy/2nd child health | – | X | – | + | |
Problems with romantic partner/Child’s Father | – | X | – | + | |
Inform | Importance of play | X | – | – | − |
Child development and autonomy | X | X | X | ||
Counsel | Organizing the schedule = the mother can take some time off for her/her couple | X | – | – | − |
Set limits for the child | X | X | X | ||
Parent child interactions | X | X | X | ||
Do | Negotiate the objectives of the home visit | X | X | X | |
Network for social, health and mental health services | X | X | X |
The main thematics that were to be addressed during this first period were (a) to negotiate the objectives of CAPEDP for each family, (b) to counsel the mother about health during pregnancy and to inform her about the main changes in post partum period and (c) to discuss the changes related to her pregnancy, her expectations regarding her future baby and the possibility to appeal to her social support network. Finally, the home visitors were asked to observe the material needs of the family and, if the situation was precarious, the visitor was asked to help the family to find financial or social resources before the delivery.
Three intervention manual themes were totally absent from the prenatal case notes: (1) Health during pregnancy; (2) Early child development; (3) Postpartum mood changes. Three themes that emerged from the case notes but that did not figure in the program manual were:
In the 0–6 months period, the home visitors had to observe the mother’s adaptation to the baby, her psychological needs. They were asked to investigate the job/education-related needs, as well as couple needs. Information was to be given relatively to health education, to the baby’s early development, to the importance of social support and to the main adaptation of the home environment. Finally, home visitors were asked to counsel parents’ on their first interactions with their children.
The 6–15 months period was principally dedicated to support the development of mother-child attachment relationship. Routine thematics were to be evaluated (psychological needs, health needs, medicosocial network), discussed (couple needs and educational plans) or supported by the home visitor information (importance of partner support, adaptation of home environment, child’s developmental stages). Counselling targeted mother self-esteem and elaboration of personal goals as well as developing strategies to set limits to her child. Home visitors actively helped to find child care, if necessary.
The last period, from the baby’s 16th to 24th month focused on parent’s empowerment and autonomy. Besides thematics that were routinely addressed (observing health and mental health needs and social network, discussing family’s social support network, informing on child development), home visitors were to discuss representations of parental authority, to inform on the importance to play with the child and to counsel on the way to set limits. They were also asked to promote the family inclusion in medical and social services.
Four further themes appearing in all four child age categories were identified by both thematic and computerized analyses:
Due to the low socio-economic status and changing living conditions of many CAPEDP participants, visitors expressed difficulties organizing home visits and intervening according to the CAPEDP curriculum. The frequency of visits and the structure of each visit were disrupted by the social situation of the families.
Home visitors frequently expressed in the case notes their concern about the parenting they observed.
The domestic setting of home visits impacts logistical aspects of a professional relationship with families. While home visits give more control to the family, they also create problems for the home visitor who is striving to achieve distinct objectives. Most of these problems concerned home visitors having difficulty maintaining the participant’s attention or discussing the intervention objectives. They also concerned unpredictable living conditions.
Home visitors frequently expressed that families were unreliable in maintaining regular visits. Home visitors reported difficulties in scheduling visits and in accessing participants, even when mothers confirmed visits. They expressed feelings of discouragement and irritation.
The current study presented a method for assessing fidelity in a French home-visiting program targeting families with low socio economic status.
To evaluate the discrepancies between what the intervention intended to offer and the services the participating families effectively received, we conducted a qualitative analysis of 1,058 case notes from 105 families, which had been written by home visitors. We compared a computer-assisted textual analysis to a thematic analysis performed by researchers who were unfamiliar with the CAPEDP training manual. Then we created tables to compare the contents of (a) the program manual, (b) the thematic analysis and (c) the computer-assisted analysis.
We learned from this study that complete fidelity to the program’s curriculum could not be achieved with this study’s sample of high-risk families. Following Kitzman, Cole, Yoos, & Olds
We chose to present those discrepancies in two categories (see
Omitted thematics: not addressed while expected | Additional thematics: addressed while not expected |
Prevent postpartum depression by observing mood changes | Discuss social, cultural and administrative issues |
Prenatal information on early child development | Discuss the mother’s relationship with her family |
Health education | Discuss relations with other services |
Information on the adaptation of the home environment to the child | Discuss the problems with the partner |
Discuss mother’s educational plans | Discuss the issues related to the 2nd child |
Inform the family on the importance of partner support | Observe the quality of maternal behaviors |
Inform about the importance of playing with the child | Observe problems in the development of the child’s language |
Help the mother to organize her personal schedule | Feedback on intervention |
The objectives of the intervention that were not addressed during the actual home visits despite expectations outlined in the home visitor training manual.
The aspects of the intervention that home visitors applied despite their absence from the program’s training manual
The intervention manual drew upon the experience of health promotion programs which were offered to less vulnerable populations. The CAPEDP program differed from many other health promotion programs in that participants were facing very challenging social situations. Almost half of the recruited families were eligible for health services financed entirely by the French government. The focus on the mental health of mothers and their new-born children as well as on their relationship were two additional aspects that distinguished the CAPEDP program from other home visit programs. Hence, the intervention was slightly modified by the home visitors’ educational training in psychology. This affected the fidelity to the curriculum in two ways:
Although the training manual urged home visitors to discuss and counsel participants on health-related behaviors during pregnancy, according to analysis of the case notes, home visitors did not address this issue. The absence of health education from case notes may be explained in three ways: First, we speculate that home visitors and families prioritized social and material issues over health education due to the urgency of social and material issues. Secondly, participants received a mean of 3 prenatal visits (SD = 2.0; 0–11, i.e. 55,3% of the intended number of visits). These 3 prenatal visits most likely sufficed in negotiating and shaping the objectives of the intervention and the relationship with the family, but were likely not substantial enough to negotiate and implement a health education intervention. Lastly, the home visitors training as psychologists may have led them to focus on mental health support rather than on health education to families.
Given the many social adversities confronted by CAPEDP participants, social-related topics were the focus of most home visit conversations. While social-related topics were expected to be a secondary objective of the intervention, it became the intervention’s main focus for a significant number of the families. As a consequence, health, relational and educational issues became secondary themes in the intervention.
Although most home-visiting programs employed nurses as home visitors, the CAPEDP team, seeking to specifically address attachment and the mother-child relationship, consisted of 9 clinical and developmental psychologists. Despite the lack of health education topics in case notes, the home visitors focused particularly on building an alliance with the families, and particularly the mothers. Consequently the themes “discussing the home visitor-mother relationship” and “home visit feedback from the mother” emerged frequently across case notes from each of the four intervention periods. This finding can be assimilated to what Kitzman et al. identified in the NFP Memphis trial as a barrier to program implementation (gaining and maintaining access to the family)
The home visitors provided frequent feedback concerning:
Preoccupations about the quality of maternal parenting
Problems in child language development
Our thematic analysis revealed that these two categories appeared within the “observe” category. This means that the home visitor provided feedback to the case note reader but did not directly address these subjects with the family.
The lack of material and social resources in participating CAPEDP families was a major focus of the intervention. Home visitors thus often discussed a family’s social and administrative problems first, before focusing on the health promotion/prevention contents of the intervention. These findings are consistent with results from Darius Tandon et al.
Immigration to France and assimilation to French culture were also discussed, often alongside the issue of social isolation. Lastly, discussing the family’s relationship with other services, such as the Maternal and Infant Protection Agency services (
The mother’s relationship with her partner or the child’s father were recurrent themes in the intervention (particularly when the relationship was conflictual). Romantic relationships and difficulties in the relationship with the family were also discussed. Finally, the relationship between the mother and additional children she may have had within the time of the intervention, constituted an additional topic for home visitors.
The CAPEDP project intended to bring out the expertise of psychologists within a home visitation intervention. This decision to hire psychologists to assume the role of home visitors impacts how mental health professionals can conceptualize traditional psychological interventions as well as the paradigm for home visit interventions.
With regards to conventional psychological interventions in France, CAPEDP, by using a home visitation protocol, allowed professionals to develop their relational skills within an ecological context. It enabled new psychological practices in the field of prevention to be sketched out.
On the other hand, the results from this qualitative study question the idea of having a homogeneous team of home visitors in terms of their backgrounds. As Darius Tandon and colleagues stressed, home visitors trained to be health care providers can be unsettled by the social situation of the families
Results from the CAPEDP trial and from this qualitative study will help mental health professionals understand mechanisms underlying a home visit intervention that was led by psychologists and to judge the extent to which psychologists can impact the social and health conditions of vulnerable families through home visits.
We presented a qualitative evaluation of a home-visiting program’s adherence to its original protocol in Paris, France from 2006 to 2011. The use of two qualitative methods (thematic/textual) was developed to analyze 1,058 home visit case notes from 105 families, written by the home visitors. We learned from this study that the home visitors partially followed the intervention’s original curriculum. Several of the program’s objectives were not addressed, mainly because of the urgent needs of the participating families that took precedence over certain program objectives. While confronted with practical issues, it is necessary to think about the malleability of the structure of such a home-visiting
Home visitation programs should allow for enough flexibility in the intervention for home visitors to adapt their visits to the needs of families. Adapting the intervention can be especially helpful when families live in impoverished or otherwise aversive social conditions and have urgent needs related to these circumstances. Future home visitation programs should use a strategy in which home visitors partially negotiate the intervention with the family. They should also adapt their curriculum to the reality and the social, psychological and health needs of the targeted population to ensure that the intervention has an impact on the individual and the community. In light of these findings, the professional background of home visitors should be considered carefully and alongside the targeted population’s needs. The training of professionals’ should be ongoing, provided prior to beginning the intervention, and at designated time points throughout the intervention. Home visitors in this study stated that it was difficult to address social issues presented by participating families. This feedback highlights the importance of continuous training, in which home visitors would be trained to address challenges as they arise from the intervention. In the current study, psychologists were hired to serve as home visitors. However the benefits of interventions using mixed teams formed with social workers, psychologists and nurses, remains unknown and should be the object of future research.
The authors would like to thank the 440 families who participated in the study, the members of the home-visiting team and the research assistants, without whom this project would have been impossible: Joan Augier, Amel Bouchouchi, Anna Dufour, Cécile Glaude, Audrey Hauchecorne, Gaëlle Hoisnard, Virginie Hok, Alexandra Jouve, Anne Legge, Céline Ménard, Marion Milliex, Alice Tabareau, Francine Messeguem. We also thank the team of the Binet Children Community Mental Health Center, Sebastien Favriel and the research team of l’EPS Maison Blanche for technical support, Estelle Marcault for the logistical support and research implementation, Véronique Laniesse and Alexandra Avonde for the administrative support, Cécile Jourdain, Pierre Arwidson, Béatrice Lamboy and Gérard Guillemot for help with the research administration, Nathalie Fontaine, George Tarabulsy and Michel Boivin, for assistance with developing the research and intervention instruments, and the members of the supervision team: Laure Angladette, Drina Candilis, Judith Fine, Alain Haddad, Joana Matos, Anne-Sophie Mintz, Marie-Odile Pérouse de Montclos, Diane Purper-Ouakil, Françoise Soupre, Susana Tereno, Bertrand Welniarz and Jaqueline Wendland.
The CAPEDP Study Group:
Elie Azria, Emmanuel Barranger, Jean-Louis Bénifla, Bruno Carbonne, Marc Dommergues, Romain Dugravier, Bruno Falissard, Tim Greacen, Antoine Guédeney, Nicole Guédeney, Alain Haddad, Dominique Luton, ?Dominique Mahieu-Caputo, Laurent Mandelbrot, Jean-François Oury, Dominique Pathier, Diane Purper-Ouakil, Thomas Saïas, Susana Tereno, Richard E. Tremblay, Florence Tubach, Serge Uzan and Bertrand Welniarz.