Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Evaluating the Feasibility and Participants’ Representativeness of an Online Nationwide Surveillance System for Influenza in France

  • Marion Debin,

    Affiliations: INSERM, U707, Paris, France, UPMC Univ Paris 06, Faculté de Médecine Pierre et Marie Curie, UMR S 707, Paris, France

  • Clément Turbelin,

    Affiliations: INSERM, U707, Paris, France, UPMC Univ Paris 06, Faculté de Médecine Pierre et Marie Curie, UMR S 707, Paris, France

  • Thierry Blanchon,

    Affiliations: INSERM, U707, Paris, France, UPMC Univ Paris 06, Faculté de Médecine Pierre et Marie Curie, UMR S 707, Paris, France

  • Isabelle Bonmarin,

    Affiliation: Department of Infectious Diseases, Institut de Veille Sanitaire (InVS), St Maurice, France

  • Alessandra Falchi,

    Affiliations: INSERM, U707, Paris, France, UPMC Univ Paris 06, Faculté de Médecine Pierre et Marie Curie, UMR S 707, Paris, France

  • Thomas Hanslik,

    Affiliations: INSERM, U707, Paris, France, UPMC Univ Paris 06, Faculté de Médecine Pierre et Marie Curie, UMR S 707, Paris, France, Assistance Publique Hopitaux de Paris, Service de Medecine Interne, Hopital Ambroise Pare, Boulogne Billancourt, France

  • Daniel Levy-Bruhl,

    Affiliation: Department of Infectious Diseases, Institut de Veille Sanitaire (InVS), St Maurice, France

  • Chiara Poletto,

    Affiliations: INSERM, U707, Paris, France, UPMC Univ Paris 06, Faculté de Médecine Pierre et Marie Curie, UMR S 707, Paris, France, Institute for Scientific Interchange (ISI), Torino, Italy

  • Vittoria Colizza

    Affiliations: INSERM, U707, Paris, France, UPMC Univ Paris 06, Faculté de Médecine Pierre et Marie Curie, UMR S 707, Paris, France, Institute for Scientific Interchange (ISI), Torino, Italy

Evaluating the Feasibility and Participants’ Representativeness of an Online Nationwide Surveillance System for Influenza in France

  • Marion Debin, 
  • Clément Turbelin, 
  • Thierry Blanchon, 
  • Isabelle Bonmarin, 
  • Alessandra Falchi, 
  • Thomas Hanslik, 
  • Daniel Levy-Bruhl, 
  • Chiara Poletto, 
  • Vittoria Colizza


The increasing Internet coverage and the widespread use of digital devices offer the possibility to develop new digital surveillance systems potentially capable to provide important aid to epidemiological and public health monitoring and research. In France, a new nationwide surveillance system for influenza-like illness,, was introduced since the 2011/2012 season based on an online participatory mechanism and open to the general population. We evaluate the recruitment and participation of users to the first pilot season with respect to similar efforts in Europe to assess the feasibility of establishing a participative network of surveillance in France. We further investigate the representativeness of the population along a set of indicators on geographical, demographic, socio-economic and health aspects. Participation was widespread in the country and with rates comparable to other European countries with partnered projects running since a longer time. It was not representative of the general population in terms of age and gender, however all age classes were represented, including the older classes (65+ years old), generally less familiar with the digital world, but considered at high risk for influenza complications. Once adjusted on demographic indicators, the population is found to be more frequently employed, with a higher education level and vaccination rate with respect to the general population. A similar propensity to commute for work to different regions was observed, and no significant difference was found for asthma and diabetes. Results show the feasibility of the system, provide indications to inform adjusted epidemic analyses, and highlight the presence of specific population groups that need to be addressed by targeted communication strategies to achieve a higher representativeness in the following seasons.


Influenza surveillance systems provide essential information to describe the spatiotemporal diffusion of the disease, monitor the incidence through the season and across affected groups, and for the activation of specific response plans and the evaluation of their effectiveness [1], [2]. These systems are traditionally based on national networks of physicians who report cases of patients with influenza-like illness (ILI) and collect samples from a subset of these patients for virological tests, and also on hospital and emergency services [3].

From paper records to electronic systems, developments in modern communication have changed the monitoring systems in the last 30 years, speeding up the transmission of information and facilitating the storage of the data and its access [4], [5], [6]. Additional substantial changes appear nowadays potentially feasible if we consider that digital progress has also changed the way we create and exchange information, shifting the point of view from infrastructures to their users. New technologies have opened the way to real-time, collaborative and mobile mechanisms, allowing a massive amount of digital data to be created, stored, analyzed, modeled, and accessible/shareable on the Web [7], [8]. With much of this data provided directly from the user through digital devices and thanks to the continuously increasing widespread coverage of the Internet access, such digital behaviors also offer the possibility to develop a digital counterpart of the traditional epidemiology [9], [10]. This applies to disease surveillance systems as well, with the development of innovative approaches that are able to detect emerging outbreaks or infer the epidemic activity levels from web search records [11], [12], online news [13], [14], or tweets exchanged by users [15].

Following previous experiences in other European countries [16], our approach leveraged on the social and collaborative aspects of the Web for the setup of an online surveillance system for ILI aimed at gathering data on a large spectrum of information. Not only illness incidence in the community, but also information on health-care usage, contacts with specific classes of individuals (e.g. children, patients, and others), and infection-related attitudes or change of behaviors is collected. To complete the picture on the impact of an influenza epidemic in the general population, we rely on Web-based questionnaires as an appealing alternative option to the traditional methods of data collection based on offline surveys with paper questionnaires and interviews, often requiring high costs in large study populations [9].

The system is called and was launched in January 2012. Participants are volunteers recruited in the general population who register to the project providing general background information and who are then invited to self-report on their ILI symptoms on a weekly basis. While establishing efficient recruitment strategies poses a series of challenges in terms of engagement, it potentially enables to largely increase the coverage in the population. Participatory surveillance has also the great advantage of making it possible to profile the population joining the project, in order to assess the representativeness of the surveillance system and determine risk groups for ILI. Such population may also include individuals who do not seek for health care treatment following infection (an estimated 57% for secondary cases in France [17]). In addition, profiling the population with a large set of indicators is generally not possible in other digital approaches for surveillance, since they rely on the unsupervised collection of information [11], [13], [14], [15], and it may be difficult also in the case of sentinel influenza surveillance because of its protocols and the limited set of information that are routinely collected (age, gender, geography) [18].

Focusing on the pilot season, this work aims to (i) assess the feasibility of the collaborative influenza surveillance system in France, and to (ii) evaluate the representativeness of the population using a set of indicators on geographical, demographic, mobility, socio-economic and health aspects. The goal is to provide useful information for the validation of the study, for the future recruitment of participants in the following seasons, and for identifying possible sample biases to be taken into account into epidemic analyses.


Study Design is part of a European multicenter study (Influenzanet [16]) for ILI surveillance in the general population through online systems. Such systems were first implemented in Belgium and The Netherlands in 2003 [19], [20], followed by Portugal in 2005 [21], [22], Italy in 2008 [23], the UK in 2009 [24], [25]. France and Sweden joined the system in the 2011–2012 season [26], followed by Spain in the 2012–2013 season [27]. is coordinated by the Sentinel Network team of the National Institute of Health and Medical Research [28] and University Pierre et Marie Curie [29] (also responsible in France for the disease surveillance network of general practitioners (GPs) [30]), and by the National Institute for Public Health Surveillance [31]. Launched on the 25th of January 2012, it consists of a website used to register participants, provide access to participants to their secure and private areas where to fill in the surveys, communicate the results of the study and report on related activities.

During its first season, the project was open to the recruitment of individuals aged ≥18 and resident in mainland France (overseas territories excluded). Recruitment occurred through press releases of the supporting institutions, communications on the Internet, radio, TV, newspapers, and through emails and word of mouth (more information can be found on the webpage ‘ in the news’ [32]). A weekly report on the results updated on influenza surveillance and on the participating population was published every Wednesday within the official Weekly Bulletin of the Sentinel Network of GP Surveillance [33], and the Weekly Bulletin of the National Institute for Public Health Surveillance [31].

Participation was voluntary and anonymous. Registration took place on the website where participants could create a user account. A valid email address was required to confirm the registration and avoid unsupervised registration attempts. The email address was also used to maintain a communication channel with each participant as a motivation purpose during the season, as explained in the Data collection subsection. The website contains several pages that are publicly accessible dedicated to the presentation and aims of the project, instructions on how to participate, general information about influenza virus and influenza epidemics and pandemics, and the results of the study provided in real time.

Privacy and Ethical Approval

This study was conducted in agreement with French regulations on privacy and data collection and treatment and was approved by the Comité consultatif sur le traitement de l’information en matière de recherche (CCTIRS, Advisory committee on information processing for research, authorization 11.565) and by the Commission Nationale de l’Informatique et des Libertés (CNIL, French Data Protection Authority, authorization DR-2012–024).

Data Collection

Upon registration, the user was asked to complete an intake survey covering demographic factors (age, gender), geographical factors (location of home and work/school expressed at the municipality level), socio-economic factors (household size and composition, occupation, educational level, number of daily contacts with groups of patients, children or elderly, daily transportation means), and health-related factors (height and weight, diet, vaccination status, pregnancy status, smoking habits, major risk conditions, and others). At-risk population was identified as the population eligible for a vaccination voucher, based on the vaccination system for this population group in France (see subsection National Data Sources for more details). The full list of variables for which data was collected in the intake survey is reported in Tables 1 and 2, and in Table S1 in File S1. The survey is available both in English and in the original French version in File S1.

Table 1. Socio-demographic indicators gathered in the intake survey and comparison with national statistics.

Table 2. Health-related indicators gathered in the intake survey and comparison with national statistics.

Participants were also allowed to register and create accounts for other individuals on their behalf, given consent, thus enabling, for instance, to grant access to the project to individuals less comfortable with Internet tools (e.g. in the older age classes). The data was collected only once at the enrollment, however the participant could access her/his intake survey anytime and modify it following any update (e.g. change of residence, of vaccination or pregnancy status). In presence of multiple instances of intake surveys, we considered the last completed.

Each week participants were asked to fill in a weekly symptoms survey, independently of possible ILI symptoms. An automatic email message was sent weekly in the form of a newsletter that contained a reminder to fill in the symptoms survey as well. The newsletter was also meant to keep a high motivation in the participation to the study by providing preliminary results on the analysis of the symptoms reported the week before, a general description of the influenza epidemic situation in France and in Europe, and an entertaining ‘Did you know?’ column about various flu curiosities and historical facts.

The description of the weekly symptoms survey will be reported elsewhere, as here we focus on the representativeness of the project participants and will analyze exclusively the data collected through the intake survey. A final feedback survey was conducted during spring 2012, after the closure of the data collection campaign, to gather information on the perception of users regarding their participation to the study.

Inclusion Criteria

All questionnaires filled in between the 25th of January 2012, launch date of the project, and the 29th of April 2012, closure date of the first online surveillance season, were considered in the analyses, covering a total of 14 weeks. To prevent that a short-term participant may bias our results [34], we include in our sample only those participants who completed an intake questionnaire and at least three symptoms surveys during the season [22] – such individuals being referred to, hereafter, as active participants (in the following we will equally refer to population or active population to indicate the population of active participants). No additional specific constraint was required on the frequency of compiling the symptoms survey during the 14 weeks of surveillance of the project. We tested these inclusion criteria and performed a sensitivity analysis of our results by imposing that participants would fill at least 3 symptoms surveys, and at least one symptoms survey every calendar month since they joined the study. This prevents us to consider participants who may have filled in the symptoms surveys in three consecutive weeks and then dropped out. The representativeness of participants who did not specify their age and/or gender in their intake questionnaire cannot be assessed, so we excluded those participants from the analyses.

National Data Sources

The study required gathering data from a number of socio-demographic datasets and health datasets that are described hereafter.

The demographic and socioeconomic data of the French population were provided by the French National Institute of Statistics and Economic Studies (INSEE). In absence of data for the year of 2012, we relied on the most recent available sources: age-gender composition updated to January 1, 2010; household size and composition as of 2009; employment data updated to December 31, 2010; education data as of 2010; city size as of 2009. All data concern only mainland France [35].

In the following we indicate as French adult population or general population, the population of mainland France being 18 years old or older.

In France, the at-risk population who is recommended for vaccination by public health authorities includes the following categories: individuals aged over 65, pregnant women, obese individuals, and individuals with chronic health conditions (diabetes, pulmonary, kidney or heart disease, and others). Vaccination for these classes of individuals is provided by the national health insurance. More in detail, the system relies on mailing vaccination vouchers to individuals who comply with the above definitions (except pregnant women and obese individuals who are not identified by national health insurance). Vouchers can be used in pharmacies and medical practices to receive the influenza vaccination for free. The used vouchers are then mailed back from pharmacies and medical practices to the national health insurance main office for reimbursement purposes. The percentage of individuals who have been recommended for vaccination in French population was estimated through the number of vouchers sent by the national health insurance in the fall 2011, within the French population registered to the national health insurance (data of the CNAM [36]). Taking into account that this population represented 85% of the overall French population [37] and assuming that the percentage of at-risk population is similar in the population of the CNAM and the overall French population we can calculate the percentage of the French population below 65 years of age and registered to the national health insurance who received a voucher, thus representing the estimated at-risk population in France aged <65 years.

The data on influenza vaccination coverage in the French general population was provided by two different sources: the Caisse nationale d’assurance maladie des travailleurs salariés (CNAMTS, French National Health Insurance Fund for Employees) for 2010–2011 for several age groups [38] and 2011–2012 for age groups over 65 years old in the general population, and for at risk population in the <65 years classes [36]; the Groupe d’Expertise et d’Information sur la Grippe (GEIG, French Group of Expertise and Information about Influenza) for 2011–2012 for age groups under 65 years old (data obtained through a phone-based post-epidemic survey conducted yearly by a private independent organization from a representative sample of the French population over 15 years).

Data on body mass index (BMI) were provided by the ObEpi study for 2012 [39]. Data on diabetes was obtained from the CNAMTS through the Ecosante database for 2011 in mainland France [40]. Incidence rates were calculated with 2010 demographic data obtained from INSEE. Data on smoking was provided by the Institut national de prévention et d’éducation pour la santé (Inpes, French national institute for prevention and health education) [41]. Data of asthma prevalence were issued from a previous study [42], using the figure corresponding to the definition “individuals who had an asthma crisis in the last 12 months and/or who currently have a treatment for asthma”, as we consider that the question of the intake questionnaire identified individuals who currently take a treatment, and individuals who take a treatment when they have an asthma crisis.

Commuting data across the 22 regions of mainland France was obtained from the data set provided by INSEE for 2009 [43], the most recent source for commuting. The original data provides the number of daily commuters from a given municipality of origin (corresponding to the location of residence) to a given municipality of destination (corresponding to the location of school or workplace). All individuals aged more than 3 years old are recorded in the data set. The original data is then mapped to the region level in order to obtain the number of commuters who live in a given region and daily commute for educational or professional reasons to another region, for any pair of regions in France.

Data Analysis

The representativeness of the population was assessed through the comparison of its characteristics with the ones of the general French population, considered as the reference population, with χ2-test for non-continuous sociodemographic variables, and Student’s t-test for mean comparisons. All comparisons used bilateral tests and a 5% cutoff point. Statistical analyses were performed using the R software version 2.13.2 (R Development Core Team, R Foundation for Statistical Computing, Vienna, Austria,

Apart geographical and demographic characteristics, a standardized analysis on age group and gender [24] was done on the other variables, considering the following age categories : 18–29, 30–39, 40–49, 50–59, 60–69, ≥70.

The commuting data, extracted from the census and from the population, were analyzed in terms of networks of nodes and links [44], [45], where nodes represent the regions and a directed link from a region of origin O to a region of destination D is drawn if there exist at least one individual resident in O and daily commuting to D. A weight was also assigned to each link to indicate the number of commuters on that connection. The census datum was adjusted for possible discrepancies in the geographic distributions of population with respect to the national data. Then, the most relevant backbone of the commuting network was extracted through the disparity filter algorithm [46] to be compared with the network. The details of the analysis are reported in File S1.

We also calculated the probability to observe the directed links in the commuting, given the probability that an individual living in region O would commute to region D and the sample of the participants in region O (see File S1). These probabilities can then be directly compared to the occurrence of the commuting links in the sample.


Descriptive Analysis

The number of participants reached about 3,500 in 5 weeks and continued to increase throughout the season (see Figure S1 in File S1). Between the 25th January and the 29th April 2012, 3,936 persons filled at least one intake survey, and a total of 29,843 symptoms surveys were completed, for an average of 2,130 symptoms surveys per week.

Based on the inclusion criteria, we selected 3,055 active participants. 11 persons did not mention their age and/or gender and were excluded from the analyses. 3,044 individuals were therefore considered in the following analyses, corresponding to 77.3% of the entire sample. Among them, 2827 (92.9%) had a single membership account, 190 (6.2%) belonged to a household (i.e. multiple) account with 2 active participants, 27 (0.9%) with 3 active participants. Active participants filled in between 3 and 28 symptoms surveys, with a median number per active participant equal to 10 (Q1 = 7, Q3 = 11).

Users’ satisfaction concerning their participation to the project was in general very high. Only 0.9% of the active participants who completed the feedback survey during summer 2012 (for a total of more than 1,900 respondents) affirmed that they were not willing to participate to the following season. Moreover, only 1.4% of them declared that the weekly questionnaire takes too long, whereas 87.7% of them found the website easy to access and did not find any difficulty in performing the task. General interest to the project beyond the data contribution was also observed, with the majority of responders declaring that they frequently read the newsletter sent every week, and 40.9% of them visiting the website for further information, besides the access to the surveys.

Geographical and Demographic Characteristics

Our study reached a coverage of 6.2 individuals per 100,000 inhabitants and participation was widespread in the country, with active participants coming from all 96 mainland French departments, for a total of 1,783 distinct municipalities participating to the project (see Figure S2 in File S1). The geographic distribution of the participant population in French regions is statistically different from the distribution of French population (), with, e.g., the largest overrepresentation in the Corse region, and a high underrepresentation in the regions of Nord-Pas-de-Calais and Picardie. We note that, with the exception of the Ile-de-France, under-representation is generally localized in northern regions (see Figure 1). The majority of the regions (12 out of 22, 54%) has however a relative difference between regional distribution of population and French population contained in the range [−15%;+15%].

Figure 1. Distribution of the active participants per region.

a) Comparison with the distribution of the over-18 French population per region. b) Relative difference (RD) between regional distribution of population and French population.

Inhabitants of big cities are more likely to participate to than those of smaller cities; 55.3% (95% C.I. = 53.6%–57.1%) of active participants coming from cities with population >10,000 inhabitants, against 49.6% obtained for the French population ().

The population was on average older than the French population, with a mean age for active participants being equal to 52.1 years old (median equal to 53.5; 95% C.I. = 23.4–80.7) against 48.6 years old in France (once individuals under 18 are not taken into account) (). A larger proportion of females (65.1%, 95% CI = 63.5%–66.8%) participated to the study, with respect to 52.3%, the baseline value (). All age classes were represented in the sample, however a significant difference between the repartition in age and gender of active participants and French population is observed (see Table 3) (). Individuals aged <30 were found to be underrepresented in both female and male populations (as well as the 70+ class in the female population), whereas the opposite occurs for the age class [60–69]. A large over-representation was also observed in the female [50][59] class, contrary to the results found for male individuals in the same age group (Figure 2).

Figure 2. Distribution by age and gender of the active participants.

Comparison of the distribution by age class of the female (panel a) and male (panel b) population between the sample and the over-18 French population.

Table 3. Overall comparison of population with French population, for different age and gender groups (all differences are significant,).

Mobility Features

Among the active participants, 1,078 individuals (61.6% of the studying/working population) provided information on their school/work locations, specifying the corresponding municipality codes. Once the data was analyzed at the regional level, most of the participants declared to commute within the administrative region of their residence. The ratio of the cross-region commuters over the within-region commuters is 5.4% [0%,19%], very close to the corresponding value for census data (3.7% [1%,10%]). commuting network is composed of 22 nodes as in the census commuting network, since all regions are covered by the study. Only 28 links over the 462 of the census network are however represented in the participant population. Four regions (Corse, Limousin, Poitou-Charentes, Bourgogne) are disconnected from the rest of the network, as they do not have either incoming or outgoing commuters. If we compute the relative difference of the distribution of incoming commuters (or outgoing commuters) of the sample vs. census of a given region, we find quite a large disagreement between the two datasets, with only one region (Provence-Alpes-Côte-d’Azur) and two regions (Haute-Normandie and Centre) lying in the interval [−15%,15%] for in-commuters and out-commuters, respectively (see Figure S4 in File S1 for more details).

Going beyond the comparison between absence or presence of the commuting links, we tested whether the sample well reproduces the largest flows of mobility within the country, given the large variability of commuting fluxes of the French census commuting network (see Figure S3 in File S1) [47]. Half of the census commuting links having a probability larger than 60% are represented in the population (Figure 3), despite that they are very few (less than 1% of the total). A smaller representation (less than 30% of the links) is obtained for the links having probabilities of occurrence in the range [20], [60]%, whereas links with probabilities smaller than 10% are almost not found in the commuting network.

Figure 3. Percentage of commuting links recovered by the population.

The percentages are presented as a function of the probability of occurrence of the link. The bar plot shows the percentage of links of the census network represented in the sample per probability interval. The probability of the link is a measure of the likelihood that a participant would commute, taking into account the probability of commuting for an individual in the French population and the sample. The red curve indicates the number of links in the French census network with a given probability of occurrence.

In addition to the most probable links, we can also test the representativeness of the statistically most relevant links forming the backbone of the commuting network extracted with the disparity filter algorithm. commuting network is able to capture some of the relevant features of the French commuting pattern at the level of regions (Figure 4). The centrality of Île-de-France is recovered, as well as the strong commuting pattern among the southern regions, and the connections between Northern regions and Southern regions. A larger discrepancy is instead observed in the absence of connections between the Corse region and the rest of the country.

Figure 4. Comparison of the commuting network structure between the and the French population.

a) Backbone of the census network as extracted by the disparity filter algorithm. b) GrippeNet network. Arrows represent directed links of commuters from a region of origin to a region of destination. The backbone of the census network has 49 links and includes 17 of the 28 links of the GrippeNet network.

Socio-economic Factors

Active participants more often had at least a high school level of education (equivalent to “baccalauréat”), ranging from 71.8% (95% C.I. = 65.8%–77.9%) in the [55][64] age group (vs. 29.9% in the general population) up to 96.9% (95% C.I. = 94.9%–99.0%) in the [25][34] (vs. 61.7%) among males (see Table 1). Analogous results were found in the female sample ().

A significant difference () was found in the occupation profile of the population with respect to the general population, with 50.9% (95% C.I. = 49.1%–52.7%) vs. 48.7% of paid employed individuals, and 4.5% (95% C.I. = 4.3%–4.6%) vs. 4.9% of self-employed individuals, respectively.

Health Factors and Vaccination

The prevalence of overweighted and obese individuals is significantly lower in population than in French population, for men () and for women (respectively and ) (Table 1).

No significant difference was found between and French populations for diabetes prevalence in the age classes of [45–74] and of ≥75 (), however no national data is available for comparison in individuals between 18 and 45 years old (see Table 2).

Among participants, 5.9% (95% C.I. = 5.1%–6.7%) declared that they take regular medication for asthma, while the prevalence for asthma is 6% in French adults (non significant difference,).

Vaccination coverage to seasonal influenza in the 2011–2012 season was higher in our sample (Table 4) (). The difference between the two populations is greater in younger age groups (12.6%, 95% C.I. = 10.0%–15.2% vs. 6% in [25][34] age group) and decreases when the age group increases (63.1%, 95% C.I. = 59.6%–66.6% vs. 55.2% in individuals older than 65). A comparable difference is observed for individuals older than 65 in the 2010–2011 season (62.8%, 95% C.I. = 59.3%–66.4% vs. 53.8%).

Table 4. Percentage of seasonal influenza vaccination coverage, comparison between and French population (all differences are significant, ).

In the fall 2011, within the French population registered to the national health insurance, 2,074,437 vouchers were sent to at risk individuals aged <65, representing 4.7% of the French population below 65 years of age and registered to the national health insurance. 819,841 of them got vaccinated (39.5%). 10.7% (95% C.I. = 9.5% –12.0%) of participants under 65 received a vaccination voucher in the fall 2011, thus leading to an overrepresentation of the at-risk population aged <65 years in our sample (). This particular group of the population was also more likely to be vaccinated in the population with respect to the French population (vaccination rate, 53.0%, 95% C.I. = 46.8% –59.3% vs. 39.5%, ).


The surveillance system was launched during the 2011/2012 season to serve a surveillance function complementing the traditional disease surveillance [30], [48] with a rapid, flexible, and collaborative Web approach. This first study aimed to assess the feasibility of the approach regarding the participation of the French population, following other European countries, and to evaluate the representativeness of the population participating to the project.

Almost 4,000 individuals registered to the project during its first season, from all regions of the country. These results on the voluntary uptake of a newly developed project are encouraging, especially if we note that no massive communication campaign was put in place during the pilot season. Dropouts or highly discontinuous participation was very low and limited to less than 23% of the total number of registered individuals. The remaining 77% represents the set of active participants (considered here for the analysis of the representativeness), displaying an active participation rate within the sample consistent with previous experiences in other countries (from 69% in Portugal to 87% in The Netherlands in the 2006/2007 influenza season [22]). With one season only of data collection campaign, the system reached a national participation of 6.2 active participants per 100,000 inhabitants, attaining the same levels of participation of countries running the online surveillance system since a longer time (see Table S2 in File S1). The number of participants is clearly a critical aspect for obtaining accurate estimates that can be used to run epidemiological surveillance and cohort studies. With the next challenge being to increase the participation rate of the population, it is reassuring to note that >99% of the participants who responded to the satisfaction survey conducted at the end of the first season confirmed their willingness to continue their participation in the following seasons.

Ensuring a satisfactory level of representativeness of the sample compared to the general population is however crucial to allow further analyses. The population is not representative in terms of age and gender, with a large overrepresentation of women, and an underrepresentation of the youngest and oldest age classes. This is most likely due to two factors: the non-representative nature of the Internet population, thus translating into coverage biases, and the self-selection of participants, also called the ‘volunteer effect’ [49]. While many studies are advocating the use of the Internet and of other digital tools to collect valuable information about disease and health dynamics (see e.g. [10] and references therein), the use of the Web and the adoption of social media services may strongly vary among population groups [50]. In France more than 64% of the households had an Internet access in 2010 [51], showing a dramatic dependence on age (93.3% in [15][29] age class; 85.7% in [30][44]; 44.3% in [60–74], and only 8.3% for individuals aged 75+), and on education (29.1% for individuals with no qualifications vs. 91.1% in the highly qualified population). This may therefore explain the underrepresentation of both the elderly class and the class of individuals with low education level in the population, a characteristic commonly found in web-based surveys [19], [22], [52], [53], [54]. To achieve a better representativeness of individuals in these classes, the system will need to design targeted communication campaigns for these groups and facilitate the accessibility to the project. The latter could be achieved along two directions. On one side, it could aim at increasing availability and improving physical accessibility promoting digital education (see Ref. [55] as an example set in a European country with low Internet penetration); on the other, it could focus on the improvement of the graphical user interface of the online system to make it easier and more accessible for individuals not familiar with the Web and its applications.

Two discrepancies however arise that are not consistent with the Internet access statistics – namely, the strong underrepresentation of young individuals (aged [18][29]) and a strong overrepresentation of the population in the [60–69] age class. These features are common to both males and females, with the female sample also showing an overrepresentation in the [50][59] age class, similarly to the results found in other European countries [24], [34], [56]. Internet and social media usage are widespread among the younger portion of the population, however our results seem to indicate a low interest towards influenza-related topics that may affect their voluntary participation to the project. Different findings may be obtained in the case of similar web-based scientific surveys focusing on other topics than influenza (e.g. food behavior, diet and health [57]), and will be the object of future work. These discrepancies should be considered when planning the future developments of the system, e.g. by considering the creation of increasingly engaging mobile applications for data collection.

An opposite behavior in the self-selection bias may be at play in the [60–69] age class, as these individuals may be more interested in influenza-related topics, because for instance they may be more frequently concerned by influenza and its complications. We assessed that our sample is indeed non-representative with respect to risk status, with an over-sampling of the population in this group.

The substantial imbalance in the participation per gender is similar to the one observed in the UK [52], and more pronounced than the discrepancy observed in the Netherlands [19]. Evidence suggests that women are on average more interested in health-related topics and also exhibit a more active information-seeking behavior [58], [59]. For these reasons they appear to be more likely to volunteer participation in health questionnaires.

Once the non-representative nature of the population in terms of age and gender is discounted, the mobility of participants well represents the ratio between within-region and across-region commuters observed in the country, a feature that has important implications for the spatial spread of influenza across the country. The sampled commuting network reproduces half of the links with higher probability of occurrence and includes the majority of the statistically relevant connections of the census commuting network. Discrepancies are most likely due to the limited sample size, given that commuting across distant regions is not highly probable, and we expect an improvement in the comparison of the mobility behavior with higher participation rates.

The population is characterized by a higher vaccination rate with respect to the French population, and it includes a larger number of at risk individuals in the <65 class. We expect the same mechanism already discussed for the larger representativeness of the [60–69] age class to be at play. Surprisingly, the prevalence of other conditions, such as diabetes and asthma, is however not statistically different from the values observed in the French population. Given that at risk individuals under 65 years of age are defined by the presence of chronic pathologies, we expect that the overrepresentation of this category is due to other conditions different from asthma and diabetes. However no data at the national level is available to test this hypothesis. participants are less frequently found to be overweight and obese than the French population, reflecting a higher concern in health-related issues, as well as a higher socio-educational level of participants [39].

The overrepresentation of the vaccinated class is a critical point for the evaluation of the surveillance system in terms of incidence and risk factors. Adjusting for age, gender, and vaccination status would require an age/gender classification of vaccinated individuals in the general population that is not yet available in France, whereas it is routinely collected in other European countries [60], [61]. While in principle the inclusion of other parameters should be considered as well in the reweighting procedure (such as e.g. chronic diseases, educational level), detailed cross-reference data at the national level are lacking, and moreover this strategy would face sample size issues in underrepresented strata of population. It is important to note however that for other countries ILI incidence trends based on similar systems as associate well with sentinel GP data, and in some cases also with Google Flu Trends data, without correcting for the unrepresentativeness of their population [19], [22], [34], [56].

The online surveillance system is open to all individuals resident in France. Besides the demographic and socio-economic indicators used, we may expect that individuals belonging to minorities or ethnic groups, or foreigners living in France were underrepresented in this study. For ethical reasons, no questions were asked to evaluate the representativeness of the population on those criteria.

As a sensitivity analysis, we have modified the inclusion criteria to test the robustness of our results with respect to the definition of active participants adopted, and imposed a minimum frequency between consecutive symptoms surveys filled (at least one questionnaire per month, between the month of registration and the ending month of the study, April). 2,560 individuals (84% of the total) comply with the modified inclusion criteria. Re-analysis of all the data based on the more restrictive inclusion criteria disclosed no material differences in the interpretation of the results presented in the study and no change in the statistical significance of the results.


One of the potential limitations of the study is the possible lack of accuracy in the data reported by third parties. indeed allows participants to include multiple individuals under the same account, where one person who is the account manager fills in the data for all other account members. This is very convenient in a family setting, for example, where one member of the family fills in the information for the other members, or to include respondents who have limited computer expertise or have no access to the Internet. It is not possible to directly explore the accuracy of the information reported for third parties, however we could in principle test the robustness of the results obtained from the whole sample with respect to the subsample of accounts with multiple memberships.

Our sample of active participants was composed by a vast majority of individuals having a single membership account, with only 7.1% of the active participants belonging to multiple members accounts. The limited size of the sample with multiple members does not allow us to perform statistically significant tests after the pilot season, and we plan to investigate further this aspect in the following years after having reached a larger participation rate. We note however that these biases may be less important when completing the intake survey and may play a more important role in the case of the symptoms survey, where a full list of questions on symptoms, details of occurrence, and actions taken are posed to the participants. We will further investigate this aspect in future work.


The results of the pilot season confirm the feasibility of the introduction in France of participatory ILI surveillance based on online systems, with participation rates comparable to previous experiences in other countries. Once adjusted on age and gender, the sample is statistically representative of the French population for diabetes and asthma prevalence. The classes of individuals at risk, vaccinated individuals, or with a higher level of education are however all oversampled. These results need to be taken into account to correctly interpret epidemic data and also to plan changes and updates in future seasons. Targeted strategies for communication and the organization of specific activities can be informed by the findings of this study to increase coverage and participation rates, as well as to improve physical accessibility and easiness of use of the platform. Finally, it would be important to assess the generalizability of our results to the other European countries participating in the Influenzanet project, for the validation of the system across countries. This multi-country assessment may also provide useful indicators for further improvements in participation and coverage at the national level, learning from other countries’ experiences.

Supporting Information

File S1.

The file contains the intake survey (French and English versions), additional descriptive indicators non-comparable with national data, a description of the methodology used to analyze the commuting networks and corresponding supporting additional results, and the coverage of partner projects of in other European countries.




We thank the Influenzanet teams for assistance in setting up the survey. We particularly thank all those who took part in the project since its start in January 2012. We are also grateful to the members of the Scientific Committee for helpful discussions and advise.

Author Contributions

Conceived and designed the experiments: TB TH VC. Performed the experiments: MD CT TB IB AF TH DLB VC. Analyzed the data: MD CT CP. Wrote the paper: MD VC. Participated in the management and editing of the website: MD CT TB IB AF TH DLB CP VC. Contributed to the writing of the manuscript: MD CT TB IB AF TH DLB CP VC.


  1. 1. World Health Organization (2013) Surveillance and monitoring (updated April 2013). Available: Accessed 2013 April 25.
  2. 2. European Center for Disease Prevention and Control (2013) Influenza surveillance. Available: Accessed 2013 April 25.
  3. 3. Mandl KD, Overhage JM, Wagner MM, Lober WB, Sebastiani P, et al. (2004) Implementing syndromic surveillance: a practical guide informed by the early experience. J Am Med Inform Assoc 11: 141–150.
  4. 4. Parsons DF, Garnerin P, Flahault A, Gotham IJ (1996) Status of electronic reporting of notifiable conditions in the United States and Europe. Telemed J 2: 273–284. doi: 10.1089/tmj.1.1996.2.273
  5. 5. Fourquet F, Drucker J (1997) Communicable disease surveillance: the Sentinel network. Lancet 349: 794–795.
  6. 6. Halperin W, Baker EL, Monson RR (1992) Public health surveillance. New York: Van Nostrand Reinhold. 238 p.
  7. 7. Eysenbach G (2009) Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J Med Internet Res 11: e11. doi: 10.2196/jmir.1157
  8. 8. Lazer D, Pentland A, Adamic L, Aral S, Barabási A-L, et al. (2009) Computational Social Science. Science 323: 721–723. doi: 10.1126/science.1167742
  9. 9. van Gelder MM, Bretveld RW, Roeleveld N (2010) Web-based questionnaires: the future in epidemiology? Am J Epidemiol 172: 1292–1298. doi: 10.1093/aje/kwq291
  10. 10. Salathe M, Bengtsson L, Bodnar TJ, Brewer DD, Brownstein JS, et al. (2012) Digital epidemiology. PLoS Comput Biol 8: e1002616. doi: 10.1371/journal.pcbi.1002616
  11. 11. Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, et al. (2009) Detecting influenza epidemics using search engine query data. Nature 457: 1012–1014. doi: 10.1038/nature07634
  12. 12. Pelat C, Turbelin C, Bar-Hen A, Flahault A, Valleron AJ (2009) More Diseases Tracked by Using Google Trends. Emerg Infect Dis 15: 1327–1328. doi: 10.3201/eid1508.090299
  13. 13. Brownstein JS, Freifeld CC, Reis BY, Mandl KD (2008) Surveillance Sans Frontieres: Internet-based emerging infectious disease intelligence and the HealthMap project. PLoS Med 5: e151. doi: 10.1371/journal.pmed.0050151
  14. 14. Brownstein JS, Freifeld CC, Madoff LC (2009) Digital disease detection–harnessing the Web for public health surveillance. N Engl J Med 360: 2153–2155, 2157.
  15. 15. Signorini A, Segre AM, Polgreen PM (2011) The Use of Twitter to Track Levels of Disease Activity and Public Concern in the U.S. during the Influenza A H1N1 Pandemic. Plos One 6: e19467. doi: 10.1371/journal.pone.0019467
  16. 16. Influenzanet (2013) (updated April 2013). Available: Accessed 2013 April 25.
  17. 17. Carrat F, Sahler C, Rogez S, Leruez-Ville M, Freymuth F, et al. (2002) Influenza burden of illness: estimates from a national prospective survey of household contacts in France. Arch Intern Med 162: 1842–1848. doi: 10.1001/archinte.162.16.1842
  18. 18. Harcourt SE, Edwards DE, Fleming DM, Smith RL, Smith GE (2004) How representative is the population covered by the RCGP spotter practice scheme? Using Geographical Information Systems to assess. J Public Health (Oxf) 26: 88–94. doi: 10.1093/pubmed/fdh111
  19. 19. Marquet RL, Bartelds AIM, van Noort SP, Koppeschaar CE, Paget J, et al. (2006) Internet-based monitoring of influenza-like illness (ILI) in the general population of the Netherlands during the 2003–2004 influenza season. Bmc Public Health 6: e242.
  20. 20. deGroteGriepMeting (2013) (updated April 2013). Available: Accessed 2013 April 25.
  21. 21. gripenet (2013) (updated April 2013). Available: Accessed 2013 April 25.
  22. 22. van Noort SP, Muehlen M, Rebelo de Andrade H, Koppeschaar C, Lima Lourenco JM, et al. (2007) Gripenet: an internet-based system to monitor influenza-like illness uniformly across Europe. Euro Surveill 12: 229–233.
  23. 23. Influweb (2013) (updated April 2013). Available: Accessed 2013 April 25.
  24. 24. Tilston NL, Eames KTD, Paolotti D, Ealden T, Edmunds WJ (2010) Internet-based surveillance of Influenza-like-illness in the UK during the 2009 H1N1 influenza pandemic. Bmc Public Health 10: e650. doi: 10.1186/1471-2458-10-650
  25. 25. flusurvey (2013) (updated 2013). Available: Accessed 2013 April 25.
  26. 26. influensakoll (2013) (updated April 2013). Available: Accessed 2013 April 25.
  27. 27. (2013) (updated April 2013). Available: Accessed 2013 April 25.
  28. 28. Inserm (2013) Institut national de la santé et de la recherche médicale (updated April 2013). Available: Accessed 2013 April 25.
  29. 29. UPMC (2013) Université Pierre et Marie Curie(updated April 2013). Available: Accessed 2013 April 25.
  30. 30. Flahault A, Blanchon T, Dorleans Y, Toubiana L, Vibert JF, et al. (2006) Virtual surveillance of communicable diseases: a 20-year experience in France. Stat Methods Med Res 15: 413–421.
  31. 31. InVS (2013) Institut de veille sanitaire(updated April 2013). Available: Accessed 2013 April 25.
  32. 32. (2012) dans la presse (updated 2012). Available: Accessed 2013 April 25.
  33. 33. Réseau Sentinelles (2013) Bulletin hebdomadaire (updated April 2013). Available: Accessed 2013 April 25.
  34. 34. Friesema IHM, Koppeschaar CE, Donker GA, Dijkstra F, van Noort SP, et al. (2009) Internet-based monitoring of influenza-like illness in the general population: Experience of five influenza seasons in the Netherlands. Vaccine 27: 6353–6357. doi: 10.1016/j.vaccine.2009.05.042
  35. 35. Institut national de la statistique et des études économiques (2013) Les estimations de population (updated January 15, 2013). Available: Accessed 2013 April 25.
  36. 36. Caisse Nationale d’Assurance Maladie, Ministère des affaires sociales et de la santé, Agence nationale de sécurité du médicament et des produits de santé, Institut de veille sanitaire (2012) Campagne de vaccination contre la grippe saisonnière 2012–2013. Press release. Available: Accessed 2013 April 25.
  37. 37. Institut national de la statistique et des études économiques (2011) Les variables issues de la Caisse Nationale d’Assurance Maladie des Travailleurs Salariés (CNAMTS). Available: Accessed 2013 April 25.
  38. 38. Tuppin P, Choukroun S, Samson S, Weill A, Ricordeau P, et al. (2012) [Vaccination against seasonal influenza in France in 2010 and 2011: Decrease of coverage rates and associated factors.]. Presse Med 41: 568–576.
  39. 39. Eschwege E, Charles M-A, Basdevant A, Moisan C, Bonnélye G, et al.. (2012) ObEpi Roche 2012 - Enquête épidémiologique nationale sur le surpoids et l’obésité. Available: Accessed 2013 April 25.
  40. 40. Ecosante (2012) Les bases Eco-Santé en ligne (updated December 2012). Available: Accessed 2013 April 25.
  41. 41. Institut national de prévention et d’éducation pour la santé (2010) Premiers résultats du baromètre santé 2010 - Evolutions récentes du tabagisme en France. Available: Accessed 2013 April 25.
  42. 42. Delmas MC, Fuhrman C (2010) [Asthma in France: a review of descriptive epidemiological data]. Rev Mal Respir 27: 151–159.
  43. 43. Institut national de la statistique et des études économiques (2007) Bases sur les flux de mobilité (updated 2007). Available: Accessed 2013 April 25.
  44. 44. Barabasi AL, Albert R (1999) Emergence of scaling in random networks. Science 286: 509–512. doi: 10.1126/science.286.5439.509
  45. 45. Barrat A, Barthelemy M, Vespignani A (2008) Dynamical processes on complex networks. Cambridge: University Press. 361 p.
  46. 46. Serrano MA, Boguna M, Vespignani A (2009) Extracting the multiscale backbone of complex weighted networks. Proc Natl Acad Sci U S A 106: 6483–6488. doi: 10.1073/pnas.0808904106
  47. 47. Balcan D, Colizza V, Goncalves B, Hu H, Ramasco JJ, et al. (2009) Multiscale mobility networks and the spatial spreading of infectious diseases. Proc Natl Acad Sci U S A 106: 21484–21489. doi: 10.1073/pnas.0906910106
  48. 48. Hannoun C, Dab W, Cohen JM (1989) A new influenza surveillance system in France: the Ile-de-France “GROG”. 1. Principles and methodology. Eur J Epidemiol 5: 285–293. doi: 10.1007/bf00144828
  49. 49. Friedman C, Wyatt J, Smith A, Kaplan B (1997) Evaluation Methods in Medical Informatics. New York: Springer. 404 p.
  50. 50. Smith A, Brenner J (2012) Twitter use 2012. Available: Accessed 2013 April 25.
  51. 51. Institut national de la statistique et des études économiques (2011) Deux ménages sur trois disposent d’internet chez eux. Available: Accessed 2013 April 25.
  52. 52. Brooks-Pollock E, Tilston N, Edmunds WJ, Eames KTD (2011) Using an online survey of healthcare-seeking behaviour to estimate the magnitude and severity of the 2009 H1N1v influenza epidemic in England. BMC Infect Dis 11.
  53. 53. Shah SM, Cook DG (2008) Socio-economic determinants of casualty and NHS Direct use. J Public Health (Oxf) 30: 75–81. doi: 10.1093/pubmed/fdn001
  54. 54. Ross MW, Mansson SA, Daneback K, Cooper A, Tikkanen R (2005) Biases in internet sexual health samples: comparison of an internet sexuality survey and a national sexual health survey in Sweden. Soc Sci Med 61: 245–252. doi: 10.1016/j.socscimed.2005.01.019
  55. 55. BiblioNet Program (2013) Biblionet programme, aiming to put free public access computers and internet coupled with a trained librarian in 2,200 public libraries across Romania (updated 2013). Available: Accessed 2013 April 25.
  56. 56. Vandendijck Y, Faes C, Hens N (2013) Eight years of the Great Influenza Survey to Monitor Influenza-Like Illness in Flanders. Plos One 8: e64156. doi: 10.1371/journal.pone.0064156
  57. 57. Etude Nutrinet-Santé (2013) (updated April 2013). Available: Accessed 2013 April 25.
  58. 58. Centre de Recherche pour l’Etude et l’Observation des Conditions de Vie (2004) Les opinions des femmes et des hommes sont-elles semblables ou différentes ? Cahier de recherche 195.
  59. 59. Fox S, Rainie L (2000) The online health care revolution: How the Web helps Americans take better care of themselves. Available: Accessed 2013 April 25.
  60. 60. Italian Ministry of Health (2012) Vaccination coverage per region. Available: Accessed 2013 April 25.
  61. 61. Nunes B., Branco MJ (2012) Vacinação antigripal da população portuguesa, em 2011–2012: cobertura e algumas características do acto vacinal. Available: Accessed 2013 April 25.
  62. 62. Institut national de la statistique et des études économiques (2012) Professions de santé en 2012. Available: Accessed 2013 April 25.
  63. 63. REGUS group (2010) Un actif français sur cinq passe plus d’1h30 par jour dans les transports. Press release Wellcom. Available: Accessed 2013 April 25.
  64. 64. Chambre Syndicale des Fabricants d’Aliments pour Chiens Chats Oiseaux et autres animaux familiers (2010) Enquête FACC/TNS SOFRES 2010 sur le Parc des Animaux Familiers. Available: Accessed 2013 April 25.