Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Agroecosystem resilience. A conceptual and methodological framework for evaluation

  • Cindy Córdoba ,

    Contributed equally to this work with: Cindy Córdoba, Catalina Triviño, Javier Toro Calderón

    Roles Conceptualization, Formal analysis, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing

    Affiliation Centro Interdisciplinario de Estudios sobre Desarrollo (CIDER), Universidad de los Andes, Bogotá, Colombia

  • Catalina Triviño ,

    Contributed equally to this work with: Cindy Córdoba, Catalina Triviño, Javier Toro Calderón

    Roles Conceptualization, Investigation, Writing – original draft

    Affiliation Instituto de Estudios Ambientales, Universidad Nacional de Colombia, Bogotá, Colombia

  • Javier Toro Calderón

    Contributed equally to this work with: Cindy Córdoba, Catalina Triviño, Javier Toro Calderón

    Roles Conceptualization, Methodology, Writing – review & editing

    Affiliation Instituto de Estudios Ambientales, Universidad Nacional de Colombia, Bogotá, Colombia

Agroecosystem resilience. A conceptual and methodological framework for evaluation

  • Cindy Córdoba, 
  • Catalina Triviño, 
  • Javier Toro Calderón


This article proposes a conceptual and methodological framework for analyzing agroecosystem resilience, in which aspects such as agrarian structure and peasant community agency are included as determining factors. The methodology is applied to a comparison of two peasant communities in Latin America (Brazil and Colombia), emphasizing the capacity to transform unsustainable power structures in place of adapting to them. We find that when agrarian structure is more equitable and peasant agency is strongly developed through political formation, organization and women’s participation, then there is a greater construction of resilience that improves peasant livelihoods and dignity. This application demonstrates that when agency is strongly developed, as in the case of Brazil, it is possible to transform structural conditions that restrict resilience. The inclusion and consideration of biophysical variables, management practices, agrarian structure and agency, through a participatory approach, allows for the identification of factors that inhibit or potentiate the resilience of agroecosystems.


The concept of resilience has evolved from an ecological perspective to that of complex systems analysis. Initially, it was conceived as the capacity to confront, absorb and adapt to disturbances, without changing, in order to return to a state of normality [1,2]. Resilience was calculated or evaluated depending on the amount of time it would take to return to this condition [3]. Analysis and discussions in the context of socio-ecological systems challenged the idea of normality, adopting an understanding of multiple equilibriums and accepting the inevitability of change [2,4]. In this sense, many proposed that resilience is systems adaptation based on learning, planning and reorganization for the purpose of preserving function, structure and identity [57].

Still, socio-ecological systems such as agroecosystems, conceptualized from a perspective of “fully integrated system[s] of people and nature” [8], do not exhibit unique identities, functions or structures [9]. Agroecosystems are systems composed of physical, biological, socioeconomic and cultural subsystems that coalesce and interact within the framework of human-led agricultural processes [10,11]. In this sense, human intervention, expressed in different interests, values and criteria, impede the determination of a unique structure and system function [9,12].

Any system involving human interaction holds power relations that can form or influence resilience [13], since these determine which groups have access to and control of resources, assume the burden of risk, and have the possibility of participation and political decision-making [14,15]. Additionally, the fluctuating nature of these systems clashes with the concept of identity, which can be understood as seeking a static and invariable condition [16].

The complementary concept of resilience offered in this article is not necessarily neutral or inherently positive, due to the lack of consensus across society on the objectives and strategies for responding to or interacting with change or disturbances [17]. The resilience of agroecosystems is often power-dependent. While resilience can increase through the operation of privileged groups with greater access to resources and political participation, it can also decrease under groups with less economic power [18]. Therefore, it is necessary to question, resilience for whom and for what purpose? [16,19]. In this study, resilience is analyzed from the perspective of peasant and rural communities in Latin America. From the point of view of the elite, resilience is understood to be adaptation to conditions of inequality and injustice, which agrees well with neoliberal [20] and Keynesian discourse, in other words, maintaining the status quo. On the other hand, those with less power understand resilience to be transformation conducive of conditions of justice, which can lead to the destruction of the predominant social system [1,2124].

To draw on the ideas of several authors [4,2530], resilience is proposed as an emergent property of complex systems (family, region, nation) and as a result of the dialectical multi-level interaction of its elements. This property allows the system to buffer, to adapt to and most importantly, to innovate and to be transformed, not only in response to specific points of tensions, but also to continuous and inevitable biophysical and social changes in the environment. Resilience is not a neutral concept. It is lodged within power relations and should be analyzed considering point of view, interests, access to resources, and the researchers’ place in society. Therefore, it does not only or mainly represent a system’s return to a “normal” state. Rather, on the contrary, it necessarily implies discontinuities, structural changes and dynamic developments within the system.

The purpose of this article is to present a conceptual framework and complementary participatory methodology for analyzing and evaluating/measuring agroecological resilience, including factors relevant to agrarian structure and peasant community agency. We consider that these factors are determinants in building resilience, since greater degrees of peasant agency as well as equity in agrarian structure generate radical changes in social structures, allowing for the construction of resilience, and therefore generating conditions for the dignifying of the human condition. This approach and methodology are applied in the comparison of two rural peasant communities in Latin America (Brazil and Colombia), emphasizing the capacity to transform unsustainable power structures instead of adapting to them.

The first part of the article refers to the elements that are included in calculating resilience indicators, followed by an analysis of the reach and limitations of methodologies that have been applied in rural contexts. On this basis, a new methodology is proposed for analyzing resilience. This new methodology is then applied to two locations in Brazil and Colombia. The results are presented and discussed, followed by general conclusions.

Agrarian structure

Whenever agroecological systems are analyzed, it becomes necessary to define the agrarian structure (AS), whose nucleus is the property of the land, based on which all other economic, social, cultural and political interactions are built. This concept combines a set of factors including the size of agroecosystems, the use and control of resources, labor conditions, relationships among social actors and between social actors and the market, infrastructural aspects and other features [31].

Agrarian structure is understood as a territorial configuration that includes the way in which the entire territory is distributed and occupied [32,33]. From this formation, power relations are established as determined by development models that promote changes mainly in the size of the property, its concentration and / or fragmentation, as well as changes in land use [34]. [32], describes the following configurations of agrarian land structure:

  • Unimodal:—Most of the land is distributed among medium-sized landowners (low concentration of property).
  • Bimodal: Most of the land is distributed among a few owners, leaving a small proportion in the hands of many small owners (high concentration of property).
  • Multimodal: Small and medium-sized property dominates the structure, coexisting in turn with large, highly use property. All these are articulated through alliances, business and cooperation systems (low concentration of property).
  • In Latin America, land has been employed as an instrument of power and social domination [35,36]. High levels of land concentration (called “latifundia”) or small subsistence-oriented (called “minifundia”) farms constitute the principal motor for the backwardness and underdevelopment of the rural sector [3740]. Since AS is transcendentally vital to productive power relations, peasant marginalization, territorial sovereignty, food production and access to dignified living conditions, it is surprising that it has not been included within analyses of resilience in the rural sector.

Land has historically been configured as a central means of production, whose appropriation and accumulation lay the groundwork for the construction of social power relations that determine the peasant population’s access to resources, goods and services, is a main element of their dignity and identity, and defines a great extent of their autonomy, socioeconomic conditions and the development of their means of livelihood. Additionally, in small portions of land it is not possible to develop traditional cultivation methods, such as rotations, patches of vegetation, fallow areas, etc. All of these factors directly impact resilience and the capacity for transformation within rural communities [4143].

Social dynamics determine how resources are distributed among social groups and therefore those who are most exposed to risk or are most vulnerable [44]. Identifying the structural causes of such vulnerability makes it possible to determine strategies to increase the resilience of rural communities, enabling their access to resources. This is what the livelihoods approach tries to do by including different types of factors. These incorporate productive practices, which includes criteria such as soil management and biodiversity that allow us to identify the ways in which agroecosystems deal with issues such as climatic variability. Another group of factors involves agroecosystem conditions and context that determine their degree of vulnerability. This group includes farmers’ biophysical, social, and health criteria. Each criterion allows us to establish what resources communities have and where they stand in the face of continuously changing environmental conditions. The agrarian structure factor, although it has been mentioned by authors such as [44] as the most important asset for farmers, had not been quantified, nor have the criteria used for its specific analysis in agroecosystems been determined. Therefore, criteria such as productive relationships and land tenure reveal power relations that affect rural resilience and have been made invisible in agroecosystem studies. The last factor is the capacity of agency that will be developed in the next section.

The inclusion of this set of factors gives a more complete understanding of the agroecosystem within a framework of the asymmetries of power evidenced by each of the criteria. This approach allows for the simultaneous consideration of factors at multiple levels that shape resilience as a process, not only of adaptation but of transformation, towards conditions of greater dignity.

Capacity of agency

The capacity of agency is understood as the empowerment of marginalized communities to engage in collective objective-oriented action aimed at transforming societal power relations [24,45,46]. Agency goes beyond resisting, buffering or adapting to the hardships of capitalism. It implies that peasants can build new paths in response to a system they consider unsustainable [2], employing their own creativity, political decision-making and organizational power, to unravel their own development processes.

The capacity for transformation that stems from community agency allows for access to resources such as land, favoring food sovereignty and increasing true participation in political decision-making processes, leading to greater resilience. On the contrary, the capacity for adaptation could give way for a community to return to a previous state of injustice and inequality, without questioning the asymmetries of power [47]. Resilience is developed when communities construct new paths, reorganizing ecological and social structures and redistributing risks, costs and benefits to produce social transformations.

The role of organized collective agency has not been sufficiently integrated within analyses of resilience [46]. More research is needed to include the ways in which human actions shape factors such as agrarian structure. Resilience analyses in agroecosystems have not considered power relations, assuming the existence of a society in consensus, in which it is common for certain groups to support the disasters provoked by capitalism [17,48]. Nevertheless, it is relevant to include the way in which conscious choices made collectively and individually can transform conditions of inequality towards essentially new systems, this being a fundamental factor in the level of resilience acquired [9,49,50].

However, some authors have tried to integrate factors such as power, inequality and agency into resilience thinking. [44,5153] argue that resilience has been used to reproduce neoliberal discourses, reducing state participation, while promoting community self-sustainability and individual responsibility. In this way, hegemonic thoughts and values are normalized and multiplied.

Contrary to the idea of resilience as a return to a state of normalcy or adaptation to unfair social conditions, another approach raises resilience as a transformation of social and environmental problems, based on forms of community activism and organization [46,54]. Articulating resilience with the fundamental problems of capitalism is still a field to explore at a theoretical and empirical level [51].

[55] On the other hand, points out that although conflicts, inequality and power relations are part of human societies, there is a generalized assumption that there is consensus among social groups. This ignores unequal exchanges of technology, information and resources in the predominant development model. Therefore, the concept of socio-ecological resilience must address the historical problems of conflicts of interest, asymmetries of power and inequality in distribution [24,25]. However, it would not be logical for capitalism to admit or promote its antithesis, that is, a definition that questions it in its structural aspects.

Sectors with power in society seek to create a generalized approach to sustainability, which marginalizes and excludes populations with less political and economic power. In this way, the latter forge a path through citizen action and social movements to exercise governance based on principles of learning, participation and trust [56].

In the case of agroecosystems, it is necessary to consider that beyond the biophysical space they occupy, they are also influenced by symbolic, economic, political and technological relationships that interact on multiple scales and hierarchies [57]. Therefore, agroecosystems also reflect disputes over natural and economic resources and are affected by social struggles.

In this sense the proposed methodology for evaluating resilience includes the decisions peasants make about the use of resources for agricultural production (both infrastructure and subsistence), as well as the level of organization, training and political decision-making power [18,58]. It is relevant to incorporate a differentiated analysis, not only of the economic situation of women (pay for market-oriented work, subsistence and caregiving), but also of aspects related to their empowerment, such as the levels of organization and participation in political decision-making processes [35]. The participation of women is essential because they are considered to be political subjects who organize and participate in decision-making regarding economic, productive, technical and political aspects, thus transforming power relations [5963]. This set of factors allows farmers not only to endure any type of stress (economic, political, ecological, etc.), but to act on these factors to transform them in their favor.

Methodologies for evaluating resilience

There are many methodological problems and few evaluation frameworks for resilience in rural contexts [49,57,64]. Some methodologies are centered around ecological and productive variables, employing indicators such as landscape complexity, vegetation diversity, slope and soil conservation, energy efficiency, subsistence, water and soil conservation practices, input and technology dependence and others [10,6569]. These approaches address social factors only in a limited and tangential way through their general definition of resilience as the capacity of communities to adapt to extreme stressors within the productive sector.

Authors such as [4,46,49], recognize that the social aspects of resilience are weakly developed, especially with regards to empowerment. [54,70] include notions of collective community agency as important to resilience, but they do not propose measurement instruments. [71] presents eight [8] dimensions of community resilience with metrics that have not been applied in practice and that are centered on the capacity to adapt to change. [72] employ official statistics to propose an index of rural diversity, considering natural economic and social capital, under the premise that diversity increases resilience. Other authors include, in addition to ecological variables, factors such as food security, income, access to services and support networks [73]. These are, however, included without numerical qualifiers or variable weights. Although [74] quantify variables such as land size, financial sources, credit and network participation, these are limited to describing the way in which these influence the adoption of agricultural technologies. [57] present 13 indicators of agroecosystem resilience which include social organization, learning, local knowledge and autonomy. Nevertheless, none of these variables consider social inequalities or access to land, determining factors for peasant livelihoods. [75,76] Introduce aspects such as social inequality and land property, recognizing that the socio-cultural context limits resilience. They center their attention on the capacity of farmers to respond with productive agroecological practices and define empowerment as decision-making for adaptive farm management in response to disturbances.

None of these studies includes the role of peasant agency in the transformation of structural factors that subvert power relations, bypassing the role of political organization and the building of new pathways, not only in the productive or ecological sense, but also in the social and political spheres. Productive relationships, working conditions and the use and control of resources are not evidenced, neither is it specified what social group’s perspective is being analyzed in terms of resilience. All of this leaves unanswered the questions of resilience for what end? and for whom? raised by [16]. Authors that consider the transformation of the status quo instead of its preservation [2,70], do not develop methodological proposals for the quantification of principal variables.

Resilience is the result of complex interactions among ecosystems, economic, social and cultural systems and cannot be analyzed through a fragmented consideration of each component in isolation from the whole [72]. With this challenge in mind, a methodology is proposed for measuring resilience in rural peasant communities, through the quantification and weighing of differing attributes. In addition to aspects related to AS and peasant agencies, related factors are incorporated to the conditions and context in which productive activities are developed, including biophysical, social and health variables, as well as practices used in agricultural production. In addition, market interactions were considered, which represent the effect of variables out of the peasants’ control that exercise a strong impact on income level and livelihood development.

Therefore, it is necessary to present a complementary conceptual and methodological framework that allows the identification of factors that support or inhibit resilience in Latin American peasant communities. The complex analysis of diverse factors that constitute resilience, with an emphasis on AS and the capacity of collective agency, allows for an understanding of substantial aspects in need of transformation. This allows peasants to generate their own development dynamics based on their own interests and needs, favors processes of empowerment for implementing radical changes in the generation of public policy, access to resources and capital, and potential for autonomy [7779]. In this sense resilience refers to social change and challenges the status quo to give place to alternative scenarios [1,22,23].

Resilience cannot necessarily be measured after specific events that cause shock or stress, since when dealing with complex systems that are subject to constant change and fluctuation, it is possible to identify their degree of resilience at any time. In addition, not every change or alteration is negative, but can lead to alternative scenarios and new directions desired by the farmers. Resilience can be measured to identify crucial points needed to achieve structural transformations through public policies, management plans and demands for social struggle.

Material and methods

The community of peasants interviewed in the research was involved in the planning and development of the methodology, as well as in the analysis and discussion of the results. The ethical procedure included informed consent on the use of their testimonies in interviews and surveys. This involved specifying that the use of the information has an exclusively academic purpose and that they could stop participating at any time. The peasants contributed and benefited from the research by participating and discussing the study topics. The methods applied in this research and the peasants and professionals interviewed were approved and a endorsed by the National University of Colombia, Faculty of Agricultural Sciences, in the framework of the Ph.D. Thesis “Resilience and Climate Variability in Coffee Agroecosystems in Anolaima (Cundinamarca, Colombia)” (Record/Acta No. 013, 28th May 2012, Council of the Faculty of Agricultural Sciences, National University of Colombia).

Proposed methodology for evaluating resilience

The procedure for evaluating resilience consists of three phases: (i) selection and weighting of factors, criteria and variables, (ii) scoring of variables, (iii) assigning quantified values to resilience.

Selection and weighting of variables

A scoring matrix was built with a hierarchical structure composed of four [4] factors, eight [8] criteria and seventeen [17] variables (Fig 1). Weighting coefficients were assigned to each variable, factor and criteria, through consultation with principal actors in each community as well as expert opinion from several disciplines (anthropology, agroecology, health sciences, environmental sciences and administration). The final values were determined in a participative manner using the Delphi method (Table 1), which establishes a structured communication between experts and community members who are knowledgeable of study sites, to validate each category used in the analysis [80,81].

Fig 1. Hierarchical structure for the evaluation of agroecosystem resilience.

Table 1. Weighting matrix of factors, criteria and variables for the assessment of resilience.

This methodological approach is mixed methods and participatory, since communities and local actors are consulted with to determine both qualitative and quantitative values of factors, criteria and variables. This characteristic allows the methodology to be applied or reproduced in other territorial contexts, because the variables are evaluated according to the characteristics of the study site and are qualified according to generic scores applicable in various rural contexts (Tables 1 and 2). The weights are adapted to the criteria of local actors that participate, whether indigenous, black or afro communities or others.

Scoring of variables

Data for differing variables were reported in different measurement units. For example, the area of land is expressed in hectares and the level of income in currency, while other characteristics are qualitative (land ownership or pertinence to organizations). Therefore, all measurement units were transformed to a standard 0 to 5 scale, where 0 represents the lowest level of resilience and 5 the greatest. This methodological strategy has been utilized and validated in several similar studies [8290]. The values were negotiated in a participatory manner, employing questionnaires, semi-structured interviews, expert opinion and literature review. Table 2 presents the consolidated matrix with scoring criteria.

Quantitative assessment of resilience

The value of agroecosystem resilience is the result of the sum of the 17 weighted variables. Where: AgRe: Agroecosystemic Resilience; Vi: Variables; Wi: Weight.



The proposed methodological model was applied to two localities in Colombia and Brazil: the municipality of Marulanda within the state of Caldas in Colombia (Lat 5° 17’ 3” North, Long 74° 15’ 48” West), and the municipality of Varzelandia within the North of Minas Gerais in Brazil (Lat 15° 42’ 5” South; Long 44° 1’ 39” West) (Fig 2). These sites were chosen because they share certain aspects such as the bimodal structure of land ownership, where “latifundia” and fragmented smallholder farms are predominant, with self-sustainable agricultural family units (Family Agricultural Units or UAF) and inspection units (Fiscal Modules) under the recommended area (18,83 ha) by the Colombian Institute for Rural Development (Colombian Institute for Rural Development or INCODER) in Colombia and the recommended area (50 ha) by the National Institute for Colonization and Agrarian Reform (INCRA) in Brazil. This land concentration generates inequality in power relations, that should be considered when measuring resilience in rural communities. On the other hand, the marked differences between these two communities to transform their socioecological systems allows a comparison of their level of agency and how this influences the final evaluations of resilience in each case.

The municipality of Marulanda has an area of 37,857 hectares and is divided into 14 neighborhoods. Soils are predominantly susceptible to erosive processes and high slopes are prevalent. There is a wide network of rivers and streams, an average temperature of 19°C and an altitudinal range between 1550 and 3750 meters above sea level (ms.asl).

In this municipality, which has an approximate population of 2,081 people [91] and whose Family Agricultural Unit (UAF) is defined at 18.83 hectares, the base of the economy in the agroecosystems located at an average altitude of 1800 ms.asl is the cultivation of coffee, sugarcane cane and avocado. On the other hand, agroecosystems located at an average height of 2800 ms.asl rely mainly on the livestock sector through dairy farming and cattle raising. There is also a small number agroecosystems dedicated to sheep raising for meat and wool [92].

Varzelandia has an area of 81,499 ha, an altitude of 745 meters above sea level, an average temperature of 29°C and an estimated population of 19320 inhabitants [93]. The area is predominantly dry forest which has been diminished due to the extensive livestock activities that occupy almost 77% of the municipality. These large agroecosystems dedicated to livestock belong to 8 landowners [94].

The agroecosystems that occupy the remaining percentage of the municipality are allocated to subsistence agriculture with corn, beans and sugarcane plantations as well as small-scale livestock production. The production is for self-supply. Many of the surplus products are sold internally and others in the urban area of the municipality. The main product that is transformed is cassava flour and in some productive units, panela is produced [94].

Data collection

Qualitative and quantitative methods were combined for the analysis of biophysical and sociocultural conditions that come into play in the resilience of both communities. The following data collection instruments were used:

Participatory workshops.

5 group workshops were conducted in the municipality of Marulanda and 8 in the municipality of Varzelandia, including main actors in each municipality. In the workshops variables and resilience scoring criteria were defined in a participatory manner.


surveys were conducted in each of the studied agroecosystems (N = 34), employing a questionnaire composed principally of close-ended multiple-choice questions and forecasting [95].

Semi-structured interviews.

23 semi-structured interviews were conducted in Marulanda and 31 in Varzelandia, with town officials, peasants, leaders of political and local organizations, which permitted a greater degree of flexibility and depth in obtaining information [96]. The interviews were conducted in different workspaces of planting and harvesting, local commerce and the home.

Results, discussion, conclusions

It is necessary to clarify that resilience was measured at the present time, the way in which each community arrived at its current state of resilience state was analyzed and qualified, as well as the main factors that led them to this point and that determined the degree of resilience of each location.

In consensus, the communities of both municipalities and experts assigned a coefficient of 0.5/1.0 to the capacity for agency, since it represents an indispensable factor for the construction of resilience. Agency is directly related to the ability of the community to self-organize and strengthen autonomy and participation in decision-making spaces, generating transformations, adjustments and modifications at different scales in each social, economic, political, ecological, and livelihood context.

The factor that was given the second most important weight was agrarian structure (0.37/1.0), which consists of the size of the agroecosystem, the type of ownership and other factors derived from the first two, such as market relations, working conditions and income level. The remaining criteria, "conditions and context" and "productive practices", were given lesser relevance in the construction of resilience, since they can be modified by human agency. Therefore, they were assigned a weight of (0.1/1.0) and (0.03/1.0) respectively.

Fig 3 shows that the resilience of agroecosystems in the municipality of Marulanda, Colombia is lower than that of agroecosystems in the municipality of Varzelandia, Brazil. The municipality of Marulanda had low scores (< 2.2), while the municipality of Varzelandia had scores between 1.9 and 3.5.

Fig 3. Total values of resilience in Brazil and Colombia.

The analysis of variance showed, with a confidence level of 95%, that there are statistically significant differences between the resilience of the municipalities of Varzelandia and Marulanda. Also, the resilience of agroecosystems in Varzelandia is significantly greater than the resilience of agroecosystems in Marulanda, and that this result is not due to chance [97]. Variables with statistically represenative differences include: the degree of membership in organizations, the degree of training and political decision-making power, political participation of women, infrastructure, land ownership and working conditions.

The inclusion of AS and agency criteria allows for a closer representation of reality and explains why some variables held higher ratings than others. In the case of Varzelandia, the peasants’ capacity for agency modified certain aspects related to agrarian structure, for example, through the occupation and ownership of fertile lands (average score of 4.6) and flat lands (average score of 4.3), which were previously owned by powerful landowners (77% of the territory was held by 8 landowners). This factor also allowed for a transformation of productive relationships by developing a collective production area where women, youth and elderly are remunerated through hourly pay (average score of 2.2 versus 0.4 for the municipality of Marulanda). Prior to developing the collective area, the peasants in Varzelandia worked under local landowners and were often exploited. This implies that beyond adapting, they managed to transform structural conditions, enhancing their resilience.

In addition, the capacity of organization and community-level management created enough pressure for the Mayor and city council to provide materials and machinery for the construction of a deep well and bridge over the river Arapuim, thus improving the infrastructure score (3.6 versus 2.8 for the municipality of Marulanda). In addition, the community committed itself to facilitating labor for these two projects, carrying out the process collectively. The installation of the deep well guarantees irrigation for the collective production area, and the construction of the bridge improves connectivity, transport and quality of life.

On the contrary, in the municipality of Marulanda, the community adapted to social conditions without achieving transformations that would improve peasant livelihoods. Therefore, in general, the score for pertaining to or connecting with organizations or cooperatives (average 2.4), as well as the degree of training and family-level political decision-making power (average 1.0) and especially women’s decision-making power (average 1.0), was low in all cases. In this municipality, the peasant smallholder has restricted access to resources, goods and services, and productive activities use unpaid family labor intensively, in order to increase their precarious income and improve living conditions. It is evident in this case that simple commodity production, developed individually, limits the accumulation of capital [98,99].

Socio-economic conditions influence community agency. In the municipality of Varzelandia, the deep history of land struggles and strong peasant organization has allowed farmers to solve problems related to land tenure and production relationships. However, in the municipality of Marulanda, the historical absence of land struggles has maintained a limited division of land parcels through informal agreements, perpetuating the dominant economic position of powerful landowners [98].

Factors at all scales affect the resilience of the agroecosystem. For example, peasants have no impact on the prevailing factors governing market relations, and therefore, market relations are scored as zero in both municipalities, regardless of the capacity for agency. The fixing of product prices is determined by various dynamics of the capitalist market and by local economic powers [100].

Dependence on the country's agricultural policies or international fluctuation of prices negatively affect resilience [2,101]. Therefore, it is necessary to include power relations derived from global scales, which prevent peasants from reaching full autonomy in decision making or real participation in processes of political definition [8,16,19,102].

Weighing criteria and landscape indicators give a closer sense of the reality of the case studies and allows a greater understanding of the factors that most strongly affect resilience. Without weighing variables, certain factors such as soil fertility, slope or access to public services would be considered on an equal level as criteria related to community agency or agrarian structure. The proposed methodology includes aspects that are normally invisible, revealing power relations and transformation processes that alter structures and predominant social dynamics within communities [21,103].

Fig 4 shows the results of calculating resilience without considering AS or agency, utilizing criteria associated with productive practices and biophysical conditions in comparison with the weighted average using all the proposed variables. The results of the municipality of Marulanda are higher in scenario Y than scenario X, with a variance between 3% and 74%. On the contrary, the municipality of Varzelandia showed lower results for scenario Y, lowering the mean values of resilience, with soil fertility and slope being the variables with the greatest weight.

Fig 4. Total resilience in Brazil and Colombia comparing all proposed variables (scenario X) vs. only biophysical factors and management practices (scenario Y).

When only biophysical factors and agricultural practices are considered, there is only a difference of 0.2 points between the two localities (an average score of 2.2 for Colombia vs. 2.4 for Brazil). On the contrary, when all variables are considered, the difference between average values is almost a whole point (an average score of 1.8 for Colombia vs. 2.8 for Brazil).

The values with the highest scores in the municipality of Marulanda were distance to forests and water sources (average 4.4 vs. 1.0 in Varzelandia) as well as the presence of rivers and water sources within ecosystems (1.5 vs. 0.1). In this sense it would be difficult to adopt strategies to increase resilience, since the criteria are already a part of the environment in which the agroecosystems are immersed and therefore difficult to modify.

Including all variables allows for an evaluation and analysis that can be used as an instrument to support decision-making in the short, medium and long term, as well as a tool for planning and determining effective solutions in the social sphere [104,105]. The transformations peasants require to increase their resilience involve power structures, markets, institutions and predominate societal values [101,102]. Beyond the biophysical factors and productive practices, rural populations are immersed in social contexts, within which they are challenged by political and economic differences, not only at the local scale but also at the global scale [16,18,106].


The findings reveal that the level of political organization and participation in decision-making processes regarding economic, productive, technical and political components of agroecosystems, as well as the acknowledgement of rights and the determination to organize to demand them, are factors that favor the transformation of structural aspects in the municipality of Varzelandia. Therefore, the capacity of agency received a greater weight in the overall quantification of resilience.

Our attention should not only be focused on the local population’s capacities to transform their conditions while understating the importance of the political, social and economic context that conditions these capacities. Conducts, values and the distribution of risks and benefits are formed by structures and social norms. Both factors are decisive in analyzing resilience.

The peasants of Marulanda have adapted to many circumstances without achieving transformation, while the peasants of Varzelandia have built effective social networks, strengthening their capacity for agency and transformation before conditions of social inequality.

The proposed methodology can be replicated in other contexts, including other indicators and weights that represent what is valued by a society, along with its knowledge and perceptions.

The proposed resilience is directed towards the formulation of strategies and policies aimed at inducing radical change at the local and regional level. In this way it cannot be constrained by access to technology or biophysical resources that favor adaptation and a limited sense of wellbeing for peasant communities.


This paper gratefully acknowledges from Interdisciplinary Center for Development Studies, Universidad de los Andes (Bogotá, Colombia); the project: “Environmental Impact Assessment in Colombia. Critical analysis and Improvement”, Code Hermes: 13129, Universidad Nacional de Colombia, sede Bogotá; Der Deutsche Akademische Austauschdienst (DAAD) and The Center for Development Research (ZEF) is an institute of the University of Bonn, Germany.


  1. 1. Martin-Breen P, Anderies JM. Resilience: A Literature Review. IDS; 2011.
  2. 2. Walker B, Holling C, Carpenter S, Kizing A. Resilience, adaptability and transformability in social-ecological systems. Ecol Soc. 2004;9(2).
  3. 3. Holling C. Engineering Within Ecological Constraints. Washington, D.C.: National Academies Press; 1996.
  4. 4. Berkes F, Folke C. Linking social and ecological systems: management practices and social mechanisms for building resilience. Cambridge UK: Cambridge University Press; 1998. 437 p.
  5. 5. Joseph J. Resilience as embedded neoliberalism: a governmentality approach. Resilience. 2013 Apr;1(1):38–52.
  6. 6. Carpenter S, Walker B, Anderies J, Abel N. From metaphor to measurement: resilience of what to what? Ecosystems. 2001;4(8):765–81.
  7. 7. Manyena SB. The concept of resilience revisited. Disasters. 2006 Dec;30(4):434–50.
  8. 8. Cumming G. Spatial resilience in networks. Spatial Resilience in Social-Ecological Systems. Cape Town: Springer; 2011. 121–142 p.
  9. 9. Davies J, Robinson LW, Ericksen PJ. Development Process Resilience and Sustainable Development: Insights from the Drylands of Eastern Africa. Soc Nat Resour. 2015 Mar;28(3):328–43.
  10. 10. León T, Toro J, Martínez F, Cleves A. The main agroecological structure (MAS) of the agroecosystems: concept, methodology and applications. Sustainability. 2018;10(3131):1–21.
  11. 11. Faden R, Beauchamp T, King N. A history and theory of informed consent. Oxford: Oxford University Press; 1986. 392 p.
  12. 12. Ute K, Rhys K. Resilience, solidarity, agency–grounded reflections on challenges and synergies. Resilience. 2017;5(1):10–28.
  13. 13. Pratt C, Kaly U, Mitchell J. Manual: how to use the environmental vulnerability index (EVI). SOPAC technical report 383 [Internet]. 2004. Available from:
  14. 14. Granderson AA. Making sense of climate change risks and responses at the community level: A cultural-political lens. Clim Risk Manag. 2014 Jan;3:55–64.
  15. 15. Ensor J, Park S, Hoddy E, Ratner B. A rights-based perspective on adaptive capacity. Glob Environ Chang. 2015;31:38–49.
  16. 16. Friend R, Moench M. What is the purpose of urban climate resilience? Implications for addressing poverty and vulnerability. Urban Clim. 2013 Dec;6:98–113.
  17. 17. Hatt K. Social Attractors: A Proposal to Enhance “Resilience Thinking” about the Social. Soc Nat Resour. 2013;26(1):30–43.
  18. 18. Béné C, Godfrey R, Newsham A, Davies M. Resilience: new utopia or new tyranny? reflection about the potentials and limits of the concept of resilience in relation to vulnerability reduction programmes [Internet]. Brighton, UK; 2012. (IDS Working Paper). Report No.: 405. Available from:
  19. 19. Lebel L, Anderies J, Campbell B, Folke C, Hatfield-Dodds S, Hughes T, et al. “Governance and the Capacity to Manage Resilience in Regional Social-Ec” by Lebel L., Anderies J. M. et al. Ecol Soc [Internet]. 2006;11(1):19. Available from:
  20. 20. Bohle HG, Etzold B, Keck M. Resilience as agency. Int Hum Dimens Program Updat. 2009;2:8–13.
  21. 21. Sinclair K, Rawluk A, Kumar S, Curtis A. Ways forward for resilience thinking: lessons from the field for those exploring social-ecological systems in agriculture and natural resource management. Ecol Soc. 2017 Nov;22(4):art21.
  22. 22. Hahn T, Nykvist B. Are adaptations self-organized, autonomous, and harmonious? Assessing the social–ecological resilience literature. Ecol Soc. 2017;22(1):12.
  23. 23. Matarrita-Cascante D, Trejos B, Qin H, Joo D, Debner S. Conceptualizing community resilience: Revisiting conceptual distinctions. Community Dev. 2017 Jan;48(1):105–23.
  24. 24. Pelling M, Manuel-Navarrete D. From resilience to transformation: the adaptive cycle in two mexican urban centers. Ecol Soc [Internet]. 2011;16(2):11. Available from:
  25. 25. Friend R, Moench M. What is the purpose of urban climate resilience? Implications for addressing poverty and vulnerability. Urban Clim [Internet]. 2013 Dec [cited 2016 Feb 23];6:98–113. Available from:
  26. 26. Folke C, Carpenter SR, Walker B, Scheffer M, Chapin T, Rockstrom J, et al. Resilience thinking: integrating resilience, adaptability and transformability. Ecol Soc [Internet]. 2010 [cited 2017 Jun 18];15(4). Available from:
  27. 27. Davoudi S, Shaw K, Haider LJ, Quinlan AE, Peterson GD, Wilkinson C, et al. Resilience: A Bridging Concept or a Dead End? “Reframing” Resilience: Challenges for Planning Theory and Practice Interacting Traps: Resilience Assessment of a Pasture Management System in Northern Afghanistan Urban Resilience: What Does it Mean in Planni. Plan Theory Pract. 2012;13(2):299–333.
  28. 28. Pendall R, Foster KA, Cowell M. Resilience and regions: building understanding of the metaphor. Cambridge J Reg Econ Soc [Internet]. 2010 Mar 1 [cited 2016 May 6];3(1):71–84. Available from:
  29. 29. Carpenter SR, Westley F, Turner MG. Surrogates for resilience of social-ecological systems. Ecosystems [Internet]. 2005 Dec 28 [cited 2016 May 20];8(8):941–4. Available from:
  30. 30. Pelling M, Navarrette M. From resilience to transformation: The adaptive cycle in two Mexican urban centers. Ecol Soc. 2011;16(2).
  31. 31. Arroyo M. On the Concept of Agrarian Structure (Sobre el Concepto de Estructura Agraria). Rev Geográfica [Internet]. 1990;112:141–52. Available from:
  32. 32. Machado A. De la estructura agraria al sistema agroindustrial. Bogotá: Universidad Nacional de Colombia; 2002. 320 p.
  33. 33. Mançano-Fernandes B. Sobre a tipologia de territórios. In: Territórios e territorialidades: teorias, processos e conflitos. Sao Paulo: Expressão Popular; 2009. p. 197–215.
  34. 34. Escobal J, Armas C. Agrarian structure and dynamics of rural poverty in Peru (Estructura agraria y dinámica de pobreza rural en el Perú) [Internet]. Vol. 79, Documento de Investigacion-GRADE. Lima: GRADE Group for the Analysis of Development; 2015. 72 p. Available from:
  35. 35. OXFAM. Unearthed: land, power and inequality in Latin America [Internet]. Oxford; 2016. Available from:
  36. 36. Gómez S. The land market in Latin America and the Caribbean: concentration and foreignization [Internet]. Soto F, Gómez S, editors. Santiago: Food and Agriculture Organization of the United Nations-FAO; 2014. 128 p. Available from:
  37. 37. Barrientos-fuentes J, Torrico-albino J. Socio-economic perspectives of family farming in South America: cases of Bolivia, Colombia and Peru. Agron Colomb. 2014;32(2):266–75.
  38. 38. Keswell M, Carter M. Poverty and land redistribution. J Dev Econ [Internet]. 2014;110:250–61. Available from:
  39. 39. Oyvat C. Agrarian structures, urbanization, and inequality. World Dev [Internet]. 2016;83:207–30. Available from:
  40. 40. Barraclough S, Domike A. Agrarian structure in seven latin american countries. Land Econ. 1966;42(4):391–424.
  41. 41. Adger WN. Social and ecological resilience: are they related? Prog Hum Geogr. 2000 Sep;24(3):347–64.
  42. 42. Berkes F. Understanding uncertainty and reducing vulnerability: lessons from resilience thinking. Nat Hazards. 2007 Apr;41(2):283–95.
  43. 43. Mayunga JS. Understanding and Applying the Concept of Community Disaster Resilience: A capital-based approach. In: Megacities as Hotspots of Risk: Social Vulnerability and Resilience Building. Munich, 22–28 de julio de 2007; 2007. p. 16.
  44. 44. Hess JJ, Mcdowell JZ, Luber G. Review Integrating Climate Change Adaptation into Public Health Practice: Using. Environ Health Perspect. 2012;120(2):171–9. pmid:21997387
  45. 45. Béné C, Newsham A, Davies M, Ulrichs M, Godfrey-Wood R. Review article: Resilience, poverty and development. J Int Dev. 2014 Jul;26(5):598–623.
  46. 46. Davidson D. The applicability of the concept of resilience to social systems: Some sources of optimism and nagging doubts. Soc Nat Resour. 2010;23(12):1135–49.
  47. 47. Córdoba Vargas CA, Hortúa Romero S, León Sicard T. Key points of resilience to climate change: a necessary debate from agroecological systems. Clim Dev [Internet]. 2019 Sep 19;1–11. Available from:
  48. 48. Kolers A. Resilience as a Political Ideal. Ethics, Policy Environ. 2016 Jan;19(1):91–107.
  49. 49. Berkes F, Ross H. Community resilience: toward an integrated approach. Soc Nat Resour. 2013;26(1):5–20.
  50. 50. Gunderson L, Carpenter S, Folke C, Olsson P, Peterson G. Insight, part of a special feature on exploring resilience in social-ecological systems water RATs (resilience, adaptability, and transformability) in lake and wetland social-ecological systems. Ecol Soc. 2006;11(1):16.
  51. 51. Cretney R. Resilience for whom? Emerging critical geographies of socio-ecological resilience. Geogr Compass. 2014;8(9):627–40.
  52. 52. MacKinnon D, Derickson KD. From resilience to resourcefulness. Prog Hum Geogr [Internet]. 2013 Apr 8 [cited 2019 Dec 4];37(2):253–70. Available from:
  53. 53. Ribot JC. Authority over forests: Empowerment and subordination in Senegal’s democratic decentralization. Dev Change. 2009;40(1):105–29.
  54. 54. Brown K, Westaway E. Agency, capacity, and resilience to environmental change: lessons from human development, well-being, and disasters. Annu Rev Environ Resour. 2011;36(1):321–42.
  55. 55. Hornborg A. Zero-Sum world: Challenges in conceptualizing environmental load displacement and ecologically unequal exchange in the world-system. Int J Comp Sociol [Internet]. 2009 Jun 20 [cited 2019 Dec 4];50(3–4):237–62. Available from:
  56. 56. Leach M, Scoones I, Stirling A. Dynamic Sustainabilities. Technology, environment, social justice. First Edit. Dynamic Sustainabilities: Technology, Environment, Social Justice. London and New York: Earthscan Publications Ltd; 2010. 143 p.
  57. 57. Cabell J, Oelofse M. An indicator framework for assessing agroecosystem resilience. Ecol Soc. 2012;17(1):1–18.
  58. 58. Norberg J, Cumming G. Complexity theory for a sustainable future [Internet]. New York: Columbia University Press; 2008. 315 p. Available from:
  59. 59. Adger N. Social capital, collective action, and adaptation to climate change. Econ Geogr. 2003;79(4):387–404.
  60. 60. Coulthard S. Can we be both resilient and well, and what choices do people have? incorporating agency into the resilience debate from a fisheries perspective. Ecol Soc. 2012;17(1):4.
  61. 61. Long NE, van der Ploeg JD. Heterogeneity, actor and structure towards a reconstitution of the concept of structure. Longman; 1994. p. 62–89.
  62. 62. Osbahr H. Building resilience: adaptation mechanisms and mainstreaming for the poor [Internet]. Oxford; 2007. Available from:
  63. 63. Ostrom E, Burger J, Field C, Norgaard R, Policansky D. Revisiting the Commons: Local Lessons, Global Challenges. Science (80-). 1999 Apr;284(5412):278–82.
  64. 64. Darnhofer I, Fairweather J, Moller H. Assessing a farm’s sustainability: Insights from resilience thinking. Int J Agric Sustain [Internet]. 2010 Aug 8 [cited 2018 Nov 26];8(3):186–98. Available from:
  65. 65. Altieri M. Agroecology: the science of natural resource management for poor farmers in marginal environments. Agric Ecosyst Environ. 2002;93:1–24.
  66. 66. Casimiro L, Casimiro J. How to make prosperous and sustainable family farming in Cuba a reality. Elem Sci Anthr. 2018;6:1–15.
  67. 67. Altieri M, Funes-Monzote F, Petersen P. Agroecologically efficient agricultural systems for smallholder farmers: Contributions to food sovereignty. Agron Sustain Dev. 2012;32(1):1–13.
  68. 68. Koohafkan P, Altieri M, Holt E. Green agriculture: foundations for biodiverse, resilient and productive agricultural systems, International Journal of Agricultural Sustainability. Int J Agric Sustain. 2011;10(1):61–75.
  69. 69. Sarandón S, Flores C, Gargoloff A, Blandi M. Analysis and evaluation of agroecosystems: construction and application of indicators. In: Sarandón S, Flores C, editors. Agroecology: theoretical bases for the design and management of sustainable agroecosystems [Internet]. La Plata, Argentina: Editorial de la Universidad de La Plata; 2014. p. 375–410. Available from:
  70. 70. Wilson G. Community Resilience and Environmental Transitions. London: Routledge; 2012. 240 p.
  71. 71. Magis K. Community Resilience: An Indicator of Social Sustainability. Soc Nat Resour. 2010 Apr;23(5):401–16.
  72. 72. Quaranta G, Salvia R. An Index to Measure Rural Diversity in the Light of Rural Resilience and Rural Development Debate. Eur Countrys. 2014;6(2):1–18.
  73. 73. Tambo JA, Abdoulaye T. Climate change and agricultural technology adoption: the case of drought tolerant maize in rural Nigeria. Mitig Adapt Strateg Glob Chang. 2012 Mar;17(3):277–92.
  74. 74. Peredo S, Vela M, Jiménez A. Determination of resilience/vulnerability levels of urban agroecology initiative in the south west of Andalusia. IDESIA [Internet]. 2016;34(2):5–13. Available from:
  75. 75. Gazzano I, Altieri MA, Achkar M, Burgueño J. Holistic Risk Index: A Case Study of Cattle Producers in the Protected Area of Farrapos Estuaries—Uruguay. Agroecol Sustain Food Syst. 2015;39(2):209–23.
  76. 76. Panpakdee C, Limnirankul B. Indicators for assessing social-ecological resilience: A case study of organic rice production in northern Thailand. Kasetsart J Soc Sci. 2018;39(3):414–21.
  77. 77. Bristow G, Healy A. Regional resilience: an agency perspective. Reg Stud. 2014;48(5):923–35.
  78. 78. Brown K. Global environmental change i: a social turn for resilience? Prog Hum Geogr. 2014;38(1):107–17.
  79. 79. Korzilius H. Researching resilience: some methodological is ssues. Syst Res Behav Sci. 2017;34(4):463–8.
  80. 80. Skeiker K, Ghani BA. Advanced software tool for the creation of a typical meteorological year. Energy Convers Manag. 2008 Oct;49(10):2581–7.
  81. 81. Lee Y, Altschuld J, Hung H. Practices and challenges in educational program evaluation in the Asia-Pacific region: Results of a Delphi study. Eval Program Plann. 2008;31:368–75. pmid:18849074
  82. 82. Arrieta G, Requena I, Toro J, Zamorano M. Adaptation of EVIAVE methodology for monitoring and follow-up when evaluating the environmental impact of landfills. Environ Impact Assess Rev. 2016;56:168–79.
  83. 83. Dee N, Baker J, Drobny N, Duke K, Whitman I, Fahringer D. An environmental evaluation system for water resource planning. Water Resour Res. 1973;9(3):523–35.
  84. 84. Glasson J, Therivel R, Chadwik A. Introduction to environmental impact assessment. 4th ed. Built Environment, Environment and Sustainability. London: Routledge; 2012. 416 p.
  85. 85. Lohani BN, Evans JW, Everitt RR, Ludwig H, Carpenter RA, Tu S-L. Environmental Impact Assessment for Developing Countries in Asia. Vol. 1, Asian Development Bank. 1997.
  86. 86. Ortolano L, Shepherd A. Environmental impact assessment: challenges and opportunities. Impact Assess. 1995;13(1):3–30.
  87. 87. Phillips J. The Geocybernetic Assessment Matrix (GAM)—A new assessment tool for evaluating the level and nature of sustainability or unsustainability. Environ Impact Assess Rev [Internet]. 2016;56:88–101. Available from:
  88. 88. Solomon R, Colbert B, Hansen W, Richardson S, Center L, Vlachos E. Water resources assessment methodology (WARM). Impact assessment and alternative evaluation [Internet]. 1977. Available from:
  89. 89. Martínez F, Toro J, León C. A complex network approach to environmental impact assessment. Impact Assess Proj Apprais. 2018;1–14.
  90. 90. Toro J, Requena I, Duarte O, Zamorano M. A qualitative method proposal to improve environmental impact assessment. Environ Impact Assess Rev. 2013 Nov;43:9–20.
  91. 91. DANE. National Census of Housing Population 2018-Colombia (Censo nacional de población vivienda 2018-Colombia) [Internet]. 2019 [cited 2019 Oct 10]. Available from:!/
  92. 92. Alcaldía de Marulanda. Development Plan 2016–2019 Marulanda for Unity and Development (Plan de Desarrollo 2016–2019 Marulanda por la Unidad y el Desarrollo) [Internet]. Marulanda-Colombia: Mayor of Marulanda; 2016. p. 281. Available from:
  93. 93. Instituto Brasileiro de Geografia e Estatística (IBGE). Estimativa populacional Varzelândia-Brasil [Internet]. 2019 [cited 2019 Oct 11]. Available from:
  94. 94. Ministerio de desenvolvimento agrarío du Brasil. Plano de Etnodesenvolvimento Quilombo de Brejo dos Crioulos [Internet]. Varzelandia–Minas Gerais: Centro de Agricultura Alternativa do Norte de Minas. (CAA-MN). Ministerio do Desenvolvimento Agrario. Plano de Etnodesenvolvimento Quilombolo de Brejo dos Crioulos; 2012. p. 28. Available from:
  95. 95. Benedetti R, Bee M, Espa G, Piersimoni F. Agricultural Survey Methods. New York: John Wiley and Sons; 2010. 395 p.
  96. 96. Galletta A. Mastering the semi-structured interview and beyond: from research design to analysis and publication. New York: New York University Press; 2012. 284 p.
  97. 97. Fisher R. Statistical Methods for Research Workers. Edinburgh: Oliver & Boyd; 1925.
  98. 98. Murillo-Lozano M. Colombian coffee culture in the XXI century: a revision to resent literature. Gestión y Región [Internet]. 2010;(9):127–52. Available from:
  99. 99. Marx K. Der achtzehut brumaire des Louis Napoleon [Internet]. Vol. 1, Die revolution. New York: Joseph Weydemeyer; 1952. Available from:
  100. 100. Lanka S V., Khadaroo I, Böhm S. Agroecology accounting: biodiversity and sustainable livelihoods from the margins. Accounting, Audit Account J. 2017 Sep;30(7):1592–613.
  101. 101. Darnhofer I. Resilience and why it matters for farm management. Eur Rev Agric Econ. 2014;41(3):461–84.
  102. 102. Pahl-Wostl C. A conceptual framework for analysing adaptive capacity and multi-level learning processes in resource governance regimes. Glob Environ Chang. 2009 Aug;19(3):354–65.
  103. 103. Walker B, Salt D. Resilience Thinking: Sustaining Ecosystems and People in a Changing World. Washington, D.C.: Island Press; 2006.
  104. 104. Belacel N. Multicriteria assignment method PROAFTN: Methodology and medical application. Eur J Oper Res. 2000;125(1):175–83.
  105. 105. Hajkowicz S, Collins K. A review of multiple criteria analysis for water resource planning and management. Water Resour Manag. 2007;21(9):1553–66.
  106. 106. Koppel J. A multiple knowledge approach for adaptation to environmental change: lessons learned from coastal Louisiana’s tribal communities. J Polit Ecol. 2014 Dec;21(1):61.