What makes a development project successful? I ask myself this question each day I spend designing and managing projects. Development certainly isn’t easy, or perfect. As a social science, it is inexact, subjective, and ever-evolving, striving for objective measurement in imperfect, variable-ridden environments. How can one rely on data to indicate success in what is at its beginning and end a human experience, intervention, or endeavor?

As foreign aid passes the quarter and half-century marks in some countries, we practitioners must become ever more mindful of our presence, and the effects thereof. Although investment varies from project to project, monitoring and evaluation is widely accepted as a fundamental pillar of effective development. In their 2013 annual letter, Bill and Melinda Gates re-emphasized the importance of measurement in the global efforts to improve the human condition.

Unlike profit generation in the business world, the “bottom line” of development is less straight-forward.  Standards of success are predicated on a subjective selection of inherently imperfect data, making “good” development nearly impossible to define. In project management, success is often considered the accomplishment of a specific goal within a designated time frame. When these are self-selected, proper goal-setting and measurement tools become all the more necessary to minimize confirmation and self-reporting bias. Shifting an indicator from number of toilets built to number of toilets used, for example, can have tremendous implications on the focus and direction of a project. Toilets abandoned and in disrepair several years after construction are an avoidable repeat investment. Dissemination of all data—the good, the bad, and the ugly— is critical for avoiding these kinds of duplications.

Depth Before Breadth

The Joint Initiative for Village Advancement, or “JIVA” as it is known, has sought to develop metrics that will inform the effectiveness and sustainability of its integrated approach. This multi-year community development program, funded by the John Deere Foundation, aims to improve the lives of those residing in three rural villages in Rajasthan, India. The program employs a multi-sector approach focused on three priorities: agriculture and income security, education, and small-scale infrastructure.

In a country of 1.2 billion people, a smaller, three-village investment may raise some eyebrows. Significant development efforts typically span broad geographies. By reaching more beneficiaries, the theory is that projects will scale more quickly and have a greater social impact. Yet these projects are often designed by officials or experts seated far from the communities they are aiming to serve. In India, more than two-thirds of the population lives in rural areas. With languages, customs, social structures, natural resources, and available services often changing in any given fifteen-mile radius, shouldn’t approaches to its rural development be equally diverse and varied?

Earlier this year, Nate Clark, Vice President of The John Deere Foundation, highlighted the importance of JIVA’s partnerships to its participatory approach in the first installment of this series. The project model depends heavily on the expertise and resources of its partners to successfully execute a multi-sector approach. JIVA’s local NGO partner, Jatan Sansthan, has been actively engaged since the project’s beginning and serves as the basis for the John Deere Foundation and PYXERA Global’s long-term exit strategy.

A Commitment to Measurement

Beyond developing a foundation of partnership for long-term sustainability, the JIVA team remains deeply committed to quantitatively evaluating not just the outputs, but more importantly the impacts of its interventions. Following a needs assessment in 2012 that informed the overall program design, a census baseline study was conducted six months later to validate previous findings and provide a baseline for comparison over the next five years. Two years later, the early results are encouraging.

In less than eight months, 100 percent of village drop-outs were enrolled in JIVA’s after-school tutoring program, 60 percent were reintegrated into government schools, and 71 percent of those reintegrated were attending class regularly. Children from castes historically ostracized and living on the outskirts of villages enrolled in school for the first time. A year later, several of these students now rank among the top-five performers in their class.

Due to JIVA’s relatively short duration, addressing complex, long-standing social challenges is not an explicit focus area. Yet with a participatory, systematic approach, it is impossible to ignore their omnipresence in everyday village life. Shayari Bagariya, a teacher in the JIVA after-school program, remarked on the changes she witnessed in her students over the last year, “I used to teach only to Bagariyas, who are my caste. Gradually, change has happened and all children have started calling me teacher. They even take water from me now. People from other castes do not like to take things from the hands of Bagariyas. Now [Bagariya and other children] sit together like brothers and sisters in the JIVA after-school tutoring program.” A small but tremendous step forward, this kind of change is atypical to see in the given time frame.  Solid impact analysis over time will enable us to learn the implications of this change for both the JIVA villages and similar future efforts.

In agriculture, demonstration plot yields increased by 34.5 percent and 48.1 percent for sorghum and maize, respectively. More than 70 percent of farmers participating in demonstration plot trainings adopted one or more improved agricultural practices before the completion of the first cropping season. Contrary to Everett Rogers’ Diffusion of Innovations Theory, the risk-taking behavior of these ‘innovators’ and ‘early adopters’—who, in theory, account for only 16 percent of the population—was not correlated to social status or financial liquidity.

“I used to teach only to Bagariyas, who are my caste. Gradually, change has happened and all children have started calling me teacher. They even take water from me now. People from other castes do not like to take things from the hands of Bagariyas. Now [Bagariya and other children] sit together like brothers and sisters in the JIVA after-school tutoring program.” —Shayari Bagariya, JIVA after-school program teacher

Village contribution to community infrastructure work, such as school toilets and classroom repairs, has totaled more than $6,000—a staggering amount for those living on less than a dollar a day. Local contracting and hiring for JIVA’s infrastructure–related construction resulted in 9,479 hired person days, in addition to the 40 full- and part-time jobs generated by JIVA.

Continuous Self-Reflection and Learning

In the face of positive outcomes, it is easy to gloss over failures. Yet, recognizing shortfall in performance is essential to course correction, and therefore must be a focus of any effective monitoring and evaluation effort. Continuous self-evaluation allows JIVA to identify and troubleshoot issues early and adapt to the local context. For example, the after-school program had great early success in enrolling the majority of village children within two months and getting drop-out students back in school. Yet month after month, many students’ test scores remained stagnant in all three villages. By re-evaluating different indicators such as school attendance and parent engagement, JIVA was able to isolate the “pull” effect some students, mainly drop-outs, were having on their peers and restructured the program accordingly. Shortly thereafter, student test scores improved markedly.

JIVA also struggled with public perception and ownership in the beginning. Initially viewed by villagers as an opportunity for free services, project staff received endless requests for hand-outs the first few months of operations. Understanding the risk this posed to long-term ownership and sustainability, JIVA quickly decided that all infrastructure work would be contingent upon village participation and monetary contribution. Less than a year later, villagers now take an active role in the planning and execution of infrastructure improvements, creating Village Development Committees to monitor construction and manage community funds.

Read this article in the print version

Neeta Kumawat, JIVA Field Coordinator and a village resident, described the transformation she’s seen in the villagers’ perception of JIVA’s work: “Earlier, people thought improving the villages was the job of JIVA and that they would do the work in any case. Now, after working for a year, there is a change in the villagers’ perspective of JIVA. Now they understand that it is not JIVA’s duty, but it is their own responsibility.”

Evaluating a community’s development is no easy task. There are innumerable confounding variables and symbiotic relationships, and causation can never be definitively proven. But systemic change requires systematic interventions. JIVA’s integrated approach recognizes that each stakeholder plays multiple, interactive roles in a community’s growth. A female farmer attending agricultural trainings is also a mother of a child enrolled in the after-school program and a wife of a carpenter building classroom desks. Evaluating a system holistically allows the project to introduce activities based on each sector’s unique and complementary developmental opportunities. With more than seventy different indicators that can be disaggregated by various socio-demographic variables, JIVA strives to piece together potential correlations and track the effects of its interventions.

“Earlier, people thought improving the villages was the job of JIVA and that they would do the work in any case. Now, after working for a year, there is a change in the villagers’ perspective of JIVA. Now they understand that it is not JIVA’s duty, but it is their own responsibility.” —Neeta Kumawat, JIVA Field Coordinator and a village resident.

Setting the Bar for “Good” Development

In some ways, JIVA is akin to thousands of projects, past and present, around the world. It is not the first time implementers have attempted to marry a multi-sector approach with locally-driven solutions, emphasizing partnerships, a mid-term duration paired with a long-term focus, robust monitoring and evaluation, and flexibility to innovate and adapt to local situations. But it is among a rarer few that have attempted all of these factors simultaneously at a micro, village level where correlation is easier to gauge.

Less than two years in, the project’s early results are impressive. Yet many questions remain unanswered. Which aspects of JIVA’s model are most important? Are they mutually inclusive? Is the model replicable? Is it sustainable? The answers to all of these questions will help determine whether or not carrying out the project in a few villages can have broader impact. As the John Deere Foundation and PYXERA Global seek to build on JIVA’s encouraging early results and identify opportunities to test the model’s scale, replication, and impact elsewhere, the answers to these questions are critical.

So, what makes a development project successful? In the absence of definitive criteria, examples of the alternative come quickly to mind. Indeed, sharing missteps is important for the success of future programs, but success is not the opposite of failure. Understanding what doesn’t work is equally as important as understanding what does. Lessons learned from projects like JIVA can help shape the expectations of future initiatives and test innovations at scale. With better goals and measurement thereof, practitioners can better evaluate the industry’s progress, where success means one day becoming obsolete.