The Consultative Group on International Agricultural Research (CGIAR) was formed in 1971 to help foster technical solutions to production-related constraints affecting developing-country agriculture, through international applied research activities. Initially composed of four commodity-oriented International Agricultural Research Centres (IARCs), funded by three multilateral cosponsors, several non-governmental organizations and a smattering of bilateral donors, the visible successes of the System allowed it to quickly expand in size and scope of research agenda. Presently, the Group has 62 members supporting 16 IARCs, with research foci ranging from crop breeding to forest policy and strengthening of National Agricultural Research Systems (NARS).
Since establishment, the CGIAR community has invested approximately US $ 6.9[3] billion (2001 inclusive, in 1990 dollars) in various research and research related activities. According to Anderson (1998), the Group has been recently “threatened by a downturn in total real commitments that mismatches the increased demand arising from the recent expansion of the scope of the system.” Despite widely acknowledged successes, funding for the CGIAR has stagnated in real from 1988 through 2001. Such a trend may imply that economic impact assessments conducted for the System thus far have not adequately convinced donor audiences that the System portfolio is an exceptionally efficient and productive investment. Accordingly, one principal point of scepticism expressed by donors regarding previous evidence of efficacy is that such have “cherry picked” research successes, while ignoring the costs of failures (TAC Secretariat, 2001). The present analysis offers an answer to such criticism by compiling highly reliable estimates of widely-recognized benefits, and setting such against total investments in the System to-date.
The question addressed in this study is: Do the documented benefits from CGIAR research justify the total investment in the CGIAR so far? Thus, the basic objective is to derive a set of plausible and highly credible aggregate estimates of the benefits accruing from System innovations, and to set such against the present value of the entire CGIAR expenditure. In so doing, all undocumented benefits or documented benefits that do not meet certain criteria for selection make no contribution to the numerator of aggregate benefit-cost ratios, while all System costs (including those of facilities and related activities, such as communications or training) are included in the denominator. Consequently, the present approach is biased towards conservatism by the fact that impact assessment has only been applied to a small proportion of CGIAR research activities. Ratios derived through this method should provide meaningful insights into the aggregate effectiveness and efficiency of the CGIAR investment. Most importantly, these estimates avoid the common criticism that only successful projects are often compared, while the costs of the unsuccessful are ignored.
The overall approach used is presented in Figure 1. This study begins by providing background on methods typically applied for assessing the impacts of agricultural research, particularly in terms of basic conceptual approaches behind economic impact assessments. The methodology applied in the present study is subsequently described, beginning with the means by which studies were identified. Next, a model of “best practices” for economic impact assessments (IAs) is developed, so as to have a consistent basis for reviewing the diverse set of impact assessments identified. These “best practices” are then used a basis for developing specific criteria and indicators for assessing study quality. After the development of these criteria, the aggregation process for benefit values produced in individual studies, as well as the limitations of the approach taken in the present study are presented as a final step in the methodology section. Subsequently, the results section describes key outcomes, including benefit-cost ratios under six scenarios. Then, the discussion section examines the significance and likely accuracy of the benefit-cost ratios produced under the different scenarios. In turn, the discussion focuses on the distribution of benefits generated and the pathways by which such diffused. Next, key aspects of and trends in the quality of the reviewed studies are discussed. Finally, the conclusions section draws upon key points of the discussion to recommend future actions for the CGIAR and its Standing Panel on Impact Assessment (SPIA).
Figure 1. Stylized analytical and methodological pathway of the benefit-cost meta-analysis of the Consultative Group on International Agricultural Research
To provide some background for the methodology applied in the present meta-analysis, it is illustrative to offer an overview of the general economic impact estimation techniques applied to agricultural research. In order to delve into impact estimation methodologies, it is first helpful to explain the generic impact pathway model conceived for productivity-enhancing research. Schultz (1964) was one of the primary economists to first argue that subsistence agricultural production systems in the developing world are technically efficient, and that farmers maximize profit given available technology. This is a principal premise behind the benefits envisioned as a result of agricultural research, as if producers are efficient optimizers, only by raising the production frontier through the transfer of new technology will productivity be improved. Research thus offers a key means to develop new technologies which will raise this productivity frontier, so that higher quantities of agricultural products will be supplied at any given price, and that this enhanced supply will drive down prices. Consequently, this improved producer income through higher productivity, coupled with lower prices, which raise consumer purchasing power, will underpin economy-wide growth. Agricultural development as the engine underpinning broad economic progress has now been accepted as a central tenet of modern development theory, as indicated by Shultz’s reception of the Nobel Prize in 1979 (for these and subsequent ideas).
Generally, economic impact, as commonly assessed for agricultural research at large spatial scales, is not on poverty, but is a quantification of gains resulting from productivity improvement. It is often taken as granted that such productivity increases will foster gains throughout the broader target economies, and thereby achieve ultimate goals of poverty alleviation. Thus, the values presented are, in most cases, quantifying intermediate, rather than mission-level impacts. In this sense, impact assessment for agricultural research differs from the World Bank’s definition of IA as “intended to determine more broadly whether the program had the desired effects on individuals, households and institutions and whether those effects are attributable to the program intervention” (Baker, 2000). This is not to say, however, that this mission-level focus has not been investigated in impact assessments conducted within the System, as numerous studies have explored impacts of more immediate proximity to poverty than gross research benefits (David and Otsuka, 1994; Hazell and Haddad, 2001; Hazell and Ramasamy, 1991; Kerr and Kolavalli, 1999; Renkow, 1993). However, these are largely case studies, and there are few well developed methods by which such impacts may be scaled up to large geographic areas or represented by singular aggregate statistics.
Within the CGIAR, there has been no strong formal institutional control mechanism for impact assessment at the System level. Integration and guidance at the System level were initiated with the inauguration of the Impact Assessment and Evaluation Group (presently entitled the Standing Panel on Impact Assessment, or SPIA) in 1995, prior to which the individual IARCs developed their own IA standards, approaches and techniques relatively autonomously to deal with their own research. Thus, there is great diversity in standards and techniques applied in the Centres.
Most economic impact assessments of agricultural research rely on economic surplus techniques, which in some cases are fed by the results of econometric methods. Economic surplus techniques build upon the approach first utilized by Griliches (1957) in his pioneering study on hybrid corn, in which adoption of a technological innovation fosters a downward shift in the supply curve, through reductions in the unit cost of production. Costs per unit of production may be lessened through loss reductions or increased yield potential. The value of this per hectare increase in productivity attributable to the innovation, multiplied by the adopting area planted during a single year, then gives the gross annual research benefits. Figure 2 illustrates the market effects of increased productivity for a simple “closed economy.” The counterfactual scenario is embodied in continued application of the previous, or next-best technology, represented as supply curve S0, while the curve with the innovation applied is S1. Benefits may be presented in an aggregate social form, including changes to producer as well as consumer surplus[4], or if price elasticities of demand and supply are utilized, may be partitioned between producer and consumer groups. Benefits to the latter are usually determined to be prevalent through reductions in food prices for “closed economy” models, represented here as Q0*DP + (DQ*DP)/2.
Source: Alston and Pardey 1996.
Figure 2. Representation of a general model for valuing technological innovation within a "closed economy"
Open-economy models are also often used, and these do not base benefits on consumer price reduction, but assume that supply does not affect average prices, due to the presence of imports. Under such assumptions, benefits are measured through the value of increased production or inputs saved per unit of production, and are implicitly assumed to be received by producers.
Alternatively, econometric techniques allow for arguably more precise estimation of research benefits through explicit specification of endogeneity versus exogeneity of included variables during statistical analysis. However, econometric techniques are also arguably more liable to the assumptions underlying the model of relationships utilized, as spurious correlations may be interpreted to imply causality. In addition, the step of valuing additional output or savings in inputs essentially still relies upon an implicit economic surplus analysis (Alston et al., 1996). In this regard, this approach is a derivative of the economic surplus technique, rather than a separate set of methods, as some authors may contend.
The scope of economic impact assessments is usually that of a single innovation or series of innovations. Since research is an uncertain process characterized by many “dry holes” producing little beneficial impact, and a few “gushing wells” producing substantial benefits, analysis at the project or programme level cannot necessarily provide meaningful insights at the System level, unless the costs of associated unproductive investments are considered, as well. Accordingly, the CGIAR’s Task Force on Impact Assessment recommended that there was need for impact assessment at the System level as well as at the Centre level, to ensure that the costs of both successful and unsuccessful projects within the CGIAR are included in assessing research impact (Özgediz, 1995).[5] The present study should help to operationalize this recommendation.
|
[3] Additional investments are made in the first four IARCs prior to the CGIAR’s establishment, and those relevant to the present analysis are included in cost figures cited later in this document, which appear as 7,120 million 1990 US Dollars. [4] If elasticities are not utilised, annual prices or an average price for the period of benefit estimates may be often used to encompass the price effects of supply shifts attributable to research, as total benefits are not necessarily sensitive to elasticities of supply and demand (Alston et al., 1996). [5] The term “unsuccessful”
is used in the economic sense related to intended ultimate beneficiaries.
It is of course true that such “unsuccessful” research can be
highly successful from the point of view of expanding the general scope
of scientific knowledge, which in turn may provide the necessary foundation
for future discoveries. |
