3. RESULTS

The literature survey, in combination with the criteria for selecting “plausible” analyses, results in a selection of 15 studies for the more comprehensive aggregate benefit values, of which a smaller subset is rated as “significantly demonstrated” (Table 3). Application of the review principles, criteria and indicators to the studies reporting the five largest benefit values is illustrated in Table 6, while key characteristics of all 15 studies are described in Appendix II.

As illustrated by Table 4, only a small sampling of the probable total benefits generated by the CGIAR is represented by even the most comprehensive assortment of studies. The small sample of research programmes included and limited time period covered by the studies results in limited potential for comparative evaluation, so such is not a major emphasis within the present analysis. Cumulative profiles of benefits produced over time under the five scenarios are shown in Figure 8, against the investments in the System, which total $7.12 billion by the end of 2001 (in 1990 dollars). Aggregate annual benefit streams under different scenarios and discount rates are presented in Table 8, while overall benefit-cost ratios are depicted in Figure 9.

Table 3. Numbers of studies included in the five scenarios of the present meta-analysis

1. “Significantly demonstrated & empirically attributed” scenario

To be extremely conservative, and go beyond commonly accepted impact analysis standards, only “significantly demonstrated” analyses that include empirically-derived attribution of CGIAR contributions to impacts may be included in an aggregate estimate of benefits. Application of this very restrictive standard allows four^[7] studies to be included, which represent four of the Centres. One of these studies utilized computer modelling techniques, while the remaining three used economic-surplus measures. Collectively, these studies produce a benefit-cost ratio of 1.94 (with a 2% real discount rate), which is substantially higher than unity, and a respectable internal rate of return of 17.29% (Table 7; Figure 10). With no discounting applied this ratio becomes 1.95, and if a 10% rate is used, the ratio produced is 1.56.

This type of estimate is of higher confidence than estimates utilising assumed attributive coefficients, but since attribution by institution has not been a recommended IA practice, not many studies fulfilled this criterion, and several robust analyses were omitted. Still, the likely contributions of collaborating organizations may be underestimated through assumed attributive coefficients, and an empirical basis provides much more solid grounding for assessing CGIAR additionality.

2. “Significantly demonstrated” scenario

The very conservative small sampling of “significantly demonstrated” assessments within the System includes seven studies representing six of the Centres. Two of these studies utilized computer modelling techniques, while the remaining five used economic-surplus measures.

Those studies classified as “plausible” but not “significantly demonstrated” may indeed encompass comparable rigour, but such was not clearly evident in the publication, as methodological details often remained unclear. To be extremely conservative, such opacities were denied “the benefit of the doubt,” and studies without clearly presented analytical details, such as survey sample sizes, or types of data sources for productivity estimates were excluded from the “significantly demonstrated” scenario. Where methodological details were more apparent, a few studies were excluded on the basis of potential significant subjective bias in key data sources, such as estimates of adoption or productivity impacts based chiefly on the testimony of a single expert or on a very small number of beneficiary interviews. The methodological sophistication of “significantly demonstrated” studies is higher, in general, than is the sophistication of “plausible” studies, as productivity impacts were disaggregated by agro-ecological conditions in all of these studies, while few “plausible” studies took this measure.

The bulk of values reported in this scenario have been generated by three key innovations - modern varieties of rice, modern varieties of wheat and cassava-mealybug biocontrol, as such comprise 98.1% of “significantly demonstrated” benefits. After discounting and nominal adjustment, this scenario produced a benefit cost ratio of 3.77, which is considerably higher than unity. Sensitivity analyses that raise the social discount rate to 10% and lower it to zero result in ratios of 3.40 and 3.72, respectively^[8]. Even with this highly conservative sampling and sensitivity analysis, the ratio produced is a substantial multiple of unity, and the IRR of 33.91% is considerable.

3. “Plausible” scenario

Within the broader “plausible” assortment of 15 studies, 11 focussed on crop genetic improvement, two on biological control of pests, one on policy guidance and one on seed production technology. Nine of the 16 IARCs are represented in the selected studies. Twelve studies utilized simple economic-surplus measures, while two used simulation software, and a final study used econometric techniques. All but two of the studies have been or are in the process of being published in peer-reviewed scientific books or journals. The assortment of studies included under this standard appears in Table 4, and additional details on the studies producing the five largest benefit values are presented in Table 5.

Table 4. "Plausible" aggregate benefit estimates and periods of coverage within the scope of the CGIAR research agenda. Black shading indicates relatively comprehensive geographic coverage, while grey indicates partial coverage for values included. Note that the reported values have been adjusted for attribution, deflated and discounted (2% real rate) to a 1990 base year.

From the included assessments, it is clear that spring bread wheat breeding, rice genetic improvement and cassava-mealybug biocontrol have had particularly high research payoffs, as these three research programmes have collectively generated 93.4% of this wider array of estimated benefits (Figure 7). Although comparisons are not possible due to the lack of analyses for other programmes, it is clear that research in these three areas had pervasive and large-scale impacts on wide regions of the globe.

Figure 7. Percentage of benefits derived from different research areas in the scenario of "plausible" studies, with no extrapolation of benefit values reported

In scenario 3, the benefit-cost ratio produced with a 2% real discount rate and nominal adjustment is 4.76 (Figure 9), with a rate of return of 34.01%. This value is considerable according to standard investment analysis. Raising the real discount rate to 10% results in a B-C ratio of 3.78, while lowering it to zero results in 4.91, indicating only modest sensitivity to the discount rate applied.

Table 5. Characteristics of the studies producing the five highest benefit values in the B-C Meta-Analysis. Collectively, these studies account for 90.4% of benefits under the "plausible” scenario (with no extrapolation).

Table 6. Basis of ratings for the five studies producing the highest benefit values in the B-C Meta-Analysis. Collectively, these studies account for 90.4% of benefits under the "plausible” scenario (with no extrapolation)

Figure 8. Cumulative estimates of benefits from and investments by the Consultative Group on International Agricultural Research in the activities of the International Agricultural Research Centres over time, according to different scenarios of study selection

4. “Plausible, extrapolated to the present” scenario

The benefits derived could also be plausibly extrapolated under the assumption that single-year benefit estimates for modern varieties also represent years immediately subsequent to the year of the estimate (i.e. a benefit estimate based on 1998 data may also apply to 1999, 2000 and 2001). Furthermore, benefit estimates over many years representing a progressively rising trend until the terminal year of the assessment, may be conservatively assumed to continue at the benefit level of the final year until the present. Benefits derived, particularly for modern varieties, cannot be expected to disappear after the end of the estimated period, as germplasm has a useful lifespan of at least a decade, even if no further improvements are made (germplasm improvement constitutes the bulk of estimated benefits). Furthermore, rising benefit trends for all of the multiyear genetic improvement studies indicate that productivity gains continue to be made, so even this may be regarded as a conservative conjecture. Under these assumptions, the benefit-cost ratio rises to 9.00, with an IRR of 34.41%. With no discounting applied, the ratio is 10.03, and if a 10% discount rate is utilized, the ratio becomes 5.40.

5. “Plausible, extrapolated through 2011” scenario

To go further, presently conducted research has a significant lag period before benefits are generated, as the core IARC innovation must be locally adapted by NARS, and often must be approved by national governments before widespread dissemination is pursued. Furthermore, the innovation must then have time to diffuse among farmers and be adopted before benefits are generated. In general, this process usually takes at least one decade. Thus, the bulk of benefits measured in these studies were generated by research that happened at least a decade ago, and most research conducted in the present will not have impact until at least ten years from now. Therefore, it can be assumed that if returns to research conducted in these specific areas have been at least constant over the past decade, benefits could continue unabated until at least ten years into the future, if research stopped at present.

Such continued benefits may be considered as a reasonable, although somewhat speculative assumption, based on observations regarding the continued relevance of assessed innovations. The impact of an innovation persists until the innovation becomes obsolete or replaced by antecedent alternative technology (Alston et al., 1998a). Accordingly, it can be plausibly expected that semi-dwarfism and disease resistance will not become obsolete in the next decade. For instance, adoption of IARC derived varieties of wheat and rice, which comprise most of the assessed benefits, has continued to rise in recent years, and shows no sign of decreasing, according to the reviewed literature (Heisey et al., 2003; Hossain et al., 2003). Furthermore, yield gains from past research have largely been maintained or expanded through second and third generation varietal replacement, which have fostered increased disease resistance (Byerlee and Traxler, 1995). In addition, biocontrol of the cassava-mealybug has been predicted to remain viable until at least 2013 (Zeddies et al., 2001). However, it should be stressed that while this assumption may be regarded as a more complete estimation of benefits from investments to-date, such is a very tenuous and rough approximation of potential events.

If this speculative assumption is combined with application of the expected future results of cassava mealybug biocontrol, the benefit-cost ratio rises to an outstanding 17.26, with 122.9 billion (1990) dollars of benefits generated (Figure 8). When the real discount rate is raised to 10%, this ratio falls to 7.07, and with no discounting it rises to 21.83. The internal rate of return produced by these benefits is not much higher than the other scenarios at 34.48%, due to the fact that most benefits occur long after investments.

Table 7. Internal rates of return produced by benefit values aggregated from large-scale economic impact assessments of CGIAR research, under five scenarios of study selection

Figure 9. Aggregate ratios of benefits derived from research and investment by the Consultative Group on International Agricultural Research in the activities of the International Agricultural Research Centres, as compared with costs, over time, under different scenarios of study selection

Figure 10. Effect of discount rate on aggregate benefit-cost ratios produced under different scenarios of study selection

Table 8. Table of benefit estimates aggregated from values reported in large-scale economic impact studies of CGIAR research, under different scenarios of study selection and interest rates (all values in millions of 1990 US dollars)