6.1 Overview
6.2 Strategic Planning, Priority Setting and Medium-Term Planning
6.3 Impact
6.4 Quality of Science
6.5 Effectiveness and Efficiency Management
6.6 CCERs - An Evaluation
The Panel's overall view of CIMMYT is extremely favourable. The Centre conducts high-quality science and has an impressive record of achievements as well as impact on the daily livelihood of hundreds of millions of rural and urban poor. CIMMYT is well managed, has strong leadership, and is a flagship centre of the CGIAR System. The Panel firmly believes that CIMMYT merits continued strong support from the donor community. CIMMYT is providing much-needed services and products for which it has a definite comparative advantage and for which there are no alternative suppliers. There is no perceived diminution in the uptake of CIMMYT's enhanced germplasm by NARS throughout the world. There are also substantial spillovers to other organizations in countries that are financial partners of the CGIAR.
The Centre is not resting on its laurels, but is taking appropriate steps to position itself strategically to meet the challenges of a changing internal and external environment. The Centre is being transformed from one of commodity-driven programmes into one of multidisciplinary projects, sometimes involving more than one commodity, that include global, regional and frontier thrusts. This requires a transition from programme- to project-based management and a major change in the way research is conducted, managed and implemented. To this end, CIMMYT is also undergoing a profound cultural change, has expanded its strategic partnerships and enhanced its external relationships. In addition, CIMMYT's funding has improved dramatically during the last two years, and the Centre remains at the forefront of advanced science.
To implement this process of change effectively, CIMMYT must continue to meet major challenges in both science and management. The Panel believes that the Centre is plotting the appropriate course to address these challenges. CIMMYT has taken effective steps to introduce project-based management and to strengthen its human resources and financial management, which were areas where improvement was long overdue. The Centre is also in the process of improving its management of information systems. In the areas of research and research support, the Panel has made several critical observations as well as recommendations and suggestions on each of the programmes, and their inter-linkages, to assist in plotting future directions. The Panel chose not to repeat these observations and recommendations here for reasons of brevity, but refers the reader to the Summary and List of Recommendations, and to Chapters 2 and 3 of this report.
Formally, CIMMYT's 1988 Strategic Plan is still valid but was regularly complemented by evergreen reports until 1993. Currently the 1998-2000 Medium-Term Plan serves as the Centre's basic document outlining its future strategic directions.
The Panel believes that CIMMYT's current mission statement to help alleviate poverty by increasing the profitability, productivity and sustainability of maize and wheat farming systems is still valid and consistent with changes in the internal and external environments.
CIMMYT's overall approach to setting priorities is presented in Section 1.5 of this report. This approach, together with TAC's report on CGIAR priorities and strategies, guided the overall resource allocation process for the MTP 1998-2000. In general, CIMMYT's priority-setting criteria can be categorized by three basic indicators: efficiency, equity and security. Based on those criteria, it was suggested that wheat should receive slightly greater investment than maize and that Asia should receive slightly greater investment than Africa. Overall, the Panel considers that CIMMYT's priorities and strategies are consistent with the CGIAR goals of poverty alleviation and protection of the environment, leading in turn to sustainable food security, and that they are highly relevant to both the rural and urban poor, particularly rural women.
It should be noted, however, that while CIMMYT's criteria were all extensively considered, this was so only in a subjective sense for the MTP 1998-2000. Although CIMMYT's priorities and strategies appear to be consistent with CGIAR goals, the links between priorities and resource allocation in CIMMYT's MTP, and from the MTP priorities to the 1998 budget, are not transparent. This lack of clarity results from the fact that CIMMYT does not use a formal priority-setting model and because a large part of the Centre's funding is of a restrictive nature. While the Panel understands CIMMYT's pragmatic approach to setting priorities and translating them into resource allocations, it also urges the Centre to proceed with its intention to design a more rational procedure for allocating resources to projects.
CIMMYT is to be commended for the impressive consultative process conducted with partners and stakeholders in the development of the MTP 1998-2000. CIMMYT's management made substantial efforts to develop more effective partnerships with NARS, NGOs, ARIs and the private sector, and to involve them more closely in the Centre's planning processes. CIMMYT has also given some consideration to issues related to alternative sources of supply, and claims (it) consciously seeks to avoid activities in which it does not have a clear comparative advantage (MTP 1998-2000). The Panel notes CIMMYT's progress to date, and encourages the Centre to continue its efforts in this direction.
On the implementation of the new project-based approach to research, the Panel was assured by management that the process for putting the proposed projects into operation is in place, using training of project staff, team building, and development of appropriate management structures. Unfortunately the Panel was able to see draft write-ups only of seven out of 21 CIMMYT projects because the necessary consultations among staff and partners had not been completed to permit finalization of the remaining write-ups. (The MTP contains project summaries only.) Within these limitations, the Panel is satisfied with the proposed project portfolio and endorses the shift towards a more multidisciplinary approach. It considers the proposed projects to be highly relevant to CIMMYT's mandate, priorities and comparative advantage. The EPMR considers that since the projects will be implemented as from 1998, it would be undesirable to make premature judgements on the effectiveness of this new approach to research management. The Panel, therefore, recommends that CIMMYT's Board of Trustees:
Commission an external review (CCER) to be conducted by mid 2000 to assess progress in implementing the project-based approach to research management as outlined in the MTP. This review should include the assessment of progress towards the attainment of development of management systems to support these projects, integration of the financial and human resource databases, the implementation of a strategy on information technology, and the operation of CIMMYT's management information systems.
The Panel had the opportunity to observe at first hand the successful involvement of CIMMYT in three regional networks in Africa, Asia and Latin America (see Section 3.6). This involvement reflects a commitment by CIMMYT to true partnerships with the NARS in the regions concerned. The Centre has evidently made a substantial impact with modest inputs. The Panel commends CIMMYT for its catalytic role in these networks and considers them to be a good model for further development.
As discussed in Chapters 1, 2 and 3, CIMMYT has made a tremendous impact on the production of wheat and maize throughout the developing world, and hence on the incomes and welfare of vast numbers of both urban and rural poor, and indirect but positive impacts on the conservation of natural resources. CIMMYT's contributions to the mission and goals of the CGIAR are truly outstanding. In particular, the impact of CIMMYT's contributions to germplasm enhancement continue to be worldwide, and research on natural resources management and economics support the efforts on germplasm. CIMMYT is to be commended on the quality of the impact studies it has conducted.
Nevertheless, the Panel questions why approximately 40% of the maize area in developing countries is still sown with unimproved varieties, nor does it have evidence that CIMMYT has seriously addressed the problem. To this end, it has suggested that the Economics Programme, in close collaboration with the Maize Programme, carefully study the reasons for the limited progress in these areas.
In addition to direct impact on the intended beneficiaries of the CGIAR, CIMMYT has also made significant contributions to science with a large number of good publications, to the preservation of biodiversity, and to improving the conservation and management of natural resources.
The quality of the applied sciences at CIMMYT, as used in the breeding programmes and by the outreach staff, when measured by output and impact, is clearly excellent. The quality of the strategic sciences at CIMMYT, including research in a wide range of disciplines that underpin the applied programmes, is more difficult to assess objectively.
One measure of the quality of CIMMYT's strategic science is the number of articles and book chapters in international refereed publications. Over the last five years, this has varied from 62 to 173, with an average of about 100/year or about 1.3 per IRS. The number of journal articles and book chapters amounted to 74 in 1992, 110 in 1993, 87 in 1994, 173 in 1995, and 62 in 1996. The output of papers has varied markedly within and between programmes. Those in the applied programmes and outreach understandably tend to publish less that those doing strategic research. Amongst the latter, the rate of publication in refereed journals averages over two per year, and for a number of individual staff members-over four per year, which is excellent.
This analysis suggests that if CIMMYT wishes to increase its output in terms of its contribution to the refereed scientific literature, then it is most likely to be able to do so by improving the opportunities for its staff in the applied programmes and outreach to publish more. This would also have the additional benefit of improving the promotional opportunities and career prospects of those staff.
The Panel is concerned with the steep decline in the number of journal articles and book chapters published by CIMMYT staff in 1996. The Panel assumes this is due to the distractions of continuing budget adjustments and the time needed for inputs by staff into the project planning process. It is clear, however, that the RCC needs to keep a watching brief in this area to ensure that the Centre maintains an acceptable level of output of scientific publications. The intention to publish needs to be included in individual work plans and progress should be monitored annually.
Another measure of the quality of science at CIMMYT is leadership in specific research areas. Each of the programmes can claim to be among the leaders in their respective areas of research. For example: the Centre's work on stress physiology and development of selection methods for stress-tolerant germplasm in maize; durable resistance to leaf rust, and wide crossing and transfer of novel genes from wide relatives in wheat; apomixis in maize; and technology adoption and impact assessment studies.
Both the publication records of individual scientists and their recognition internationally as leaders in specific research areas suggest that much of the strategic science at CIMMYT is of high quality, and this holds across all programmes.
The Panel has been impressed by CIMMYT's progress in its organization and management since the particularly difficult period of the early 1990s. The Board of Trustees has competent leadership experience and is composed of highly qualified, dedicated individuals. The Panel believes, however, that the flow of information to the Board could be improved, and that there is a need to enhance the relationships between the mission and strategy of the Centre and the decisions taken by the Board, particularly with respect to budgets.
The Director General is to be commended for initiating a change in culture and for developing a new style of participatory management. Consequently, staff morale has significantly improved and much progress has been made in the development of strategic partnerships and in improving CIMMYT's relations with the donor community. There is no doubt in the minds of the Panel members that the Director General is demonstrating very good and effective leadership. He and the programme directors have high standing in the broader CGIAR community.
The Panel commends the Centre for introducing a multidisciplinary approach to project management, sometimes involving more than one commodity, and is convinced that the proposed new way of working presents significant new opportunities and challenges. The Panel's concerns about the implementation of this approach have already been noted in Section 6.2. The Panel is also concerned about the lack of progress in introducing fall project costing but appreciates that a change is necessary in donor mentality before farther significant headway can be made. While the Panel was pleased with the general quality of the available project write-ups, it noted some inconsistency. Further thought may have to be given to the formulation of effective milestones. Overall, the Panel is satisfied that the existing project write-ups provide a good model for others to follow.
CIMMYT has made significant improvements in its human resources management, particularly in job classification studies and in performance appraisal. The Centre should also be commended for its efforts in incorporating gender concerns and introducing new approaches to developing a more conducive and pleasant work environment, to participatory management, and to the encouragement of team building.
The area of financial management has been one of the most difficult challenges in CIMMYT's administration, and difficulties identified at the time of the IER in 1993 and of the ICER of 1995 have still not been fully overcome. The Panel considers that until the end of 1995, insufficient action was taken to address the shortcomings effectively. The Panel is pleased to note that, since then, much has been done to improve the credibility of CIMMYT's financial systems, but further improvements are necessary in the accounting systems and in the development of project-based information systems. Moreover, there is also still considerable room for improvement in the integration of Headquarters and regional financial systems and for the introduction of fall project costing. The Panel also considers that urgent consideration should be given to linking the accounting and human resources databases.
On information management, CIMMYT has an enviable record for the information resources it has generated over the years and the way that information has been shared. There is an urgent need, however, now being addressed by CIMMYT, to develop strategies and procedures for the management of these information sources and the related computer systems. In particular, CIMMYT will need to make a major effort to improve its management information systems.
Overall, the Panel believes that the quality of organization and management at CIMMYT is good, although there are significant areas where progress can be made. As noted in Section 6.2, the Panel has recommended that an external review be conducted by mid 2000 to assess progress in implementing the project-based approach to management.
The CCERs are an important management tool for ensuring that the Centre's Board and Management periodically obtain external assessments of ongoing work of the Centre. Often these reports highlight issues and opportunities for substantially strengthening or re-directing programme activities and initiatives in the management area. In some cases, the centres have also commissioned strategic reviews of particular aspects of work; but as in the case of CIMMYT, this is somewhat rare. The Panel's comments on specific CCERs undertaken in the past few years are included in the sections of the report dealing with those particular issues.
Another major aspect required to be covered by the Panel is the role and-value of CCERs in the EPMR process. The Centre has commissioned 11 CCERs since 1993 (inclusive), the year of the IER. These were distributed amongst programmes as follows:
|
|
Number |
Years |
|
Maize Programme (MP) |
6 |
1993-2; 1994-2; 1995; 1997 |
|
Wheat Programme (WP) |
2 |
1993; 1994 |
|
Economics Programme (EP) |
2 |
1994; 1997 |
|
Applied Biotechnology Centre (ABC) |
0 |
- |
|
Natural Resources Group (NRG) |
0 |
- |
|
Management |
1 |
1995 |
Clearly the distribution of the CCERs amongst programmes is very uneven. The lack of CCERs for the ABC and NRG is understandable given they are "new" in the sense that they were accorded programme status only in 1995 and 1996, respectively. The uneven distribution among the "older" programmes, especially the larger ones, WP, MP and Management, is more difficult to explain.
The scope of the CCERs has varied markedly. Only those of the EP and of Management dealt with the programme-specific area as a whole. Those in the MP and WP were narrower in their focus, and in all cases dealt with subprogrammes, or in one case in the MP, an individual project.
The quality of the reports also varied substantially. Some were relatively superficial, and some were more analytical. Some review teams interpreted their terms of reference narrowly, thereby adding to the problem of limited scope, while others were more ambitious and expansive. The size of the review panels varied from three to five members (in the cases where this is documented in the reports) and usually included a member of the BoT as a " resource person". This meant that in the case of the reviews of the EP, for example, there were only two independent reviewers on the panel.
In all CCERs, the review panel had formal meetings and/or information exchange sessions with concerned staff. However, where lists of documents are included in the reports, it is clear that the documentation often falls short of what would be required for an EPMR.
For the great majority of the cases, the Centre has responded in writing to the recommendations of the reports and in many cases has implemented those recommendations. The reports sometimes provided a useful input to the EPMR because, for some aspects of the programmes, they gave a view on whether specific issues were long standing and on whether specific recommendations had been made in relation to them in previous reviews.
However, it should be pointed out that their use for the EPMR Panel was limited. First, the topics covered by the reviews were limited and apparently arbitrary. Topic selection appears to have been driven by the needs of the management for outside opinions on perceived problem areas, rather than by a perceived need for a systematic review of whole programmes. The restricted time allowance for some reviews and the small number of independent panel members also limited their value. In the Panel's view, the CCERs do not substitute for the provision of full documentation to the EPMRs, although they may still be useful as peer review documents for particular areas of work. Further, their narrow focus reduces their value in examining issues of balance between and within programmes. Finally, the Panel is unable to agree with some of the conclusions of the 1995 CCER of Management, Organization and Structure, in particular with the statement that CIMMYT was an exemplary Centre from the point of view of management.
The Panel suggests that if CCERs are to be seen as an aid to the EPMR process, they should:
i) consider only whole programmes or two or more related programmes;ii) not be obsolete;
iii) not involve Trustees but be composed of independent members;
iv) ensure that the documentation provided to the CCERs, with updates if necessary, are provided to EPMR Panel members during the initial phase;
v) be penetrating and comprehensive.
In short, while often helpful to the Centre, to critically aid an EPMR, the CCER process needs to be more systematic, relevant and directed than the process implemented by CIMMYT.
