D.C. Misra
D. C. Misra is Chief Secretary and Secretary (Agriculture), Government of Goa, Panaji, Goa, India.
A conceptual framework for monitoring
Approaches to extension monitoring
Operationalizing the definition of monitoring
Principles of monitoring
Monitoring and its main stakeholders
Frequency of monitoring
Monitoring unit
Monitoring indicators
Capability, effectiveness, efficiency, and impact
Extension monitoring indicators
Extension evaluation indicators
Action plan and chain of events
Conclusion
References
The Global Consultation on Agricultural Extension observed that monitoring and evaluation are important yet frequently neglected functions in most organizations (FAO, 1990, p. 27). In the worldwide survey of national extension systems, it was found that only about one half of all national extension systems have some type of monitoring and evaluation (M & E) capacity. The consultation noted that in many cases the M & E units are weak and are limited to ad hoc studies. Frequently, these M & E units are abandoned when project funding terminates. In many organizations, monitoring and evaluation have a negative image because these units may concentrate on problems, exposing weaknesses and failures. Instead, monitoring and evaluation should be used in a positive manner to improve extension's performance and increase its efficiency. Therefore, attitudes about and uses of M & E must be changed if this capacity is to be used to advantage in strengthening extension's performance and impact (p. 27-28).
The consultation recommended that national extension systems should be strongly encouraged to establish and use monitoring procedures and evaluation studies both to improve extension performance and to communicate the results of extension programmes to policy makers and clientele being served (p. 29).
Extension systems need to carefully consider monitoring, management information needs, sources of information, and a management information system. The word "monitor" is derived from the Latin word meaning to warn, and "evaluate" stems from the word value (Hortan, Peterson, & Ballantyne, 1993, p. 5).
Monitoring is an integral and important part of a management information system (see Chapter 17). Managers require information to keep track of an extension programme and to guide its course of action.
Management information needs include six kinds of information: (1) diagnostic information (why a situation is as it is), (2) implementation information (physical and financial or input information), (3) utilization information, (4) impact information (Murphy, 1993, p. 5-6), (5) situation information, and (6) information for review.
Sources of information cover a spectrum of sources which supply information to management, ranging from informal (say, an unscheduled encounter) to formal sources (say, a sample survey). Management should be ever alert to receive feedback on extension programmes - whatever the source. Monitoring, it is useful to realize, is only one of management's sources of information.
A management information system is a scheme by which the "right" information is obtained in the right amount at the right time and is made available to the right person or persons (Bloom, 1980, p. 28). An information system is usually created in modem organizations to cater to the information needs of management. A monitoring system is a subsystem of a management information system and has several distinguishable components.
A conceptual framework for monitoring consists of four principal components (Figure I): an extension organization, a monitoring and evaluation (M & E) unit, an information needs matrix, and a monitoring and evaluation cycle. Top management receives information from the monitoring unit and from other formal and informal sources. This influences programme implementation, leads to better programme planning, and ensures sustain-ability of extension programmes. Ultimately this leads to institutional development, which has been defined as "the process of improving the ability of institutions to make effective use of available human and financial resources" (Israel, 1987, p. 3).
Several approaches to extension monitoring are available. They arose, notably during the 1980s, as a response to the need to improve extension performance. Basically similar, these approaches, however, differ significantly in their emphasis. All of them advocate simplicity and timeliness, essential requirements of good monitoring. Half a dozen approaches are described briefly below.
Traditional (Administrative) Approach
Based on routine administrative reporting, this approach is concerned with physical and financial achievements in a programme. Its primary weaknesses include multiplicity of reports by programme personnel and absence or neglect of beneficiary contact. It is being increasingly replaced by other approaches.
Figure 1: Schematic Arrangement of Monitoring and Evaluation for Agricultural Extension Programmes
Zones-of-Concentration Approach
The Cernea and Tepping approach, introduced in 1977, concentrates on three zones: (1) visits, as the final outcome of extension efforts; (2) recommendations, as the content of the visit and means towards the end yields; and (3) yields, as the eventual consequence of the development effort (Cernea & Tepping, 1977, p. 19-20).
Methodological Approach
The Slade and Feder approach, introduced in 1981, builds upon the zones-of-concentration approach and suggests a monitoring survey early in each cropping season, a monitoring-cum-evaluation survey in each cropping season at the time of harvest, specific indicators, and reporting. Working manuals are a characteristic feature of this approach (Slade & Feder, 1985).
Expanded Monitoring Approach
Casley and Kumar suggested in 1987 an expansion of the monitoring function to cover not only physical and financial information, but also beneficiary contact information and project diagnostic studies (p. 5). Under this approach, there is greater emphasis on monitoring and less on evaluation. Project diagnostic studies are a novel feature of this approach.
Adoption Rates Approach
Murphy and Marchant suggested in 1988 an approach which concentrates on adoption rates as key indicators. This approach moves away from trying to monitor agricultural results and concentrates on directly monitoring the provision of and response to project services (p. 11). Under this approach, extension service is treated as the "medium" and the recommendations as the "message."
Marketing Approach
Lee (1990) has suggested an approach which is based on market segmentation, a standard technique in marketing. Under this approach, the need and likely demand for new technology are first assessed, and then target market segments are predicted. For example, 10 per cent of the targeted farmers are likely to readily adopt a new technology; a further 15 per cent are likely to adopt the technology when returns for wheat reach x dollars per ton; 40 per cent of the targeted farmers are likely to adopt the technology when they perceive that their peers have adopted it; and 35 per cent of the target market is not likely to adopt the technology at all in the next five years (p. 26) (enumeration supplied).
Among these approaches, Lee's approach has much to commend it, because it is based on careful assessment of the need for new technology, an exercise seldom under-taken by extension organizations.
Monitoring is a specialized, dynamic, semiautonomous, and institutionalized management resource. Preferably it is computerized. Monitoring helps to ensure the implementation of extension programmes in accordance with their design and takes into account the interests of various stakeholders.
The definition of monitoring can be operationalized by establishing principles to follow; setting up a data collection system; establishing relationships among the monitoring unit, management, the extension staff, and extension's clients (farmers); and making appropriate use of nongovernmental organizations (NGOs) (Figure 2).
In light of past experience in monitoring and its designated role, it is possible to lay down the following ten principles of monitoring:
1. Monitoring must be simple. A complex or complicated monitoring system is self-defeating. The basic task of monitoring is to simplify the field-level complexity, sifting the more important concerns from the less important.2. Monitoring must be timely. Timeliness is of the essence in monitoring. Management requires input from the monitoring system so that timely action may be taken. Also, timeliness is closely related to the credibility of monitoring itself.
3. Monitoring must be relevant. It must concern itself only with parameters which are relevant to programme objectives. This also ensures that monitoring does not generate information that is not used or is not usable by management.
4. Information provided through monitoring should be dependable. Management will rely on monitoring findings only if the information is believed to be reasonably accurate.
5. Monitoring efforts should be participatory. Effort should be made to ensure participation by all concerned with extension, be they field-level personnel, subject-matter specialists, or extension's clients (the farmers).
6. Monitoring must be flexible. It is iterative in nature. It also gets routinized with the passage of time. These two features should not, however, lead to rigidity.
7. Monitoring should be action oriented. Monitoring often leads to action. Consequently, it should follow pragmatic approaches, keeping the requirements of extension's clients uppermost in view. Generating information for which there is no intended use should be assiduously avoided.
8. Monitoring must be cost-effective. Monitoring efforts cost money and time. It is therefore essential to make it cost-effective. While principles of simplicity, time-lines, relevance, and accuracy will lead to cost-effectiveness, computerization also can help to make monitoring more cost-effective by reducing staff hours in data processing.
9. Monitoring efforts should be top management oriented. Monitoring units should keep in mind the requirements of top management when designing and operating a monitoring system. Yet at the same time, monitoring must take into account the fact that those who provide information to the system also must benefit or the quality of the information provided will decline.
10. Monitoring units represent specialised undertakings. Monitoring is not merely concerned with the collection and analysis of data, but with diagnosing problems and suggesting alternative practical solutions.
Figure 2. Organized relationship in monitoring and evaluation due to management intervention.
There are many stakeholders in monitoring efforts. The funding agency, usually the ministry of finance or planning, wants to know that money has been spent properly. The ministry of agriculture wants to be able to say it has been efficient and effective. The target beneficiaries, that is, small and marginal farmers, want help from the organization and want to improve the quality of their lives. And nongovernmental organizations (NGOs) need information generated by monitoring efforts and also want to contribute information. Thus there are four main stakeholders in monitoring - funding agencies, the implementing agency, beneficiaries, and NGOs - and their interests need to be taken into consideration when designing a system of monitoring.
Monitoring is an ongoing, continual exercise. Data collection involving crop production should be undertaken twice in a cropping season: at the time of sowing to obtain benchmark information and again at the time of harvest. The period of recall for collection of data should not be more than a month.
The monitoring unit should be staffed by technical personnel having specialized skills. The staff often consists of extension specialists, economists, sociologist or anthropologists, statisticians, computer programmers, and supporting staff. The head of the monitoring unit may come from any of these disciplines. The staff should be no larger than necessary to accomplish the work of the unit. The leader of the monitoring unit should report to one of the top managers in the organization.
Indicators, as the term suggests, are variables that help to measure changes in a given situation (ACC, 1984, p. 37). They are tools for monitoring and evaluating the effects of an activity. Indeed, indicators are the principal means by which a monitoring unit keeps track of extension's capability, effectiveness, and efficiency. Any monitoring system will therefore incorporate the use of appropriate indicators in these four aspects.
There are two approaches to indicator development: the inductive and the deductive. In the inductive approach, a system of social, economic, and demographic statistics is created and a wide range of indicators is developed on the basis of the statistics available. This is the approach followed in the case of United Nations Social Indicators (FAO, 1988, p. 5). In the deductive approach, the areas of interest are first identified, and then requisite indicators are developed. Socioeconomic indicators for the World Conference on Agrarian Reform and Social Development (WCARD) follow this approach (FAO, 1988, p. 5).
In extension monitoring, both the inductive and the deductive approaches are followed; that is, indicators are developed on the basis of available statistics, and some (for example, performance indicators) are developed as a result of specially collected data.
There are different types of indicators, for example, development indicators, socioeconomic indicators, agricultural development indicators, and extension indicators. They range from general to specific concerns.
Indicators can be categorized into direct and indirect or proxy indicators (Clayton, 1983; ACC, 1984); single and unitary or composite indicators; quantitative and qualitative indicators; primary, core, and supplementary indicators (FAO, 1988); input and output indicators; and monitoring and evaluation indicators.
The criteria for selecting indicators depend upon the purpose, resources, and time available. The following criteria are usually suggested:
· Simplicity: The indicator should be simple enough to be understood by nonspecialists (FAO, 1988, p. 8).· Unambiguous definition: It should be clearly defined (Casley & Lury, 1982, p. 31; Casley & Kumar, 1987, p. 59).
· Ready determination: The data can be obtained without undue difficulty (WHO, 1989, p. 11). This is also referred to as "timely" (ACC, 1984, p. 38) and "feasible" (FAO, 1988, p. 8; Gha, Hopkins, & McGranahan, 1988, p. 11).
· Accurate measurement: The indicator should be measured accurately, which is often difficult when dealing with farming communities (Casley & Lury, 1982, p. 32).
· Validity: The indicator should actually measure what it is supposed to measure (ACC, 1984, p. 38; FAO, 1988, p. 7; see also Gha et al., 1988, p. 13).
· Relevance: It should be geared to the specific needs of decision makers (Petry, 1983, p. 38) and be relevant to project objectives (ACC, 1984, p. 38).
· Specifity: It should reflect changes only in the situation concerned (WHO, 1989, p. 19) and should measure specific conditions that the project aims to change (Casley & Kumar, 1987, p. 59).
· Consistency: The value of indicators should stay constant so long as they are collected in identical conditions, no matter who does the collecting (Casley & Kumar, 1987, p. 69). Indicators should be objective and verifiable (FAO, 1988, p. 8).
· Sensitivity: Indicators should be sensitive to changes in the situation being observed (ACC, 1984, p. 38). They should be sensitive enough to reflect changes in the phenomenon (FAO, 1988, p. 8).
· Prioritization: Indicators should be prioritized and a minimum feasible list prepared (Gha et al., 1988, p. 11).
These four concepts are basic to monitoring and evaluation. They correspond, respectively, to operational investment (e.g., investment in agricultural extension per farm family), operational efficiency (e.g., the number of visits, meetings, demonstrations, and trials, per extension worker), technical efficiency (e.g., the number of adopters, hectorage, output, and value added), and extension-induced changes (e.g., production, productivity, income, and income distribution) (Ruthenburg, 1985, p. 120).
Capability, effectiveness, and efficiency fall in the monitoring domain. Impact falls in the evaluation domain.
Capability is the command that extension has over physical, financial, and human resources, enabling it to serve its clients (the farmers). It is reflected by extension's outreach, intensity, technical competence, and physical and financial resources. Extension performance depends directly upon its capability.
Effectiveness is defined by a handbook on productivity management as "the degree to which goals are attained" (Prokopenko, 1987, p. 9). Agricultural extension has many goals such as social goals (e.g., farmer welfare) and economic goals (e.g., increased income).
Among these, operational goals (e.g., physical and financial targets) are of special significance because their attainment makes realization of other goals possible.
Efficiency in extension is usually measured by the rates at which farmers adopt recommended practices. Adoption rates of varying degrees of complexity can be conceived (Casley & Lury, 1982, p. 37).
Impact in extension can be measured by a simple indicator, like yield of a crop per hectare, or by constructing simple productivity indices. Such indicators provide ultimate tests for the success of extension programmes.
Extension monitoring indicators can be grouped into two categories: (1) extension capability indicators, and (2) extension performance indicators. Both should be generated by the monitoring unit.
Extension Capability Indicators
Extension capability indicators must be monitored regularly not only to know the status of extension's capability at a certain point in time, but also to determine changes in it over time. These indicators should be calculated annually. They involve only desk work because they are based on in-house data.
Extension Performance Indicators
Extension performance indicators reflect extension's operational and technical efficiency. They can be grouped into two categories.
Extension Effectiveness Indicators. These can again be grouped into two subcategories: (1) single indicators and (2) unitary or composite indicators. By definition, a single indicator will reflect an aspect of extension performance, while a unitary or composite indicator will reflect two or more aspects of extension performance. It may be useful to construct a unitary or composite indicator to provide a consolidated view of extension effectiveness to management, because management is often interested in having an overall view of extension effectiveness. These indicators reflect extension's cooperational efficiency (Box 1).
Extension Efficiency Indicators. These indicators are based on adoption rates of recommended practices and reflect extension's technical efficiency (Box 2).
Extension performance indicators, as suggested here, should be calculated separately for contact farmers, other (noncontact) farmers, and all (contact and noncontact) farmers.
Yield and productivity indices occupy the central position in these groups of indicators. They are calculated on the basis of crop-cutting experiments which are conductebby provincial revenue or economics and statistics departments (Box 1).
Monitoring requires collection of data and their analysis. For this, an action plan in a monthly, quarterly, and annual timeframe is essential. Data collections will require choosing appropriate methods, which will depend upon the monitoring unit's time and physical and financial resources such as computerization and trained technical personnel for collecting and analysing data.
Monitoring methodologies can be perceived as a chain of distinguishable and sequential events comprising (1) planning and design of the study, (2) desk research, (3) selection of methods, (4) data collection and analysis, (5) report writing, (6) report presentation, and (7) follow-up action. A detailed study programme should be prepared within the prescribed time limit (Figure 3).
An array of methodologies is now available, ranging from casual, informal interactions to highly structured sample surveys, including emerging methodologies of rapid appraisals. These methodologies can be grouped into two categories: informal and formal.
Figure 3. A typical summary programme for the study. Source: Nichols (1991, p. 22).
Box 1. Extension Performance Indicators.
|
|
(I) Extension Effectiveness Indicators |
|
(a) Single Indicators | |
|
1. Awareness |
Number of farmers aware of Village Extension Worker (percentage) |
|
2. Visit |
Number of visits by Village Extension Worker to farmers a) twice a month, b) once a month, and c) no visit (average) |
|
3. Field Meetings |
Number of meetings of Village Extension Worker with farmers in their fields (percentage) |
|
4. Regularity |
Number of meetings of Village Extension Worker with farmers on the fixed day (percentage) |
|
5. Field Day |
Number of field days organized by Village Extension Worker a) in preceding month, b) quarterly, and c) annually (average) |
|
6. Demonstration |
Number of a) method demonstrations, b) result demonstrations, and c) method-cum-result demonstrations organized by Village Extension Worker(i) in preceding month,(ii) quarterly, and(iii) annually |
|
7. Supervision |
Number of supervisory visits from Agricultural Extension Officers to Village Extension Worker in the field per month (average) |
|
8. Research-Extension Linkage |
Number of research-extension linkage workshops organized per month (average) |
|
9. Farmer Training |
Number of farmers trained in farmers' training centres (institutionalized training courses) per year (average) |
|
(b) Unitary or Composite Indicators | |
|
10. Extension Effectiveness |
Arithmetic average of selected extension effectiveness indicators, say, Awareness Indicator (know the Village Extension Worker), Visit Indicator (number of visits twice a month). Field Indicator (meeting place at farmers' fields), and Regularity Indicator (visit on the same day) (Misra, 1994) |
|
|
(ii) Extension Efficiency Indicators |
|
11. Performance Index |
Actual number of farmers reached out of the target number which should be reached (Casley and Lury, 1982, p.7) (percentage) |
|
12. Penetration Index |
Number of farmers adopting the recommended practice out of the actual number reached (ibid. p.37) (percentage) |
|
13. Achievement Index |
Number adopting the recommended practice out of the target number of farmers (ibid. p.37) (percentage). Note that (13) =(12) x (11) |
|
Extension Productivity Indicators | |
|
1. Yield |
Yield per hectare for main crop(s) (average) |
|
2. Productivity Index |
Increase in yield over base year compared with base year (percentage) |
Box 2. Extension Capability Indicators.
|
1. |
Coverage |
Area under cultivation per Extension Worker |
|
2. |
Intensity |
Number of Farm Families per Extension Worker |
|
3. |
Competence |
Number of Graduate Extension Workers out of total number of Extension Workers (percentage) |
|
4. |
Subject-Matter Specialist |
Number of Subject-Matter Specialists per hundred Extension Workers |
|
5. |
Research-Extension Ratio |
Number of Agricultural Scientists per hundred Extension Workers |
|
6. |
Monitoring |
Number of Monitoring Unit Personnel per thousand Extension Workers |
|
7. |
Gender Ratio |
Number of Female Extension Personnel out of total number of Extension |
|
|
|
Personnel (percentage) |
|
8. |
Equity |
Number of Small and Marginal Farmers out of total number of Contact Farmers (percentage) |
|
9. |
Mass Contact |
Number of group meetings held per month per Extension Worker in a year(average) |
|
10. |
Computerization |
Number of personal computers in Extension Organization per thousand Extension Personnel |
|
11. |
Print Media |
Number of leaflets/pamphlets distributed per month per Extension Worker in a year (average) |
|
12. |
Audio-Visual Media |
Number of audio-visual (cinema/television) shows organized per month per Extension Worker in a year (average) |
|
13. |
Training |
Number of Extension Personnel out of total number of Extension Personnel trained in specialized training courses in a year (percentage) |
|
14. |
Finance |
Budgetary expenditure on Agricultural Extension out of total budgetary expenditure on agriculture per year (percentage) |
|
15. |
Investment |
Expenditure on Agricultural Extension as percentage of Agricultural Gross Domestic Product per year |
|
16. |
Transport |
Number of(i) bicycles, (h) motorcycles, and(iii) 4-wheel vehicles, per thousand Extension Workers |
|
Notes: |
1. Extension Personnel means total personnel engaged in extension, including Subject-Matter Specialists, whereas Extension Worker means Village Extension Worker. |
|
|
2. The data source for all the indicators will be the offices of District Agriculture Officers and Provincial Directors of Agriculture. |
|
|
3. At the national level, the indicators are required to be aggregated by the Central Monitoring Unit in the Ministry of Agriculture. |
Informal Methods
Informal methods include participant observation, case studies, key informants, individual interviews or discussions, group interviews or discussions, oral testimonoral and life histories, longitudinal studies, cross-sectional studies, interdisciplinary terms, sondeo or reconnaissance survey, diagnostic studies, rapid rural appraisal, and participatory rural appraisal (Casley & Lury, 1987; Nichols, 1991; Pratt & Loizos, 1992; Hildebrand, 1981; FAO, 1992; Beebe, 1987; Kumar, 1993). (For details on rapid rural appraisal, see chapter 6.).
Formal Methods
Formal methods include using the population and agricultural census; sample surveys such as random sampling, including simple, systematic, stratified, cluster, and multistage; nonrandom sampling, including purposive, quota, and accidental (Shaner, Phillipp, & Schmehl, 1982); and special studies.
Among these, the sample survey occupies a unique place. A sample, as a representative part of the population, enables us to draw reasonable inferences about the characteristics of the population. As such, the sample survey will constitute the "handover" of monitoring methodologies. Other methods, notably the emerging rapid rural appraisal, can be used to advantage for exploratory, diagnostic, complementary, and supplementary purposes for decision making.
Regular and systematic monitoring provides timely information to management for corrective action, justification for investment in agricultural extension, inducement to extension personnel to perform their designated tasks, a sense of participation to farmers in extension when approached for responses to questionnaires, and direction for development of future extension policies and programmes. Experience of the last two decades in extension monitoring, however, indicates that monitoring has become a routine, mechanical exercise confined to attainment of programme objectives to the neglect of the wider objectives of meeting the needs of extension's clients.
Monitoring efforts should be broadly conceptualized so that they incorporate not only farm production and productivity, but also natural resource conservation, beneficiary participation, and institution-building. This will require monitoring units to upgrade their technical competence. There is need for systematic and regular training of personnel, introduction of new electronic technology, particularly computers (see, for example, Wentling & Wentling, 1993), and appropriate use of methodologies like rapid rural appraisal that yield quick results. Investment in monitoring should then yield rich dividends through improved extension performance.
ACC (Administrative Committee on Coordination), United Nations (1984). Guiding principles for the design and of monitoring and evaluation in rural development projects and programmes. Rome: International Fund for Agricultural Development (IFAD).
Beebe, J. (1987). Rapid appraisal: The evolution of the concept and the definition of issues (p. 47-68). In Proceedings of the 1985 international conference on rapid rural appraisal. Khon Kaen, Thailand: Khon Kaen University.
Bloom, C. (1980). The managerial role (p. 21-46). In L. J. Goodman & R. N. Love (Eds.), Project planning and management: An integrated approach. New York: Pergamon Press.
Casley, D. J., & Kumar, K. (1987). Project monitoring and evaluation in agriculture. Baltimore and London: Johns Hopkins University Press (published for the World Bank).
Casley, D. J., & Lury, D. A. (1982). Monitoring and evaluation of agriculture and rural development projects. Baltimore and London: Johns Hopkins University Press (published for the World Bank).
Casley, D. J., & Lury, D. A. (1987). Monitoring and evaluation of agriculture and rural development projects. Baltimore and London: Johns Hopkins University Press (published for the World Bank).
Cernea, M. M., & Topping, B. J. (1977). A system for monitoring and evaluating agricultural extension projects. World Bank Staff Working Paper No. 272. Washington, DC: World Bank.
Clayton, E. (1983). Role, characteristics, and operational features of agricultural monitoring systems (p. 1-9). In E. Clayton & F. Petry (Eds.), Monitoring systems for agricultural and rural development projects (vol. 1). Rome: FAO.
FAO (1988). Socio-economic indicators for monitoring and evaluating agrarian reform and rural development. WCARRD: Ten years of follow-up (1979-1989). Rome: Author.
FAO (1990). Report of the global consultation on agriculture extension. Rome: Author.
FAO (1992). Sociological analysis in agricultural investment project design. FAO Investment Centre Technical Paper 9. Rome: Author.
Gha, D., Hopkins, M., & McGranahan, D. (1988). Some reflections on human and social indicators for development. Discussion Paper No. 6. Geneva: United Nations Research Institute for Social Development (UNRISD).
Hildebrand, P. E. (1981). Combining disciplines in rapid appraisal: The sondeo approach. Agricultural Administration, 8, 423-432.
Hortan, D., Peterson, W., & Ballantyne, P. (1993). M & E principles and concepts (p. 5-19). In D. Hortan et al. (Eds.), Monitoring and evaluating agricultural research: A sourcebook. Oxon, UK: CAB International.
Israel, A. (1987). Institutional development: Incentives to performance. Baltimore and London: Johns Hopkins University Press (published for the World Bank).
Kumar, K. (Ed.). (1993). An overview of rapid appraisal methods (p. 1-22). In K. Kumar (Ed.), Rapid appraisal methods. Washington, DC: World Bank.
Lee, J. W. (1990). More than accountability: Evaluating agricultural extension programmes. TRDC Publication No. 172. Amidale, Australia: University of New England, Rural Development Centre.
Misra, D. C. (1994). Agricultural extension effectiveness in India (mimeo). University of Oxford, Queen Elizabeth House, International Development Centre.
Murphy, J. (1993). Good enough, soon enough. The Rural Extension Bulletin, 1, 27-33.
Murphy, J., & Marchant, T. J. (1988). Monitoring and evaluation in extension agencies. World Bank Technical Paper No. 79. Washington, DC: World Bank.
Nichols, P. (1991). Social survey methods: Afield guide for development workers. Development Guidelines No. 6. Oxford: Oxfam.
Pétry, F. (1983). Designing a monitoring and evaluation system (p. 20-49). In E. Clayton & F. Pétry (Eds.), Monitoring systems for agricultural and rural development projects (vol. 1). Rome: FAO.
Pratt, B., & Loizos, P. (1992). Choosing research methods: Data collection for development workers. Oxford: Oxfam.
Prokopenko, J. (1987). Productivity management: A practical handbook. Geneva: International Labour Office (ILO).
Ruthenberg, I. (1985). Innovation policy for small farmers in the tropics: The economics of technical innovations for agricultural development. Oxford: Oxford University Press.
Shaner, W. W., Phillipp, P. F., & Schmehl, W. R. (1982). Farming systems research and development: Guidelines for developing countries. Boulder, CO: Westview Press.
Slade, R. H., & Feder, G. (1985). Training and visit extension: A manual of instruction (mimeo). Washington, DC: World Bank.
WHO (World Health Organization) (1989). Evaluation of programmes to ensure food safety. Geneva: Author.
