Workshop evaluation

prepared by

P. H. Starkey and M. P. Starkey

Animal Traction Development, Reading, United Kingdom

Introduction

During the final session of the workshop, evaluation forms in French and English were distributed to all participants present. People were told that the evaluation would be anonymous and that they had about 20 minutes to complete the questions. The forms included 30 fixed-response questions, in which participants were asked to rate specific aspects of the workshop on a five-point scale. People were also encouraged to add comments to clarify or expand their answers. There were also seven open-response questions asking participants to comment more generally on the workshop, and to provide suggestions for the organization of a future workshop. The final question "Any other comments?" was completely open and participants were encouraged to provide as much feedback as possible, using the back of the forms if the designated space was insufficient.

Technical content (Q1-13)

The results of the first 13 questions concerning the technical content are shown in the histograms 1-13. Many histograms appear to reflect a normal distribution with a median between acceptable and good. The field trip appears to reflect a bimodal distribution, which is probably explained by the fact that one of the eight groups had a particularly frustrating time due to the holding of a funeral in their designated village. In order to gain some rough comparisons, the five-point scale was converted to an average score, by ascribing the values 1-5 to the letters A-E. Based on this, the activity given the best rating overall was the discussion in small groups. This was followed by the keynote address, the field visits, the presentation of small group reports and the networking session of brief introductions. No activities had an "average" below the intermediate point between acceptable and good, but those that were rated lowest were the presentation of the Senegalese experience, the introduction to the field visits and the open plenary discussion on the penultimate day. From other comments written on the forms, the reasons for the relative dissatisfaction for these items appear to be as follows:

· the Senegalese team presentation had not been coordinated beforehand, and individuals attempted to present too much information for the time available;

· due to the overrunning of the Senegalese papers, there was insufficient time for an adequate introduction to the field visits;

· the plenary session on the penultimate day lacked any lively, critical debate and the chairman of the session allowed several participants to speak at great length.

Organization and management (Q14-25)

Responses to the questions concerning the preparation for the workshop, management of the workshop, the translation services and the hotel facilities are shown in histograms 15-25. (The question numbered 14 on the evaluation was actually an introduction to the subsequent questions, and did not solicit or receive any responses). When the responses to questions 15-25 were scored they showed the highest average ratings were given for the content of workshop information forms, the reproduction of workshop documents and the level of pre-workshop publicity. The lowest score in the workshop evaluation (with an average approximating to "acceptable") was given to the hotel services, and this was followed by the quality of the simultaneous translation service. Comments on the hotel suggested the main problems were repetitive menus, poor timing and food contamination. There were no annotations to explain the relatively poor rating of the translation service, and it is not clear whether the lack of acclaim was mainly attributable to the individual translators, or the equipment, or both.

Participants (Q26-28)

Responses to the questions concerning the level, range and suitability of the participants are shown in histograms 26-28. The suggestion of these histograms and the "average scores" is that people felt the participants were generally of suitable calibre and experience. No one considered there to have been too few participants, 56 people thought the number was about right, and four respondents felt there were too many participants.

There were 50 responses to the questions concerning the conditions (if any) that should be imposed for a subsequent workshop to ensure that participant numbers and quality were appropriate. 11 participants thought they should be the same as imposed for the present workshop (which included active involvement in animal traction activities, mandatory preparation of a paper and certain limits per organization and country). Several participants referred to only one of these conditions including active involvement in animal traction (10), submission and screening of papers (8) and limits imposed on organizations and countries (3). Other suggestions included narrowing the workshop scope and themes (3), submission of participant profiles (3), announcing the workshop well in advance and specifying the calibre of participant required (3), previous participation in a network activity (2) and introduction of a registration fee to reduce "no-shows" (1). In addition five people considered that there should be no limits at all, and that all who wished to attend should be allowed to do so.

Open response questions

Three open questions asked participants their opinion as to the most useful and the least useful aspects of the workshop, and solicited suggestions on how it could have been improved.

Most useful aspect (Q33)

All but two participants gave some answer to the question on the best part of the workshop. Since several people mentioned two topics, the numbers that follow refer to the actual citations (N=96). Answers clustered around eight topics, of which three were clearly dominant. Over half the respondents (34) considered the field trip to have been most useful. 18 people gave a general reply to the effect that it was the information exchange, formal or informal, that had been most valuable. 15 participants cited the small group discussions as having been most useful. Other people referred to the thematic presentations (8), discussion of the field trip (5), open discussions (4) and discussion of the future of the network (3). Three participants simply stated everything was useful.

Least useful aspect (Q34)

Only half the respondents (29) actually cited a particular aspect of the workshop that they considered to have been the least useful. One quarter of respondents (16) made some positive comment to the effect that all parts of the workshop had been valuable. A further quarter of participants (16) left the question blank. Some people noted more than one item (total citations were 34), but no single item clearly stood out as the least useful item. There were seven small clusters of answers, and many single citations of programme elements. The aspects noted by more than one person were the plenary sessions in general (5) and plenary discussions (5), the thematic presentations (3), the field trips (3), the presentations by the Senegalese team (3), the plenary discussion that followed the small-group presentations (2) and discussions on the future of the network (2). From these responses (and from the additional comments made by the respondents), it would seem that some of the open plenary discussions were felt to be frustrating when individuals were allowed to talk at length without clearly defining their arguments. It can be concluded from the various comments written on their forms that the three participants who considered that the field trip had been the least valuable aspect of the workshop had all been in the same group. This was the group that had travelled very far only to find the village due to be visited was effectively closed for the day due to a funeral.

Improvements (Q35)

50 people responded to the question on how the workshop could have been improved. Three simply made positive comments to the effect that the workshop was good as it was, while 47 participants provided a total of 66 suggestions for the improvement of the workshop. Many participants made similar recommendations, and two main clusters and several small clusters emerge from the evaluation.

Fifteen participants cited the importance of having firm chairmen, both to ensure session timing and to prevent speakers and questioners from overrunning their allotted time. Partly related to this was the desirability (noted by four people) of more active, detailed and rigorous discussion.

The second cluster of recommendations proposed more structure for the thematic presentations. These could have been broader or more specific, but clearly differentiated and more clearly grouped in the programme. Four people considered that the technical content of the thematic presentations could have been improved by greater insistence on the early submission and screening of papers. Four people made suggestions for better presentation of information by speakers and greater use of visual aids including videos.

Five participants suggested the field visits should have been to farms closer to the workshop location, and two others felt more time in the programme should have been allocated to the field visits. Five people suggested a less exhausting programme. Four participants felt the workshop could have been improved by allowing more time for special interest groups to meet, or by dividing the participants into four subworkshops to tackle the four workshop themes. There were suggestions that there should be fewer plenary sessions (2), more plenary sessions (1), more opportunity for individuals to talk to the plenary session (2), more time for group discussions (1). The workshop could have been longer (2) or shorter (2). Other individual comments related to improved social facilities for participants (2), wider involvement in workshop organizations (1), and the need for a specific secretariat for the rapporteurs (1).

Follow up and other comments (Q36-39)

All respondents (61) thought there should be a fourth workshop. 58 participants expressed a wish to come to the next workshop, one person did not wish to come and two did not answer that question. The ideal size of the workshop varied considerably around an average of about 70. The preferences can be grouped in the ranges 30-49 (8), 50-69 (15), 70-89 (15), 90-119 (9), and no upper limit (4).

Suggestions for the next workshop were provided by 45 participants. Many of the ideas proposed reflected comments made in response to other questions in the evaluation form. Nineteen respondents suggested that the next workshop should be similar to the present one, but with specific enhancements. These included "sharper" organization (1), with stronger chairmen (3), more rigorous and critical debate (2), and more use of films, models and visual aids (3). Several people stressed the value of the informal contact, referring to advantages of a loose organizational framework to create a forum for ideas (2), and to the desirability of having a less full schedule and more time for informal meetings (5) preferably with translators available (1). One person suggested the workshop should be a day longer, another thought it should be a day shorter. Two people suggested a longer time could be spent on field trips, although from comments of some other participants, it would seem that any extra time should be spent in villages and farms rather than travelling. It was suggested that the proceeding of this and previous workshops should be well distributed, discussed and reviewed prior to the next workshop (2), and that committee members should participate in meetings of other networks before the next networkshop (2). Planning, publicity and communications with potential participants should start earlier (2), and papers should be received and reviewed before the workshop commences (2). Two people felt that discussion groups should tackle individual technical themes. It was suggested that the next workshop be held in a landlocked Sahelian country (2), at a better quality (1) or cheaper (1) hotel, in July (1), February (1) or mid-rainy season (1).

Several suggestions were provided for themes or subthemes for the next workshop, or for smaller specialized workshops. They included:

· Animal traction and erosion control (4);
· Animal traction in "improved" and integrated farming systems (3);
· Animal traction and training needs (2);
· Social and economic aspects of using animal traction (2);
· The use of animal power for gears, pumps and stationery machines (2);
· Systems of linking animals and equipment (harnesses etc.) (2);
· Women and animal traction (1);
· Regional cooperation in the production of equipment (1);
· Diversification of national mechanization policies (1);
· Nutrition and health of draft animals (1);
· Weeding with animal traction (1).

Seven participants ended their papers with a note of thanks to the organizers.

Conclusion

The evaluation forms suggest a very positive overall reaction of the majority of the participants to the third regional animal traction workshop. This is indicated by the generally high level of usefulness ascribed to the different programme components (Q1-13), the clear preference of participants to identify the most useful rather than the least useful aspects of the workshop (Q33-34), the number of specifically positive comments (Q33-35, 39 and "Any other comments"), and the unanimous desire for a fourth workshop (Q36). The majority of participants found the field trip and the subsequent small group discussions particularly useful, although one group found the field visit frustrating. There was no consensus on how the balance of programme elements could have been improved, and since increasing one item would have implied decreasing another item, it may be assumed that the programme adopted was a reasonable compromise between the various preferences of the different participants It was apparent that a number of the participants were frustrated when people were allowed people to speak at too great a length, and chairmen did not encourage critical debate. Similarly, many participants felt the plenary presentations would have benefited from a more clearly structured thematic framework. In general participants were very satisfied with workshop documentation and satisfied with general workshop organization. Many participants felt let down by certain hotel services, by the translation service, and by problems associated with reconfirmation of tickets. The majority of participants provided constructive comments and suggestions to help the organizers of subsequent network activities. While some of these would be incompatible (longer or shorter duration of the workshop or its constituent elements), most could be usefully incorporated into the planning of a fourth regional animal traction workshop.

Question 1.

How useful did you find the opening ceremony and introduction to the workshop? (Thursday 7 July)

De quelle utilité avez-vous trouvé la cérémonie d'ouverture et l'introduction à l'atelier? (Jeudi 7 juillet)

Response 1 - Réponse 1

Question 2.

How would you rate the keynote address on the workshop themes? (Thursday 7 July)

De quelle utilité avez-vous trouvé le résumé des thèmes de l'atelier? (Jeudi 7 juillet)

Response 2 - Réponse 2

Question 3.

How would you rate the open networking session and self-introductions? (Thursday 7 July)

De quelle utilité avez-vous trouvé la session ouverte d'échange entre les membres du réseau et les annonces faites individuellement par les participants? (Jeudi 7 juillet)

Response 3 - Réponse 3

Question 4.

How would you rate the thematic presentations? (Friday 8 July)

Comment évaluez-vous les exposés thématiques? (Vendredi 8 juillet)

Response 4 - Réponse 4

Question 5.

How would you rate the presentation of Senegal experience? (Friday 8 July)

Comment évaluez-vous la présentation de l'expérience sénégalaise? (Vendredi 8 juillet)

Response 5 - Réponse 5

Question 6.

How would you rate the introductions to the field visits? (Friday 8 July)

Comment évaluez-vous les introductions aux visites sur le terrain? (Vendredi 8 juillet)

Response 6 - Réponse 6

Question 7.

How would you rate the field visits? (Saturday 9 July)

Comment évaluez-vous les visites sur le terrain? (Samedi 9 juillet)

Response 7 - Réponse 7

Question 8.

How would you rate the small group discussions? (Sunday 10 July)

Comment évaluez-vous les discussions en petits groupes? (Dimanche 10 juillet)

Response 8 - Réponse 8

Question 9.

How would you rate the network planning group? (Sunday 10 July) (The number of participants in the planning group was restricted, and those who did not attend were asked not to comment).

Comment évaluez-vous la session de planification du réseau? (Dimanche 10 juillet) (Le nombre de participants à la session de planification était limité, et ceux qui n'avaient pas participé ont été priés de s'abstenir de tout commentaire)

Response 9 - Réponse 9

Key		Légende
++	Excellent, very useful, very good.	++	Excellent, très utile, très bon.
+	Above average, good, useful.	+	Au-dessus de la moyenne, bon, utile.
OK	Acceptable, reasonable, average.	OK	Acceptable, raisonnable, moyen.
-	Below average, poor, not particularly useful.	-	Au-dessous de la moyen, médiocre, pas particulièrement utile.
- -	Very poor, not useful.	- -	Très médiocre, pas utile.
NC	No comment. People who missed a session for any reason (e.g. late arrival, sickness, other engagements) were asked not to comment.	AC	Aucun commentaire. Les personnes n'ayant pas assisté à un session pour un raison quelconque (p.ex. arrivée tardive, maladie, autres engagements) ont été priées de s'abstenir de tout commentaire.

Question 10.

How would you rate the presentation of the small group discussions?

Comment évaluez-vous la présentation des discussions en petits groupes? (Lundi 11 juillet)

Response 10 - Réponse 10

Question 11.

How would you rate the open discussion? (Monday 11 July)

Comment évaluez-vous le débat ouvert? (Lundi 11 juillet)

Response 11 - Réponse 11

Question 12.

How would you rate the discussion of the future of the network? (Monday 11 July)

Comment évaluez-vous le débat sur l'avenir du réseau? (Lundi 11 juillet)

Response 12 - Réponse 12

Question 13.

How would you rate the final day: recommendations and resolutions? (Tuesday 12 July)

Comment évaluez-vous la dernière journée: recommandations et résolutions? (Mardi 12 juillet)

Response 13 - Réponse 13

Question 15.

How would you rate the level of pre-workshop publicity?

Comment évaluez-vous la publicité préable à l'Atelier?

Response 15 - Réponse 15

Question 16.

How would you rate the pre-workshop correspondence?

Comment évaluez-vous la correspondance préable à l'Atelier?

Response 16 - Réponse 16

Question 17.

How would you rate the content of workshop information forms?

Comment évaluez-vous le contenu des documents d'information de l'Atelier?

Response 17 - Réponse 17

Question 18.

How would you rate the conditions defined for workshop participation?

Comment évaluez-vous les conditions définies pour la participation à l'Atelier?

Response 18 - Réponse 18

Question 19.

How would you rate the reproduction of workshop documents?

Comment évaluez-vous la reproduction des documents de l'Atelier?

Response 19 - Réponse 19

Key		Légende
++	Excellent, very useful, very good.	++	Excellent, très utile, très bon.
+	Above average, good, useful.	+	Au-dessus de la moyenne, bon, utile.
OK	Acceptable, reasonable, average.	OK	Acceptable, raisonnable, moyen.
-	Below average, poor, not particularly useful.	-	Au-dessous de la moyen, médiocre, pas particulièrement utile.
- -	very poor, not useful.	- -	Très médiocre, pas utile.
NC	No comment. People who missed a session for any reason (e.g. late arrival, sickness, other engagements) were asked not to comment.	AC	Aucun commentaire. Les personnes n'ayant pas assisté à un session pour un raison quelconque (p.ex. arrivée tardive, maladie, autres engagements) ont été priées de s'abstenir de tout commentaire.

Question 20.

How would you rate the organization of workshop plenary sessions?

Comment évaluez-vous l'organisation des sessions plénières de l'Atelier?

Response 20 - Réponse 20

Question 21.

How would you rate the organization of the field trip?

Comment évaluez-vous l'organisation des visites sur le terrain?

Response 21 - Réponse 21

Question 22.

How would you rate the organization of group discussions?

Comment évaluez-vous l'organisation des discussions de groupe?

Response 22 - Réponse 22

Question 23.

How would you rate the quality of simultaneous translation?

Comment évaluez-vous la qualité de la traduction simultanée?

Response 23 - Réponse 23

Question 24.

How would you rate the quality of conference facilities?

Comment évaluez-vous la qualité des équipements de conférence?

Response 24 - Réponse 24

Question 25.

How would you rate the quality of hotel services (rooms, food etc)?

Comment évaluez-vous la qualité des services hôteliers (chambres, nourriture, etc.)?

Response 25 - Réponse 25

Question 26.

How would you rate the level of experience of the participants?

Comment évaluez-vous le niveau d'expérience des participants?

Response 26 - Réponse 26

Question 27.

How would you rate the range of background of the participants?

Comment évaluez-vous le niveau de formation des participants?

Response 27 - Réponse 27

Question 28.

How would you rate the overall suitability of the participants?

Comment évaluez-vous l'adéquation générale des participants?

Response 28 - Réponse 28

Key		Légende
++	Excellent, very useful, very good.	++	Excellent, très utile, très bon.
+	Above average, good, useful.	+	Au-dessus de la moyenne, bon, utile.
OK	Acceptable, reasonable, average.	OK	Acceptable, raisonnable, moyen.
-	Below average, poor, not particularly useful.	-	Au-dessous de la moyen, médiocre, pas particulièrement utile.
- -	very poor, not useful.	- -	Très médiocre, pas utile.
NC	No comment. People who missed a session for any reason (e.g. late arrival, sickness, other engagements) were asked not to comment.	AC	Aucun commentaire. Les personnes n'ayant pas assisté à un session pour un raison quelconque (p.ex. arrivée tardive, maladie, autres engagements) ont été priées de s'abstenir de tout commentaire.