Required citation:

FAO. 2020. FAO guide to ranking food safety risks at the national level. Food Safety and Quality Series No 10. Rome. https://doi.org/10.4060/cb0887en

The designations employed and the presentation of material in this information product do not imply the expression of any opinion whatsoever on the part of the Food and Agriculture Organization of the United Nations (FAO) concerning the legal or development status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dashed lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or products of manufacturers, whether or not these have been patented, does not imply that these have been endorsed or recommended by FAO in preference to others of a similar nature that are not mentioned.

The views expressed in this information product are those of the author(s) and do not necessarily reflect the views or policies of FAO.

ISSN 2415-1173 [Print]
ISSN 2664-5246 [Online]
ISBN 978-92-5-133282-5
E-ISBN 978-92-5-133514-7 (MOBI)
© FAO, 2020

Some rights reserved. This work is made available under the Creative Commons Attribution-NonCommercial-Share Alike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo/legalcode).

Under the terms of this licence, this work may be copied, redistributed and adapted for non-commercial purposes, provided that the work is appropriately cited. In any use of this work, there should be no suggestion that FAO endorses any specific organization, products or services. The use of the FAO logo is not permitted. If the work is adapted, then it must be licensed under the same or equivalent Creative Commons licence. If a translation of this work is created, it must include the following disclaimer along with the required citation: “This translation was not created by the Food and Agriculture Organization of the United Nations (FAO). FAO is not responsible for the content or accuracy of this translation. The original English edition shall be the authoritative edition.”

Disputes arising under the licence that cannot be settled amicably will be resolved by mediation and arbitration as described in Article 8 of the licence except as otherwise provided herein. The applicable mediation rules will be the mediation rules of the World Intellectual Property Organization http://www.wipo.int/amc/en/mediation/rules and any arbitration will be conducted in accordance with the Arbitration Rules of the United Nations Commission on International Trade Law (UNCITRAL).

Third-party materials. Users wishing to reuse material from this work that is attributed to a third party, such as tables, figures or images, are responsible for determining whether permission is needed for that reuse and for obtaining permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user.

Sales, rights and licensing. FAO information products are available on the FAO website (www.fao.org/publications) and can be purchased through publications-sales@fao.org. Requests for commercial use should be submitted via: www.fao.org/contact-us/licence-request. Queries regarding rights and licensing should be submitted to: copyright@fao.org.

Cover photos (from left to right):

Manzini, Eswatini - Encabeni Fresh Produce Market, © FAO / Believe Nyakudjara; Lab manager Geoffrey Jagero inspecting samples at the International Centre of Insect Physiology and Ecology (ICIPE), © FAO / Simon Maina.

Design: Studio Bartoleschi. Layout: Tomaso Lezzi

CONTENTS

Preface

Acknowledgements

Abbreviations and acronyms

Executive summary

CHAPTER 1
INTRODUCTION

1.1 Objectives

1.2 How this guidance document was developed

1.3 The role of risk ranking in risk analysis

CHAPTER 2
PROPOSED RISK RANKING APPROACH

2.1 Step 1: Define the scope

2.1.1 Define the purpose

2.1.2 Select what will be ranked

2.1.3 Screen foods and hazards for overall relevance and risk potential

2.2 Step 2: Develop the approach

2.2.1 Select the risk ranking method

2.2.2 Select the metrics for ranking risks

2.2.3 Collect and evaluate appropriateness of data

2.3 Step 3: Conduct the risk ranking analysis and report results

2.4 Prioritization

CHAPTER 3
CASE STUDIES

3.1 Microbial case study

3.1.1 Step 1: Define the scope

3.1.2 Step 2: Develop the approach

3.1.3 Step 3: Conduct the risk ranking analysis and report results

3.2 Chemical case study

3.2.1 Step 1: Define the scope

3.2.2 Step 2: Develop the approach

3.2.3 Step 3: Conduct the risk ranking analysis and report results

CHAPTER 4
CONCLUSION

REFERENCES

GLOSSARY

ANNEXES

ANNEX A. RISK RANKING EXPERT BIOGRAPHICAL SKETCHES

ANNEX B. EXAMPLES OF FOOD SAFETY RISK RANKING EFFORTS

ANNEX C. LIST OF POTENTIAL MICROBIAL AND CHEMICAL HAZARDS TO BE CONSIDERED IN FOOD SAFETY RISK RANKING EFFORTS

ANNEX D. POTENTIAL SOURCES OF INFORMATION FOR CONDUCTING RISK RANKING EFFORTS

ANNEX E. EXAMPLES OF FOOD CATEGORIZATION

ANNEX F. OVERVIEW OF RISK RANKING METHODS

Qualitative methods–outcomes without numerical values

Decision flowcharts

Deliberative process

Semi-quantitative–numerical outcomes without a unit of measurement

Risk matrix

Multi-criteria decision analysis

Quantitative methods–numerical outcomes with specific units

Burden of disease methods

Quantitative risk assessment

ANNEX G. WHO ESTIMATES OF THE GLOBAL BURDEN OF FOODBORNE DISEASES

TABLES

1. Assessing causal link between a food and/or hazard and illness

2. Potential metrics for risk ranking of microbial and chemical hazards

3. Food categorization scheme used in the risk ranking

4. List of 46 potential microbial hazards considered for the risk ranking

5. Selected pathogens to be included in the risk ranking

6. WHO median estimates for the 14 selected foodborne hazards plus estimates from expert elicitation of pathogens not evaluated by WHO

7. Number of illness for Salmonella spp-non-typhoidal and T. gondii for the major food commodities consumed in Country X

8. List of potential chemical hazards that could be present in seafood

9. List of potential foods (seafood) to be considered for the risk ranking

10. Final list of foods to be included in the risk ranking

11. Matrix of food-hazard pairs to include in the risk ranking

12. Consumption data for foods included in the risk ranking

13. Reference doses (RfDs) for the four chemical compounds included in the risk ranking

A.1 Selected examples of risk ranking efforts in the area of food safety

A.2 Potential microbial food safety hazards to be considered in food safety risk ranking efforts

A.3 Selected potential chemical food safety hazards for food safety risk ranking efforts

FIGURES

1. Modified risk analysis framework (adapted by FAO/WHO, 2006)

2. Generic framework for risk management (FAO/WHO, 2006)

3. Proposed risk ranking approach

4. Hypothetical representation of the possible focus of a risk ranking

5. Example of a decision flowchart to be used when screening microbial and chemical hazards.

6. Approaches to assessing risk (Devleesschauwer et al., 2018)

7. Preliminary decision flowchart for the selection of risk ranking methods in the area of food safety

8. Graphic representation of risk

9. Relative risk ranking graphs for microbial hazards (normalized data, logarithmic scale)

10. Relative risk ranking reverse axis graph for chemical hazards found in fish

11. Decision flowchart for including or excluding microbial hazards

12. Foodborne incidence per 100 000 and DALY/Case for the 14 microbial hazards selected

13. Incidence per 100 000 and DALY/Case by hazard (Logarithmic Scale)

14. Incidence per 100 000 and DALY/Case by food-hazard pair (Logarithmic Scale)

15. Decision flowchart for including and excluding chemical hazards

16. Severity vs. likelihood graph for selected chemical hazards in seafood

A1. Interagency Food Safety Analytics Collaboration food categories with examples (in tan)

A2. Printscreen of FoodEx2 system with salmon classification possibilities, including the hierarchy group (blue pyramid), core list group (red circle), and extended list group (green circle) (EFSA, 2011)

A3. Decision flowchart providing risk ranking of different hazards for poultry (EFSA, 2012b)

A4. Examples of risk bins for likelihood and severity

A5. WHO FERG online tool

BOXES

1. Understanding Risk

2. Risk Ranking vs. Prioritization

3. Stakeholder Engagement

4. An Example of a Statement of Concern and A Statement of Purpose and Objectives

5. Weight of Evidence Assessment (EFSA, 2017)

PREFACE

The process of developing the FAO Guide to Ranking Food Safety Risks at the National Level was initiated by Sarah Cahill, Michael Batz, John Bassett, and Mary Kenny. The preliminary idea was further elaborated with RTI International (Juliana Ruzante and Barbara Kowalcyk). A workshop to collect the elements and develop this guide was convened in Washington, D.C., on 12-14 April 2016. The discussions and conclusions of the workshop were taken into consideration in finalising this document. In addition, the document was also subject to peer review before finalisation.

ACKNOWLEDGEMENTS

This guidance was prepared with the support of the RTI[1] International Food Safety Centre where the work was led by Juliana Ruzante and Barbara Kowalcyk,[2] with key contributions from Jennifer Hoponick Redmon and Katherine Woodward.

The Food and Agriculture Organization of the United Nations would like to thank Dr John Bassett, Mr Michael Batz (Food and Drug Administration,[3] the United States of America), Dr Henry Chin (retired), Ms Jessica Cox (Department of Homeland Security, the United States of America), Dr Arie Havelaar (University of Florida), Mr Greg Paoli (Risk Sciences International), Dr Fernando Sampedro (University of Minnesota), and Dr Nga Tran (Exponent) for their participation and invaluable contribution to the development of this guidance.

The FAO is also grateful to who reviewed this guide, namely Enrique Pérez Gutiérrez (Pan American Health Organization, the United States of America), Zhaoping Liu (China National Center for Food Safety Risk Assessment, China), Esther van Asselt (Wageningen University, the Netherlands) and Robert G. Ahern (Inter-American Institute for Cooperation on Agriculture, Costa Rica).

Various FAO staff provided technical inputs and insights during the entire process of the publication’s development, in particular: Sarah Cahill, Jeff LeJeune, Markus Lipp, Catherine Bessy, Vittorio Fattori, and Kang Zhou.

A special thank you to FAO for the financial support that enabled this project and to the support staff at RTI International.

ABBREVIATIONS AND ACRONYMS

ADI Acceptable Daily Intake

CAC Codex Alimentarius Commission

CDC Centers for Disease Control and Prevention

COI Cost-of-Illness

DALY Disability-Adjusted Life Year

DDT Dichlorodiphenyltrichloroethane

DSP Okadaic Acid

EFSA European Food Safety Authority

FAO Food and Agriculture Organization of the United Nations

FERG Foodborne Disease Burden Epidemiology Reference Group

GEMS Global Environmental Monitoring System

INCHEM Chemical Safety Information from Intergovernmental Organizations

INFOSAN International Food Safety Authorities Network

IPCS International Programme on Chemical Safety

IRIS Integrated Risk Information Surveillance System

JECFA Joint FAO/WHO Expert Committee on Food Additives

LD50 Median Lethal Dose

LOAEL Lowest-Observed-Adverse-Effect Level

MCDA Multi-Criteria Decision Analysis

NOAEL No-Observed-Adverse-Effect Level

NORS National Outbreak Reporting System

NSP Brevetoxins

PAHs Polycyclic Aromatic Hydrocarbons

PBDEs Polybrominated Diphenyl Ethers

PCBs Polychlorinated Biphenyls

PCDDs Polychlorinated Dibenzodioxins

PCDFs Polychlorinated Dibenzofurans

PHAH Polyhalogenated Aromatic Hydrocarbon

PSP Saxitoxin

QALY Quality-Adjusted Life Year

QRA Quantitative Risk Assessment

RfD Reference Dose

sQMRA Swift Quantitative Microbial Risk Assessment

TTC Threshold of Toxicological Concern

WHO World Health Organization

YLD Years Lived with Disability

YLL Years of Life Lost

EXECUTIVE SUMMARY

Risk analysis is internationally accepted as a key component to support decision-making around food safety. Several countries have started implementing risk-based food safety systems. Central to the risk-based approach is an assessment of food safety risks. Food safety risk ranking is the systematic analysis and ordering of foodborne hazards and/or foods in terms of public health risks, based on the likelihood and severity of adverse impacts on human health in a target population. Risk ranking provides national food safety authorities with the scientific basis to make informed regulatory decisions; enhance disease surveillance; determine how food inspections are allocated; oversee inspection and enforcement food safety efforts; inform the public of food safety threats; and continue to improve the safety of the foods produced and consumed in the country based on the public health impact of hazards and/or foods. The use of risk ranking to identify which food safety issues have the greatest public health impact facilitates objective, evidence-based, transparent decision-making and planning.

The objective of this guidance is to provide direction to national food safety authorities on how to start ranking the public health risk posed by foodborne hazards and/or foods in their countries. The guidance focuses on the ranking of microbial and chemical hazards based exclusively on their impact on public health and represents the first step toward a systematic and evidence-based approach to identify the most significant risks in the area of food safety. This guidance was developed with a wide audience in mind, including but not limited to microbiologists, toxicologists, chemists, environmental health scientists, public health epidemiologists, risk analysts, risk managers, and policy makers. The application of the proposed framework is illustrated by two hypothetical case studies, one microbial and another chemical.

The guidance presents and describes in detail a ranking approach that is composed of three iterative steps:

Step 1: Define the Scope

1A) Define the Purpose

1B) Select What Will Be Ranked

1C) Screen Foods and/or Hazards for Overall Relevance and Risk Potential

Step 2: Develop the Approach

2A) Select the Risk Ranking Method

2B) Select the Metrics for Ranking Risks

2C) Collect and Evaluate Appropriateness of Data

Step 3: Conduct the Risk Ranking Analysis and Report Results

The first step is to define the scope (i.e. the purpose of the risk ranking exercise; why it is needed). Clear statements of concern, purpose and objective are needed to help guide the effort. It is also in this first step that the hazards and/or foods to be ranked are selected and screened for relevance and overall risk potential. Screening is critical to keep the process manageable going forward. This guidance document also provides example decision flowcharts that can be used to screen foods and hazards when defining the scope.

The second step is the development of the risk ranking approach. This process involves the selection of the risk ranking method and the metrics to estimate the two dimensions of risk (i.e. likelihood and severity, as well as their uncertainty and variability), as well as the collection and evaluation of data needed to estimate risk. Selection of the approach is an iterative process and should be driven by the scope, availability of data, professional expertise, and technical and financial resources.

The third and final step is conducting the risk ranking analysis and reporting the results. Results must be discussed in detail, and information on the assumptions and limitations of the method, metrics and data used needs to be made clear. The risk ranking results (i.e. relative ranking of public health risks) can then be incorporated into prioritization efforts where other factors, such as social, economic and political factors are taken into consideration in a systematic manner to further inform decisions. Uncertainty should also be part of the discussion of how the risk ranking is used in risk prioritization. Reducing uncertainty may be part of the planned follow-up activities.

The proposed approach can be considered a starting point, since risk ranking is a complex, iterative and data-driven process that must evolve as new data and information become available. We invite countries to provide us feedback to support the improvement of this guidance so it can be a robust and pragmatic document that can be used as the basis for developing and implementing a risk-based food safety system in your country.

CHAPTER 1

INTRODUCTION

“National food safety authorities must deal with numerous food safety issues, often simultaneously. Resources inevitably are insufficient to manage all issues at any given time and ranking of issues in priority for risk management, as well as ranking risks for assessment, are important activities for food safety regulators.”

FAO/WHO, 2006

Efforts to improve food safety to protect public health from microbial and chemical hazards remain a significant challenge globally (FAO/WHO, 2006). Food safety is a complex discipline encompassing a wide range of foods, potential hazards and a multitude of production and processing systems; in addition, it must adapt to constant changes in food production and processing, and host susceptibility, as well as food consumption patterns. With limited human and financial resources, it is impossible for national authorities to efficiently address all food safety threats. Therefore, it is important to rank food safety concerns and prioritize efforts so that resources can be allocated to most efficiently minimize foodborne illness risks, where risk is defined to be a function of the likelihood and severity of an adverse event (Box 1). However, risk management decisions are often undertaken in an ad hoc and subjective manner. Without an objective, structured and scientific approach to analysing food safety issues, policy decisions can be difficult to effectively justify and communicate.

Risk analysis is internationally accepted as a key component of decision-making around food safety. Several countries have started implementing risk-based food safety systems. Central to the risk-based approach is an assessment of food safety risks. Food safety risk ranking is the systematic analysis and ordering of foodborne hazards and/or foods in terms of the public health risk based on the likelihood and severity of adverse impacts on human health in a target population. Risk ranking provides national food safety authorities with the scientific basis to make informed regulatory decisions; enhance disease surveillance; determine how food inspections are allocated; oversee inspection and enforcement food safety efforts; inform the public of food safety threats; and continue to improve the safety of the foods produced and consumed in the country based on their risk to public health. The use of risk ranking to identify which food safety issues have the greatest public health impact facilitates objective, evidence-based, transparent decision-making and planning.

The integral role of risk ranking in supporting decision-making and the development and implementation of a risk-based food safety system has been the focus of several national and international publications, including the Codex Alimentarius (Codex)[4] and an IMNRC (2010) report that served as the basis for one of the more sweeping regulatory reforms in the area of food safety in the United States.[5] However, the multitude of hazards and foods, complexity of the methods, data needs, and resource intensive nature of risk ranking efforts often pose barriers to wider adoption. Further, three key publications–the Codex guidance on ranking and prioritization of hazards in animal feed (CAC, 2013), the European Food Safety Authority’s review of risk ranking efforts (EFSA 2015; EFSA, 2012a), and a third party technical report developed for EFSA on methods for ranking food safety risks (van der Fels-Klerx et al., 2015)–demonstrate the need for more guidance in this area. For example, EFSA (2012a) found that most risk ranking efforts varied widely with regard to purpose, methodology, and risk metrics, and concluded that no universal methodology for risk ranking exists because each approach must be tailored to the specific purpose, data availability and time frame of a given evaluation. These findings illustrate the complexity and sometimes overwhelming number of possibilities and approaches that could be taken to develop a risk ranking. Therefore, the Food and Agriculture Organization of the United Nations (FAO) is working to provide scientific recommendations and facilitate the use of risk ranking in national food safety programmes.

1.1 OBJECTIVES

The objective of this guidance is to provide direction to decision-makers on how to start ranking the public health risk posed by foodborne hazards and/or foods in their countries. The primary focus is microbial and chemical hazards in foods, but the overall approach could be used for any hazard. This guidance was developed with a wide audience in mind, including but not limited to microbiologists, toxicologists, chemists, environmental health scientists, public health epidemiologists, risk analysts, risk managers, and policy makers. Political will and a strong commitment to modernize food safety are key to the successful development and implementation of any risk ranking effort at the country level.

1.2 HOW THIS GUIDANCE DOCUMENT WAS DEVELOPED

To develop this guidance, RTI International, on behalf of FAO, convened a three-day meeting of subject matter experts in Washington, DC, in April 2016. At the meeting, the experts:

> agreed on what steps comprise a risk ranking effort;

> provided guidance on how to start the risk ranking process;

> recommended an approach to risk ranking; and

> discussed how currently available microbial and chemical risk ranking methods and tools could be most effectively used.

This guidance is a summary of the discussions, conclusions, and recommendations that emerged from that meeting. This document was reviewed by the experts who attended (Annex A). We also invite users of this guidance to provide additional feedback and real-life experiences that can shape this document further into a more pragmatic guidance to risk ranking in the area of food safety.

1.3 THE ROLE OF RISK RANKING IN RISK ANALYSIS

Risk analysis is a science-based, systematic, disciplined approach that can be used to address food safety problems. As shown in Figure 1, it has three components (FAO/WHO, 2006):

> risk assessment: the scientific evaluation of risk;

> risk management: a decision-making process that considers results of risk assessment, but may also include other considerations such as cost, feasibility for implementation, and willingness to change; and

> risk communication: the approach to communicating the results of both risk assessment and risk management to stakeholders.

National governments and international organizations support the use of the Codex risk analysis framework as a way to promote improvements in public health, provide a basis for expanding international trade in foods, and tackle the growing number of food safety concerns (FAO/WHO, 2006).

Risk ranking is part of risk analysis. It is an important tool with which risk managers can gather additional and more detailed scientific information on the public health impact to help inform their decisions, and is therefore typically described as being part of the preliminary risk management activities (Figure 2); however, it can be undertaken at any point during the risk analysis process and may or may not be part of a risk assessment (FAO/WHO, 2006). For example, risk ranking provides science-based information that can be used to:

> identify the most significant public health risks;

> help identify the most appropriate interventions to reduce contamination in food;

> identify food safety issues that require regulatory action;

> target inspections of food establishments in different parts of the food supply chain (e.g. farms, slaughterhouses, processing facilities, retail, food service);

> target sampling and testing programmes for domestic and imported foods;

> evaluate the likely importance of emerging food safety issues; and

> develop consumer guidance.

Annex B provides examples of risk ranking efforts conducted throughout the world to inform regulatory priorities and address specific risk management questions and needs. Risk ranking is a separate but important precursor to completing prioritization efforts. In prioritization, foodborne hazards (or food safety issues) are systematically analysed and ordered based on the consideration of public health impacts (resulting from risk ranking), and other factors such as social, economic, political concerns (Box 2).

CHAPTER 2

PROPOSED RISK RANKING APPROACH

This guidance document presents a recommended approach to risk ranking that provides an objective and systematic way to rank risks in foods based on their public health impact (Figure 3). The proposed approach was developed with input from FAO and the expert panel. It focuses on microbial and chemical hazards, although it can be adapted for other hazards (i.e. allergens, physical hazards). The proposed approach should be considered a starting point since risk ranking is a complex, data-driven process that is iterative. The results of this effort can then be used in the prioritization process (Section 2.4) to further inform decisions.

The steps of this proposed risk ranking approach are summarized as follows:

Step 1: Define the Scope (Section 2.1)

1A) Define the Purpose (Section 2.1.1)

1B) Select What Will Be Ranked (Section 2.1.2)

1C) Screen Foods and/or Hazards for Overall Relevance and Risk Potential (Section 2.1.3)

Step 2: Develop the Approach (Section 2.2)

2A) Select the Risk Ranking Method (Section 2.2.1)

2B) Select the Metrics for Ranking Risks (Section 2.2.2)

2C) Collect and Evaluate Appropriateness of Data (Section 2.2.3)

Step 3: Conduct the Risk Ranking Analysis and Report Results (Section 2.3)

2.1 STEP 1: DEFINE THE SCOPE

The first and most fundamental step in risk ranking is defining the scope, which will guide all activities. The scope needs to be defined by the risk managers and should precede the technical analysis. Defining the scope includes identifying the risk management question(s), applicable foods and/or hazards, the population in question, availability of resources (technical and financial), as well as the timeframe for completing the work. It is necessary to subsequently screen foods and/or hazards for relevance and potential risk, narrow the focus, and to further develop the risk ranking approach. Ideally, experts from different disciplines and sectors (e.g. academic researchers, consumer groups, industry) would provide input into defining the scope (Box 3). To optimize the usefulness of the results, risk managers will need to maintain effective communication with technical experts and stakeholders throughout the risk ranking process and, if necessary, make adjustments to the scope. Defining the scope is an iterative process that results in a risk ranking approach that meets the risk manager’s needs given the current environment.

2.1.1 DEFINE THE PURPOSE

The first step to defining the scope is identifying the salient risk management questions and goals for the risk ranking. Simply put, why are you doing it? We suggest drafting a Statement of Concern and a Statement of Purpose and Objectives that describe in simple terms the motivation for conducting the risk ranking exercise and the management goals that need to be addressed. When clearly articulated, such statements can provide the foundation for developing the risk ranking exercise, improve transparency, and provide context to the analysts conducting the effort. Examples are provided in Box 4.

To define the scope, it is necessary to characterize what scenarios are relevant in conjunction with the risk management question(s). For instance, risk managers might want to increase their regulatory control and oversight on products that pose the greatest public health risks to children, or might want to identify the most significant public health risks associated with foods that are often consumed by the general population. Recent outbreaks linked to a certain food product might be the focus of risk manager concern, trying to understand the potential hazards associated with the product in order to develop preventive measures.

Below, we list some questions that can help shape the scope:

> Is the concern about new or emerging risks?

> Is the concern about hazards intentionally added to and/or unintentionally found in foods?

> Is the concern about a recent outbreak of a foodborne illness or any specific food recalls?

> Has there been a recent outbreak of a foodborne illness or any specific food recalls?

> Is likelihood of illness a primary concern, regardless of the severity?

> Is severity of illness a major concern (e.g. death, chronic disease), regardless of the frequency?

> Are specific subpopulations, such as children, pregnant women, the elderly, or immunocompromised individuals, at greater risk?

> Is the concern about specific behavioural or lifestyle factors linked to increased risk (e.g. anglers)?

> Is the concern about where the food was produced and processed? Foods produced or processed in the country? Foods imported into the country? Foods destined for export for consumption in another country? Foods from specific production systems (e.g. intensive or extensive)?

> Is the concern about a specific food production and/or processing characteristic (i.e. organic versus conventional production; fresh, frozen, or canned; a specific cut of meat)?

> Is the concern about a specific supply chain?

> Are hazards approved for use in foods (e.g. approved pesticides or food additives) or those used illegally of special interest a concern?

> Is the concern related to a recent public outcry or increased media attention?

Finally, identifying the intended audience is critical to define the scope of the risk ranking. For example, a risk ranking designed to provide advice to consumers on how to manage individual food safety consumption risks will likely be different than a risk ranking designed to inform decision-making around the allocation of inspection resources.

2.1.2 SELECT WHAT WILL BE RANKED

The foods and hazards to be evaluated in the risk ranking process should be based on the goals of the risk ranking and defined by the risk manager and policy makers. Typically, a risk assessments focus on one hazard/one food, whereas risk ranking analyses typically consider multiple hazards, multiple foods, or multiple hazards and multiple foods (EFSA, 2012a; Figure 4).

The selection of the foods and hazards to include in the ranking process is based on a variety of concerns or factors described previously, including the outcomes of interest (e.g. high severity, high likelihood, high risk). The risk manager will need to determine which foods are relevant: all food commodities (e.g. dairy, meats, produce, fish, etc.), a subset of foods (e.g. dairy), or a specific food (e.g. fluid milk). The risk manager will also need to determine which hazards are relevant: all known microbial and chemical hazards, a specific type of hazards (e.g. microbial or chemical), a subgroup of hazards (e.g. heavy metals), or a specific hazard (e.g. lead). Risk managers will likely need to rely on the expertise of their technical experts to identify an initial list of potential foods and hazards.

Identifying potential foods and hazards can be time consuming and resource intensive. Technical experts can rely on several sources of information to formulate the list of hazards and foods. A list of the most common chemical and microbiological food safety hazards relevant to public health are provided in Annex C. The list was devised based on several sources, including WHO’s recent burden of disease study (WHO, 2015), EFSA’s scientific opinions on the risks associated with several types of meat (EFSA 2013a; EFSA 2013b; EFSA 2013c; EFSA 2013d; EFSA 2012b; and EFSA 2011), and other studies published internationally (Hoffmann et al., 2015; ECDC 2015; Scallan et al., 2011; Kemmeren et al., 2006). However, it is important to note that the list in Annex C is not comprehensive; some contaminants relevant in certain countries may not be listed. Consequently, it is important to critically evaluate this listing and modify or edit as needed. Annex D contains potential sources of information for identifying other possible food safety hazards as well as other helpful resources for ranking risks. Once the potential list of hazards for each ranking is concluded, we recommend that it be reviewed by in-country food safety experts to ensure that critical hazards have not been omitted.

An important, but often overlooked, consideration in the selection of foods and hazards is the development of a categorization system. Food and hazard categorization schemes can make the risk ranking process more manageable and provide context for interpreting results, but they are only useful if they are consistent with the objectives of the risk managers. The list below adapted from Morgan et al. (2000), identifies important attributes to consider in selecting an ideal risk categorization scheme that can be applied when developing food and hazard categorization schemes.

> Logically Consistent

> Exhaustive so that no relevant risks are overlooked.

> Mutually exclusive so that risks are not double counted.

> Homogenous so that all risk categories can be evaluated on the same set of attributes.

> Administratively Compatible

> Compatible with existing organizational structures and legislative mandates so that lines of authority are clear and management actions at cross purposes are avoided.

> Relevant to management so that risk priorities can be mapped into risk management actions.

> Large enough in number so that regulatory attention can be finely targeted, with a minimum of interpretation by agency staff.

> Compatible with existing databases, to make best use of available information in any analysis leading to ranking.

> Equitable

> Fairly drawn so that the interests of various stakeholders, including the general public, are equally represented.

> Compatible with Cognitive Constraints and Biases

> Chosen with an awareness of inevitable framing biases.

> Simple and compatible with people’s existing mental models so that risk categories are easy to communicate.

> Few enough in number so that the ranking task is manageable.

> Free of observational bias, in which better understood risks are categorized more finely than less understood risks.

Importantly, food and hazard categories must be comparable in terms of scale; for example, broad categories for foods and hazards (e.g. meats, parasites) should not be compared with narrowly defined categories (e.g. pasteurized milk, Mycobacterium bovis) and vice versa. The determination also needs to be made about inclusion of foods with multiple ingredients (e.g. complex foods such as a chicken salad or a steak burrito) and how they will be categorized. Further, establishing the food and hazard categorization scheme early in the scoping process will help inform data collection. Existing databases (e.g. consumption data, prevalence/concentration data, outbreak data) will often have differing categorization schemes that will need to be mapped to the categorization schemes used in the risk ranking; this will introduce some uncertainty into the process and should be considered as the categorization schemes are developed. Annex E presents examples of food categorization schemes used in previous risk ranking efforts, and schemes used for food source attribution that meet several of the attributes listed by Morgan et al. (2000).

2.1.3 SCREEN FOODS AND HAZARDS FOR OVERALL RELEVANCE AND RISK POTENTIAL

It may be tempting to include all foods and hazards that appear to match the goals and objectives for the risk ranking, but this is not practical and, frequently, not appropriate. Therefore, it is important to screen the foods and hazards to be included in the model based on the stated scope to reduce to a manageable list. The definition of a “manageable list” will depend on the resources available to conduct the risk ranking (e.g. staff, expertise, data, time) and the desired method for conducting the ranking. For example, ranking 100 food-hazard pairs may be manageable in one resource setting but less manageable in another. Similarly, ranking a list of 100 food-hazard pairs may be manageable using qualitative methods, but not if quantitative methods are used.

In screening foods and/or hazards, it is important to be aware of the impact of misclassification at this stage in the risk ranking process. Lack of data and/or high levels of uncertainty could result in screening out relevant and/or potentially high risk foods and/or hazards. It might be useful to determine, in advance, how such foods and hazards would be handled.

Below are two approaches that can be used to narrow down to the most relevant food and/or hazards that will be ranked. We recommend going through this process even if you believe you have a “manageable list” so that time and resources are not spent on foods and/or hazards that are later determined not to be relevant.

2.1.3.1 Screen for Relevance

Foods and hazards should only be included in the risk ranking if they are relevant; that is, if they are a potential source of risk to public health. The Bradford Hill Criteria (see Table 1) provide a structured framework for assessing a causal link between exposure and effect and have been broadly used in epidemiologic research (Hill, 1965). When adapted for food safety risk ranking, the Bradford Hill Criteria can be used to screen foods and hazards for relevance and provide a transparent and science-based approach to justifying the inclusion/exclusion of selected foods and/or hazards in the risk ranking exercise.

Developing a decision flowchart based on the Bradford Hill Criteria, with yes/no outcomes and/or well-defined cut-offs, could be helpful for screening potential foods and/or hazards for inclusion in the risk ranking (see Figure 5). For example, suppose you are interested in conducting a risk ranking of emerging risks and that Klebsiella pneumonia in poultry is on the list of potential food-hazard pairs for the risk ranking because of some recent media attention. Since Klebsiella has not been traditionally viewed as a foodborne pathogen in your country, you are unsure if it should remain on the list. You could review the available evidence on Klebsiella in poultry and use the decision flowchart in Figure 5 to determine, in a transparent manner, if it should remain on the list or be removed. Figure 5 is an example decision flowchart and may need to be modified to reflect what is relevant in your situation and scope.

Available evidence should be reviewed to determine if foods and/or hazards meet the decision flowchart criteria; however, this does not need to be an exhaustive evaluation. Importantly, the risk team will need to decide how to handle foods and/or hazards for which there is no evidence. For example, suppose a certain chemical has been detected in food but the “level of concern” (e.g. benchmark dose) has not been established. Including this hazard will require more resources; however, it will reduce the chance that a relevant public health hazard is excluded from the ranking. Experts could be consulted to determine whether the hazard is likely to be present at levels of concern. Another option would be to compare the concentration found in food with national or international health-based standards for certain hazards in foods (e.g. Maximum Levels for contaminants and toxins in foods; CAC, 2011) to determine whether the hazard should be included (i.e. food concentration above the standard) or excluded (i.e. food concentration below the standard). This process is in essence a high-level ranking but, in the case of chemicals, might be necessary at this stage to reduce the number of contaminants to a manageable number.

It is important to note that few food-hazard pairs would likely meet all the Bradford Hill Criteria, and it would probably not be feasible to evaluate all food-hazard pairs on all criteria; nonetheless it provides guidance on criteria that can be considered when screening foods and/or hazards that have been initially identified by risk managers and food safety experts. At a minimum, foods and hazards should be limited to those that are relevant to the country’s diet (e.g. consumed by a significant percentage of the population), and meet the plausibility and coherence criteria. An example of how to apply this for microbial and chemical hazards is provided in Figure 5, and in one of the case studies described here, there is also another option (Figure 16).

STEP 1 SUMMARY

> Draft a Statement of Concern and a Statement of Purpose and Objectives to clearly define the scope, targeted population, and audience of the risk ranking.

> Use different data sources and input from local experts and stakeholders to produce a comprehensive list of potential hazards that are representative of your country (see Annex C for potential list of hazards).

> Define the hazards and/or food categories in a logical and consistent manner so results can be mapped to risk management actions and existing data can be leveraged.

> Screen foods and/or hazards to obtain a manageable list for the risk ranking. Make sure assumptions and definitions are clearly stated so they are reproducible.

2.2 STEP 2: DEVELOP THE APPROACH

With the scope finalized, the approach for ranking the risks according to their public health impact needs to be developed. Developing the approach consists of three stages: choose the risk ranking method, select the metrics for ranking the risks, and collect the data. The process for developing the approach is not linear (i.e. the stages can be conducted in any order) and is often iterative (i.e. stages may need to be revisited). For example, the development of the risk ranking approach may be driven by the data available in one situation while being driven by a preferred method in another. Regardless of how the approach is developed, it is important to evaluate its feasibility given the time, data, technical expertise, and financial resources available. For example, the preferred approach may be to conduct a country-specific quantitative risk ranking using disability adjusted life years (DALYs) as the metric, but data are lacking, and there are insufficient resources to collect the needed inputs, making the approach infeasible. However, in revisiting the selection of the risk ranking approach, it may be determined that there are sufficient regional data from the WHO Foodborne Disease Burden Epidemiology Reference Group (FERG) DALY estimates to start the risk ranking, and expert elicitation could be used to address data gaps. There are obvious trade-offs in each step that need to be weighed against the scope and the available resources to reach an approach that meets the needs of the risk managers.

2.2.1 SELECT THE RISK RANKING METHOD

There are many methods and tools that can be used to conduct food safety risk rankings, which have been described and evaluated elsewhere (van der Fels-Klerx et al., 2015; EFSA, 2012a). Annex F describes selected methods identified during the expert meeting as relevant for this guidance. Briefly, risk ranking methods are generally categorized as qualitative (outcomes without numerical values), semi-quantitative (numerical outcomes without a unit of measurement), or quantitative (numerical outcomes with specific units). Qualitative or semi-quantitative methods are faster and require fewer resources; however, they involve more arbitrary decisions to be made to combine different lines of evidence as there is no underlying consistent mathematical framework. Therefore, the final results may not truly reflect the risk estimate. Such methods also result in a loss of information by taking simple yes or no decisions by binning continuous data. It has been shown that binning may result in risk ranking results that are very different from quantitative models (EFSA, 2015). Many view quantitative methods as the gold standard, but such approaches may not be feasible given available resources or appropriate for the goals of the risk ranking. Further, it is important to note that the quality of the outputs from a quantitative analysis is directly related to the quality of data used in the analysis so, in cases where data quality is a concern, it may be more appropriate to use qualitative approaches. The selection of the risk ranking method should take into consideration these trade-offs as they relate to the goal of the risk ranking and should be made in collaboration with risk managers and other stakeholders.

For each of these methods, risk must be estimated as a function of severity and likelihood in one of two broad ways: top-down or bottom-up (Figure 6). In the top-down approach, the metrics for likelihood and severity (Section 2.2.2) are estimated using population attributable fractions derived from information gathered from epidemiological systems, such as surveillance or cohort studies. Thus, a top-down approach relies on the availability of epidemiological data. In the bottom-up approach, estimates are derived using the classic risk assessment paradigm that assesses risk using exposure and dose-response information. In theory, both approaches should result in similar estimates for likelihood and severity; in reality, significant data gaps and uncertainty in the metrics make that unlikely. The approach selected will likely depend on the risks under consideration and available data. For example, there are typically less epidemiologic data for chronic outcomes, making the bottom-up approach more appealing, whereas there are typically more epidemiological data for acute outcomes, making top-down more appealing (Devleesschauwer et al., 2018).

As mentioned previously, risk ranking methods can vary significantly in terms of complexity and required resources, so it is important to consider available time, data, and technical expertise when choosing a method. The decision flowchart presented in Figure 7 illustrates how those parameters could be used to influence your choice of method. Case studies presented in sections 3.1 and 3.2 demonstrate how such flowcharts can be used to select an appropriate risk ranking method. The decision flowchart is a work in progress and needs to be further tested.

2.2.2 SELECT THE METRICS FOR RANKING RISKS

Risk is calculated as a function of the likelihood of illness (probability that illness occurs) and the severity of illness (magnitude of consequences associated with illness). There are several metrics that can be used to characterize severity and likelihood (Table 2); these can be expressed qualitatively, semi-quantitatively, or quantitatively, and may be different for microbial and chemical hazards, as described below. Ideally, the choice of metric(s) would be driven by the goals and objectives of the risk ranking; however, in practicality, it may be driven by available data and/or resources.

Summary measures of population health, such as DALYs and quality-adjusted life years (QALYs), are often the preferred metrics for estimating risk because they incorporate likelihood (e.g. the number of cases) and severity (e.g. DALY/case). For example, a mild disease (i.e. low DALY/case) caused by a highly prevalent risk may have a lower total DALY than severe disease (i.e. high DALY/case) that is caused by a hazard that is rare. Developing DALY and QALY estimates are resource- and data-intensive, which can be a barrier to use and require epidemiologic data on acute and chronic health outcomes, which may not always be available. Utilizing existing DALY estimates, such as those derived by FERG (WHO, 2015), is an option; however, these estimates are not country specific and have been developed for microbial hazards and only a few chemical hazards.

In selecting severity metrics, it is important to consider both acute and chronic health effects. Surveillance data can be used to estimate hospitalization and mortality rates, which are common severity metrics for acute effects for microbial hazards. The lethal dose for 50 percent of the population (LD50) and acute toxicity endpoints (e.g. skin rashes, respiratory distress, eye corrosion) are common metrics for acute effects associated with chemical hazards. Microbial hazards have been associated with chronic health effects, but data are limited (Batz, Henke, and Kowalcyk, 2013); in these cases, the probability of sequelae derived from the published literature can be used as a metric. For chemical hazards, a health-based guidance value such as the acceptable daily intake (ADI) or reference dose (RfD) are used as common health metrics to represent the maximum daily intake that may occur without causing adverse chronic health effects. RfDs can be obtained from several international sources (e.g. U.S. EPA IRIS (US EPA, 2019), IPCS INCHEM (IPCS, 2019)). If RfDs are not available, a No-Observed-Adverse-Effect Level (NOAEL), which represents the maximum dose at which no adverse health effect occurs, or Lowest-Observed-Adverse-Effect Level (LOAEL), which represents the lowest dose that causes an adverse effect, can be used as severity metrics. Non-cancer health benchmarks for chemical hazards often focus on chronic health endpoints. As exposure exceeds health-based guidance values, however, the likelihood of acute health effects also increases. For chemical cancer endpoints, a slope factor (e.g. from U.S. EPA IRIS) is used to estimate excess lifetime cancer risk with consumption in a quantitative human health risk assessment. To assess whether a chemical may cause cancer, the use of semi-quantitative classification schemes can be employed as well. For example, the International Agency for Research on Cancer (IARC) evaluates scientific research findings to classify chemicals as carcinogenic (Group 1), probably carcinogenic (Group 2A), possible carcinogenic (Group 2B), or not classifiable (Group 3).

As stated previously, the choice of metric(s) for likelihood and severity should be driven by the questions the risk manager would like to answer with the risk ranking, the types of hazards being evaluated, and the availability of data. For example, if the risk manager is only concerned with microbiological risks that cause death, mortality rates may be the best metric for severity. In contrast, if the risk manager wants to identify microbial risks that carry the highest burden in terms of morbidity and mortality and data are available, a summary measure of population health (i.e. DALY and QALY) would be the best choice.

2.2.3 COLLECT AND EVALUATE APPROPRIATENESS OF DATA

The goal of the risk ranking exercise should be to obtain results that are reliable, scientifically valid, repeatable, and transparent. Achieving this goal requires timely, representative data. The data requirements for a risk ranking varies according to the approach selected to estimate risk (top-down, bottom-up), the method selected (qualitative, semi-quantitative, quantitative), the hazards being considered (microbial, chemical), and the metrics selected. It will most certainly be the case that data (i.e. evidence) for the risk ranking will be obtained from multiple sources, such as surveillance programmes, literature, databases, and expert elicitation. Since evidence will vary in quantity and quality, analysts must select, weigh, and integrate it according to best practices (Box 5) so that estimates of risk are as accurate and precise as possible, and uncertainty and variability are well characterized, even if the risk ranking is qualitative.

UNDERREPORTING AND UNDERDIAGNOSIS

Data on reported cases, which are often used to estimate likelihood and severity, have diagnosis and reporting biases that likely underestimate likelihood and overestimate severity. Many cases of foodborne illness do not seek medical attention and few, if any, are reported to public health agencies. More severe cases are more likely to seek medical attention and get reported. Similarly, there are limitations to using outbreak data to estimate attribution rates. Very few foodborne illnesses are associated with outbreaks and, consequently, those that are may not be representative of all illnesses. Uncertainty adjustments, such as underdiagnosis and underreporting multipliers, can be used to offset these biases. For example, in the United States, population-based surveys were used to estimate the rates of underdiagnosis and underreporting multipliers and develop appropriate multipliers. FERG also plans to produce their own estimates.

Ideally, in the top-down (or surveillance) approach, hazard-specific epidemiologic and source data would be obtained through in-country active surveillance systems. However, few countries have such systems and must rely on data from reported cases (i.e. passive surveillance), sentinel surveillance sites, cohort and case-control studies, and the published literature. Chronic sequelae have been associated with foodborne pathogens (e.g. Brucella spp and the occurrence of orchitis and Toxoplasma gondii and the occurrence of hydrocephalus, chorioretinitis, and central nervous system abnormalities), but data on the incidence and duration of these are limited (Batz, Henke, and Kowalcyk, 2013). Information on the source of illness is also needed if risks are being categorized by pathway or vehicle. However, for the majority of foodborne illnesses, the food source or vehicle is unknown. Source attribution studies can be used to estimate the proportion of illnesses that are associated with specific foods. For example, outbreak data have been used in the United States to estimate the proportion of cases attributed to various foods (USDA and FSIS, 2015; Painter et al., 2013) and other methods including the use of expert elicitation have also been used for that purpose (Hald et al. 2016; Hoffmann et al., 2007; Batz et al., 2005; Pires et al., 2009).

PREVAlENCE AND CONCENTRATION STUDIES

Published prevalence studies can be helpful sources of data to inform risk ranking efforts; however, you need to be aware of some of their potential limitations. These studies often involve small sample sizes and do not report contamination levels, the limit of detection, or the sensitivity and specificity of the diagnostic test used. These types of studies also frequently fail to capture the variation associated with different seasons, products, producers, and regions. Prevalence and concentration data provided by the industry would represent large sample sizes and can also be valuable, but might not represent a wide range of producers, products and processes used throughout the food industry.

Estimates of likelihood and severity of the risk can also be calculated by modeling pathway-specific exposures and hazard-specific outcomes, respectively (bottom-up approach). In this case, exposure is estimated by modeling the likelihood and level of the hazard in the food at the point of consumption, and the likelihood that the food is consumed. Data on the prevalence and concentration of hazards in foods from production to consumption and consumption trends are needed. National food surveillance systems, results from imported food testing, as well as government reports and peer review literature, are good sources of information (Annex D). Processing and post-processing data on product formulation, storage conditions, control measures, probability of cross contamination, and consumer practices are also needed to estimate the prevalence and concentration of hazards in foods. The food industry could be a valuable source of this information and should be engaged in the process. Consumption data can be obtained through surveys, expert elicitation, or from industry; production volume data can also be used as a surrogate. The probability of hazard-specific outcomes is estimated using dose-response models and disease-outcome trees.

After collecting and evaluating the available data, data gaps will most certainly be identified. How these gaps are addressed will largely be driven by the needs of the risk manager, the available financial resources, and the willingness to make decisions with great uncertainty; two options are discussed below. It should be noted that significant data gaps may require a re-evaluation of the metrics or approach initially selected; however, this is part of the iterative process of defining your approach to risk ranking (Figure 3).

One way to address data gaps would be to use regional or international data as a surrogate in these situations. For example, when the focus is microbiological hazards, the WHO FERG estimates might be a good place to start (See Annex G, WHO, 2015); countries could benefit from these robust quantitative measures without needing to collect the different inputs. Of course, WHO does not have DALY estimates for all hazards that need to be ranked, and therefore complementary approaches will be required. For example, the FERG regional estimates could be used as anchors or parameters to guide expert opinion on developing estimates for countries within the region and/or for developing estimates for hazards not covered by FERG but of interest to the country (see Section 3.1).

Expert elicitation can also be used to fill data gaps by quantifying subject matter knowledge in a structured manner. For example, an elicitation could be used to obtain expert-based estimates on the prevalence and concentration of a certain hazard in a particular food or information on food source attribution; experts could be asked about their “best” central judgement and, potentially, their quantitative uncertainty around this “best” estimate (e.g. 90 percent or 95 percent credible intervals). There are several methods to formally elicit expert opinion (Cooke, 1991; Cooke and Goossen, 2007; Hoffmann et al., 2007; Pires et al., 2013; Pires et al., 2009; Batz et al., 2005; Pires et al., 2014), ranging from individual interviews or online surveys where experts do not interact in face-to-face meetings where the objective is to arrive at consensus. There are limitations to using expert elicitation to fill data gaps–it does not provide empirical data or empirical estimates; the study design needs to be carefully conducted to avoid bias by factors, such as the expert’s background and scientific expertise (Pires et al., 2014; Hoffmann et al., 2007). However, when conducted properly, it is an acknowledged method to complement data gaps. Hoffmann et al., (2007) provides an example of a survey tool designed to elicit food source attribution information for burden of foodborne illness estimates.

Regardless of the approach used to fill data gaps or the use of lower levels of evidence data (e.g. another’s country disease prevalence or a single study with small sample size) (Box 5), it is crucial that the limitations of the approach be clearly identified and communicated during the risk ranking effort as well as in its final report.

Important but often overlooked considerations during data collection are the accuracy and precision of the data. Accuracy refers to how close the estimate is to true value. The difference between the estimate and the true value is called bias. Precision refers to the degree of agreement among estimates among different samples. Estimates that are precise have lower variability. Combined, bias and variability constitute uncertainty (error). All data have some level of uncertainty associated with them, and this uncertainty needs to be considered in the selection process. For example, illnesses associated with outbreaks are often used to estimate likelihood even though the majority of foodborne illnesses are not associated with outbreaks. Therefore, using outbreak illnesses introduces bias into the risk estimates. Illnesses ascertained through an active surveillance system are also biased since many illnesses are undiagnosed, but the bias is relatively less than that with outbreak illnesses. The acceptable level of uncertainty will likely depend on the scope of the risk ranking and should be discussed with the risk managers.

STEP 2 SUMMARY

> The development of the risk ranking approach is an iterative process.

> Choose your method based on the resources available and level of complexity and accuracy required.

> Select the metric that will most accurately reflect severity and likelihood and meets the risk manager’s goals for the risk ranking.

> Metrics for microbial and chemical hazards might be distinct.

> Country data are ideal; however, if data from your country are not available, regional or international estimates (e.g. WHO FERG estimates, GEMS database) can be used instead. Data can also be obtained from literature review.

> The risk ranking effort will be as good as the data used to inform it. Timely, representative, and unbiased data are essential for accurate and precise estimates. Even with high quality data, uncertainty and variability will be present and need to be taken into consideration.

> Expert elicitation is a powerful tool that, if used adequately, can be very helpful in addressing data gaps.

> Clearly state the limitations associated with the approach selected, limitations of the metrics, biases of the data, and try to quantify or describe the uncertainty in the data even if in a narrative (qualitative) manner.

2.3 STEP 3: CONDUCT THE RISK RANKING ANALYSIS AND REPORT RESULTS

The next step is to estimate and rank the risks based on the metrics for severity and likelihood using the selected method. There is a significant body of literature on how to estimate risk and conduct risk rankings for microbial and chemical hazards; the main methods are summarized in Annex F. Further, this guidance will present two case studies (Section 3) that, together with Annex F, will guide the selection of the risk ranking method.

Results from a risk ranking exercise must be interpreted with caution, taking into consideration the bias, uncertainty, and variability inherent in the metrics, and the data and method used in the analysis. The steps, assumptions, and processes used to conduct the risk ranking should be well documented to ensure transparency and reproducibility. It should be noted if local and/or regional variations in hazards and foods consumed were taken into consideration. It is also imperative to explain the strengths and limitations of the approach, so decision makers can take that into consideration. Reporting of results must be objective and include an executive summary with the main conclusions and limitations of the risk ranking exercise. The presentation of the results should be tailored to the targeted audience and easy to communicate.

Relative risk ranking charts are an appealing way to present the results of the risk ranking and can be easily understood (Figure 8). Plotting the two dimensions of risk (i.e. severity and likelihood) on a scatterplot provides high-level information of the relative risk ranking for each of the hazards and might be enough to inform risk management decisions. Depending on the scope of the risk ranking, it may be appropriate to present multiple risk ranking charts (e.g. overall, by age group, by hazard type, by food). It is possible that you will need to plot your data on a logarithmic scale and/or normalize your data (See Microbial case study, Section 3.1), since the difference between hazards and/or foods might be so dramatic that you will not be able to visually identify the differences (Figure 9). It is also important to keep in mind the risk metrics that were selected and how they might affect the interpretation of the scatterplot. For example, higher RfDs represent lower risk, making the interpretation of the scatterplot less intuitive; inverting the axis provides the standard visual that may be more intuitive for the targeted audience (Figure 10).

STEP 3 SUMMARY

> Results from a risk ranking exercise must be interpreted with caution, taking into consideration the bias, uncertainty and variability inherent in the metrics, the data and method used in the analysis.

> Plotting the severity and likelihood metrics into a two-dimensional graph is a very effective way to present results.

> Assumptions and limitations need to be clearly described.

2.4 PRIORITIZATION

Once the risk ranking has been conducted, the prioritization of food hazards and/or food safety issues is the next logical step and a critical part of the decision-making process. It is the systematic analysis and prioritization of opportunities to reduce risks from foodborne hazards and/or foods, based on public health impacts and other factors (Box 2). Prioritization produces an action list for the risk manager. The process acknowledges the fact that risk management decisions need to be made in an environment where factors besides public health may need to be considered when making a decision. Those factors may be related to social-cultural considerations, consumer perceptions, economic considerations (e.g. trade impacts), and availability of interventions to control the risk. Therefore, it provides a broader perspective than risk ranking. Prioritization is not addressed in this guidance since another FAO report describes this effort in detail and provides guidance on how to consider multiple factors when making decisions (FAO, 2017).

CHAPTER 3

CASE STUDIES

This section presents two case studies to help the general understanding of how the proposed risk ranking approach can be used in practice. The first case study focuses on microbial hazards and the second one on chemical contaminants in fish. They both follow the three steps described in this guidance document. The case studies are hypothetical examples and do not represent any specific country.

3.1 MICROBIAL CASE STUDY

The National Ministry of Health and Agriculture in Country X is responsible for food safety oversight and has been charged with identifying areas where targeted resources and control measures could have the greatest impact on public health. In the Ministry, there are food inspectors, microbiologists, chemists, and epidemiologists, but limited expertise in risk analysis. The risk ranking needs to be conducted quickly using existing information because there are no funds to collect data to inform the risk ranking. Further, the country has a passive surveillance system and most foodborne illness cases are not reported.

3.1.1 STEP 1: DEFINE THE SCOPE

The first step in the risk ranking process is to define the scope, which includes: (1) defining the purpose, (2) selecting what will be ranked, and (3) screening for relevance.

Define the purpose

Risk managers from the National Ministry of Health and Agriculture convened a 2-day meeting with food safety experts to define the purpose of the risk ranking. In the first day of the discussions, risk managers expressed their need to rank, by public health importance, all foodborne hazards and all foods using a fully quantitative method that could provide them with an estimate of risk to the overall population. After discussion and considerations regarding the level of effort needed, risk managers agreed to restrict the focus of the risk ranking to selected microbial pathogens and foods that are most frequently associated with the pathogens of highest public health impact in the country. Risk managers and food safety experts agreed on a Statement of Concern and a Statement of Purpose and Objectives for the upcoming risk ranking effort:

> Statement of concern: Food contaminated with microbial hazards causes significant public health impact and has been attracting the attention of the media. Further, several cases of foodborne illness have to occur for our surveillance system to detect an outbreak. However, there are too many foods, too many microbial hazards, and few resources to address the issue. The government needs to be able to focus its limited resources on the hazards that have the greatest public health impact and identify the major food categories most commonly associated with those “high impact” hazards.

> Statement of purpose and objectives: Identify the microbial hazards that have the greatest public health impact in the general population and then identify what major food categories are more frequently associated with the top two pathogens.

Select what will be ranked

Once the Statement of Concern and a Statement of Purpose and Objectives have been identified, the technical team at the Ministry had to identify the major food categories and microbiological hazards to be included in the risk ranking. For the food categories, they decided to utilize their existing food categorization scheme. The scheme is composed of 11 groups used to classify food-producing establishments for the purposes of inspection (Table 3). The potential list of microbial hazards provided in Annex C of this guidance was used as a starting point (Table 4). The technical team reviewed the hazard list and confirmed with food safety experts and epidemiologists that no relevant microbial hazards are being excluded from the analysis.

Screen foods and/or hazards for overall relevance and risk potential

The Ministry had limited financial resources and was unable to conduct a ranking of all 46 hazards from Table 4. Therefore, the list needed to be reduced further. The risk analysts developed a decision flowchart using some of the Bradford Hill Criteria to facilitate, in a transparent manner, the screening process (Figure 11).

Members of the risk ranking team and food safety experts from industry and academia reviewed each of the 46 pathogens using the decision flowchart; at the end of the process, 14 pathogens were selected to be included in the final risk ranking (Table 5).

3.1.2 STEP 2: DEVELOP THE APPROACH

The process for developing the risk ranking approach includes (1) selecting the method, (2) choosing the metrics for severity and likelihood, and (3) collecting and evaluating the appropriateness of data. This is not a linear process (i.e. the stages can be conducted in any order) and is often iterative. In this case study, the technical team decided to start the process by collecting and evaluating the appropriateness of the data.

Collect and evaluate appropriateness of data

In reviewing the available data, the team could not find in-country data to estimate the likelihood and severity of illness. However, data from WHO’s FERG (Annex G) were available for 11 of the 14 pathogens selected for the evaluation, with the exception of C. burnetii, C. perfringens and T. saginata. Therefore, the team decided to conduct an expert elicitation to (i) estimate the incidence and DALY/case for those three pathogens and (ii) obtain feedback on the appropriateness of WHO estimates for the remaining 11 hazards for Country X.

In-country experts from governmental agencies, industry, and academia were invited to participate in a two-day meeting to review WHO’s data. At the end of the meeting, experts were able to provide estimates for the incidence and DALY/case for C. burnetii, C. perfringens and T. saginata by comparing with the other foodborne hazards evaluated by WHO. In their deliberations, the experts agreed that it is adequate to use the DALY/case estimate for the subregion for all 11 other pathogens; however, for 7 of the 11 pathogens, the WHO foodborne disease estimates were not appropriate because they did not reflect the incidence in Country X. Therefore, expert opinion was also used to quantify the incidence per 100 000 for the seven pathogens for which WHO incidence rate was not an adequate proxy.

Select the metrics for ranking risks

Since data from FERG were being used to rank pathogens, incidence per 100 000 were selected to represent likelihood and DALY/case to estimate severity. DALYs integrate measures of morbidity and mortality and have been the metric of choice to quantify the burden of illness.

Select the risk ranking method

Since data from FERG were being used to rank pathogens, the risk ranking method was an extension of the FERG method (i.e. top-down, quantitative method to estimate the burden of disease).

3.1.3 STEP 3: CONDUCT THE RISK RANKING ANALYSIS AND REPORT RESULTS

Total DALYs per 100 000 were calculated by multiplying the incidence per 100 000 by the DALYs per case (Table 6). A risk scatterplot for the 14 hazards was also constructed (Figure 12). However, there was very little spread in the data, and it was difficult to visualize the differences between the hazards. Therefore, the data were transformed to a logarithmic scale and plotted (Figure 13) to provide greater detail. Data could have also been normalized and plotted on the logarithmic scale (Figure 14). The following formula was used to normalize the data:

In this particular example, Figures 12 and 13 did not offer additional information than the data presented in Table 6. However, the graphs served as aids to the risk manager and decision maker to quickly visualize the relationship between severity and likelihood for the different hazards being evaluated. These types of graphs could be particularly helpful when the metrics selected to estimate severity and likelihood cannot be directly multiplied to provide an estimate of risk and/or when the individual metrics might be considered in ranking the efforts. An example would be when volume of exports and number of hospitalizations are chosen as measures of likelihood and severity, respectively. These measures should not be multiplied to estimate risk, and therefore scatterplots would be very helpful in providing useful information to decision makers. Similarly, the scatterplots would be helpful if decision makers want to prioritize hazards that have more severe outcomes, regardless of their likelihood of occurring.

In Figure 13, Salmonella spp-non-typhoidal, T. gondii, Hepatitis A, STEC, Shigella, and C. perfringens appear in the upper-right quadrant, indicating that these pathogens are associated with the highest risk to public health. Since the goal of the risk ranking is to identify the top two foodborne pathogens and the major food categories associated with it, Salmonella spp-non-typhoidal and T. gondii were selected based on their total DALYs.

The next task was to identify the major food categories associated with Salmonella spp-non-typhoidal and T. gondii. Data from passive surveillance systems were reviewed to identify potential outbreaks associated with those two pathogens, but the data were sparse, and most outbreaks did not have an identified food vehicle. Therefore, another expert meeting was organized to fill the data gaps. The experts reviewed the outbreak data available and using their knowledge attributed the proportion of Salmonella spp-non-typhoidal and T. gondii illnesses associated with each food category. Using this information, the number of illnesses associated with each food-hazard pair was estimated. Total DALYs for each food-hazard pair were calculated by multiplying the number of illnesses by the DALYs per case. The data are summarized in Table 7. A risk scatterplot for the 14 hazards was also constructed (Figure 14).

Based on Total DALYs, Salmonella in eggs, produce, and dairy products has the highest public health impact. As shown in Figure 15, these results are largely driven by the number of cases rather than the severity of illness. T. gondii in pork, beef and game presents the highest risk to public health from a severity perspective. Risk managers used this information to determine where to implement targeted control strategies.

3.2 CHEMICAL CASE STUDY

Country Y is a small island country that subsists heavily on local seafood for its diet. Recently, negative media attention around health problems and chemical contaminants in domestically produced seafood has caused public concern. The Ministry of Health and Agriculture has been tasked with developing dietary guidelines and advisories for seafoods that pose the highest risk to public health from chemical contaminants. There are some exotic seafood species consumed by indigenous people in the country, but it is limited to a narrow geographic area and, as such, not considered for the dietary guidelines.

3.2.1 STEP 1: DEFINE THE SCOPE

The first step in the risk ranking process is to define the scope, which includes (1) defining the purpose, (2) selecting what will be ranked, and (3) screening for relevance.

Define the purpose

Risk assessors tasked with performing the risk ranking met with risk managers and stakeholders to define the purpose and objectives of the risk ranking. The following Statement of Concern and Statement of Purpose and Objectives and objectives were drafted:

> Statement of Concern: Some seafood products in Country Y may contain levels of metals and other inorganic compounds that pose a risk to human health, particularly for the local fishing populations that subsist on the seafood they catch. To help safeguard public health, the Ministry of Health and Agriculture has been tasked with developing dietary guidelines and advisories for seafood that poses the highest risk to public health from chemical contaminants. To achieve this, the food safety risk managers must identify the highest risk chemical contaminants and which seafood products are most likely to be affected.

> Statement of Purpose and Objectives: Identify the chemical contaminants (i.e. metal and inorganic compounds) in seafood that pose the greatest risk to public health and the main seafood products associated with those contaminants.

The answers to these questions allowed the Ministry to develop consumption guidelines and advisories to help safeguard public health and educate worried consumers.

Select what will be ranked

A comprehensive literature review was conducted to identify common metal and inorganic compounds detected in seafood within Country Y and neighbouring countries. Further, the list was sent to experts in the field and a couple more compounds were added to produce a potential list of chemical contaminants in seafood (Table 8).

The risk managers, together with risk analysts, also had to determine which seafood products to include in the risk ranking. The technical team first reviewed the available data on the type and amount of seafood caught in the country. The team then spoke with relevant stakeholders, including local fishermen and the Ministry of Agriculture, to obtain additional information on the following questions:

> What are all the seafood products caught or produced in the country?

> What are the most consumed seafood products in the country and in the local populations?

Similar to the process above for hazards, a comprehensive list of potential seafoods to include in the risk ranking was developed based on the amount caught, produced, and consumed in the country (Table 9).

Screen foods and/or hazards for overall relevance and risk potential

To determine whether or not to include each initial chemical in the risk ranking process, analysts asked a single question:

Is the chemical contaminant present in foods consumed or produced in the country?

Generally, publicly available data on chemical contaminants in foods are limited, but data are needed to answer this question. The technical team successfully enlisted the help of the expert researcher at the local university who had just submitted to a peer-review journal an article on chemicals in local seafood. After reviewing this report and speaking with the expert, the chemical list from Table 8 was further screened. The results of the university report identified detectable levels of four chemicals in at least one seafood product: cadmium, methylmercury, lead, and arsenic, and thus, those chemicals were selected to be included in the final evaluation.

To determine whether or not to include a food from Table 9 in the final food list for risk ranking, the technical team developed a decision flowchart (Figure 15). It was assumed that processing did not increase chemical contamination. Since the chemicals detected were metals, food preparation steps such as cooking would not further reduce contaminant concentrations, and it was assumed that any sediment or soil would be washed off the seafood prior to consumption. Based on the decision flowchart exercise, the food list was narrowed down to those that had detectable levels of chemical contaminants (N = 10). Any chemical contaminants that were not detected above the laboratory reporting limit were excluded from further evaluation. This initial step is an inclusion step based on any detection, rather than the level of detection. To be conservative, the technical team decided to include all 10 foods that had detectable levels of contaminants in the risk ranking because some of the contaminants being considered could potentially pose a health risk even if they were consumed in moderation (Table 10).

Further, rather than consider all four hazards in all ten foods, hazard-food pairs were screened one more time based on whether or not a given hazard was detected in a given food in known testing databases (e.g. WHO GEMS, RASFF, EFSA Monitoring – See Annex D), recent peer-reviewed journal articles, or based on expert elicitation. Ten pairs were selected to be included in the risk ranking (Table 11).

Chemicals were excluded from further evaluation if there was no detectable chemical contamination on food, and it was also clear that food processing or packaging would not introduce contamination of that chemical. Chemicals were also excluded if detectable contamination was found, but the concentrations were clearly below known health-based guidance values. If the chemical contaminant was detected above health-based guidance values, it was unclear, or there was no established guidance value, it was included in the risk ranking process even if food preparation reduced the concentration, food processing did not add more contamination (or it was unknown), and the chemical did not knowingly cause health effects. We included the chemical in the risk ranking process in these cases to ensure that these uncertainties were considered. If further evaluation indicated these chemicals were of lesser concern, the risk ranking process would show this finding by listing the chemical further down the risk ranking list.

3.2.2 STEP 2: DEVELOP THE APPROACH

The second step in the risk ranking process is to develop the approach, which consists of three stages: (1) select the risk ranking method, (2) select the metric for ranking risks, and (3) collect the data. As stated previously, this is not a linear process (i.e. the stages can be conducted in any order) and is often iterative. In this case study, the technical team decided to start the process by selecting the risk ranking method.

Select risk ranking method

The risk manager chose a bottom-up approach to perform risk ranking because outbreak or epidemiological data for chemicals in foods is rarely available. While probabilistic risk assessments methods are the gold standard for risk ranking approaches for chemicals, it was infeasible to conduct one in this situation given the lack of data and available resources. Therefore, a decision was made to use a simplified approach using expert elicitation to complete the risk ranking exercise.

Select the metric for ranking risks

Metrics were then selected for the two dimensions of risk. Consumption rate is chosen as a proxy for likelihood of exposure, and reference dose (RfD) is selected to estimate severity. The technical team selected RfDs because they are based on toxicological testing and identify a quantitative non-cancer value for the maximum acceptable daily dose of each chemical. The higher the RfD, the lower the risk, because it means a person can ingest a higher quantity of the chemical before there is increased risk of non-cancer health effects. Using a human health toxicity benchmark to estimate severity allows the various chemicals to be compared against one another to understand their relative toxicity, even in the absence of specific chemical concentration data.

Collect and evaluate appropriateness of data

Country Y did not have detailed consumption data for the population of interest. Consequently, local knowledge was leveraged using expert elicitation for commonly consumed foods and consumption categories to include in the ranking. Expert elicitation from a governmental nutritionist was used to identify high-end consumption patterns for the general population. High-end consumption values (e.g. 95th percentile) were obtained using expert elicitation and may vary from consumption rates for the average population or children (Table 12).

RfDs were obtained from the U.S. EPA’s Integrated Risk Information Systems (IRIS–https://www.epa.gov/iris) and are provided in Table 13. Because there is no RfD available for lead–but it is known to have adverse effects on cognitive development in children and a threshold has not been identified that is acceptable for exposure–the RfD was assumed to be zero.

3.2.3 STEP 3: CONDUCT THE RISK RANKING ANALYSIS AND REPORT RESULTS

Using the quantitative data collected for likelihood and severity, a graph was developed for food-hazard pairs by plotting severity (RfD) versus likelihood (consumption amount) (Figure 16). The graph shows typical consumption of the seafood products of interest in Country Y along with the relative severity of the metals that may be present in certain food-hazard pairs. As the likelihood of exposure increases along the x-axis and RfD decreases along the y-axis, food-hazard pairs increase in risk. For example, Lead-Bream is one of the highest consumed foods with the lowest (most severe) RfD, suggesting it may be one of the highest risk foods. On the other hand, Cadmium-Mackerel is consumed in lower amounts and has a relatively higher (less severe) RfD, suggesting it may be a relatively lower risk food.

Based on the results, three food-hazard pairs were identified as the highest risk for public health: tuna (methylmercury), flounder (lead) and bream (lead). Four food-hazard pairs were identified as moderate risk (arsenic in clams, sardines, lobster, and mussels). Cadmium was identified to be of potential concern in mackerel, shark and carp, but the RfD was higher (i.e. less harmful to health) compared to the other chemicals of concern. These results informed a related prioritization effort to identify where resources for developing dietary guidelines and advisories should be focused.

Understanding the relative severity and likelihood of each chemical posing an issue in foods is useful for decision-making. This information can be paired with available concentration data to further evaluate which chemical hazards are of higher concern based on their toxicity.

CHAPTER 4

CONCLUSION

The objective of this guidance was to provide direction to National Food Safety Authorities on how to start ranking microbial and chemical risks in a more systematic and structured manner to better inform food safety decisions. The risk ranking approach presented here provides a logical framework for making relative risk comparison assessments in a way that is transparent and evidence based. However, it is important to note that the results may not be actual estimates of risk, but rather relative rankings. To obtain more accurate estimates of risk and the associated uncertainty and variability, full quantitative risk assessments using high quality and representative data would need to be conducted. Further, microbial and chemical hazards could theoretically be evaluated and ranked together, but from a practical standpoint, this may not be possible due to the lack of data and uncertainties associated with the data necessary to quantify a metric for likelihood and severity that could be applied to both hazard types. As with risk assessments, risk ranking will highlight innumerable data gaps and research needs. Those gaps should be used to guide and inform data collection and research efforts to improve the risk estimates.

The food supply and our knowledge are in constant flux. Organisms mutate; new and more virulent strains emerge. Novel production systems and processes might help control the hazard or introduce chemical and microbial hazards not previously found in foods. Novel data and information are produced addressing some of the data gaps and changing safety standards. The population demographics also change, including life expectancy, birth weights, and percentage of the population that is immunocompromised, potentially making certain individuals more susceptible to specific hazards (FAO/WHO, 2006). For all these reasons, risk ranking should not be a one-time effort but rather an ongoing activity that is core to a risk-based food safety system.

Finally, the effort must be backed by political will and a recognition from high levels of the government to determine how risk ranking results can be used in decision-making on an ongoing basis. This risk ranking guidance is a work in progress, but we hope it will enable countries to more effectively inform food safety priorities.

REFERENCES

Anderson, M., Jaykus, L.A., Beaulieu, S. & Dennis, S. 2011. Pathogen-produce pair attribution risk ranking tool to prioritize fresh produce commodity and pathogen combinations for further evaluation (P3ARRT). Food Control, 22: 1865–1872.

Australia New Zealand Food Authority (ANZFA). 2001. Food Safety: The priority classification system for food businesses: A risk-based system designed to classify food businesses into priority ratings based on the risk they present to public health and safety. [Cited 12 August 2020]. https://www.foodstandards.gov.au/publications/documents/ANZFA_1578_Info_Paper__final.pdf.

Australia Department of Health (ADH). 2012. Risk Profiling Framework. [Cited 12 August 2020]. https://www1.health.gov.au/internet/fr/publishing.nsf/Content/risk-profiling-framework.

Batz, M., Henke, E. & Kowalcyk, B. 2013. Long-term consequences of foodborne infections. Infectious Disease Clinics of North America, 27(3): 599–616.

Batz, M.B., Hoffmann, S. & Morris Jr., J.G. 2012. Ranking the Disease Burden of 14 Pathogens in Food Sources in the United States Using Attribution Data from Outbreak Investigations and Expert Elicitation. Journal of Food Protection, 75(7): 1278–1291.

Batz, M.B., Doyle, M.P., Morris Jr., J.G., Painter, J., Singh, R., Tauxe, R.V., Taylor, M.R. & Lo Fo Wong, D.M.A. 2005. Attributing Illness to Food. Emerging Infectious Diseases, 11(7): 993–999.

Codex Alimentarius Commission (CAC). 2013. Guidance for Governments on Prioritizing Hazards in Feed. CAC/GL 81-2013.

CAC. 2011. Working Document for Information and Use in Discussions Related to Contaminants and Toxins in the GSTFF. CF/5 INF/1.

CAC. 1999. Principles and Guidelines for the Conduct of Microbiological Risk Assessment. CAC/GL-30.

Chen, Y, Dennis, S.B., Hartnett, E., Paoli, G., Pouillot, R., Ruthman, T. & Wilson, M. 2013. FDA-iRISK–A comparative risk assessment system for evaluating and ranking food-hazard pairs: case studies on microbial hazards. Journal of Food Protection, 76: 376–385.

Cooke, R.M. 1991. Experts in Uncertainty: Opinion and Subjective Probability in Science. Oxford University Press.

Cooke, R.M. & Goossen, L.H.J. 2007. TU Delft Expert Jugement Data Base. Reliability Engineering and System Safety, 93: 657–74.

Devleesschauwer B., Scharff R., Kowalcyk B. & Havelaar A. 2018. Burden and risk assessment of foodborne disease. Food Safety Economics Incentives for a Safer Food Supply, pp. 83-106. Springer International Publishing.

European Centre for Disease Prevention and Control (ECDC). 2015. The European Union summary report on trends and sources of zoonoses, zoonotic agents and food-borne outbreaks in 2014. EFSA Journal, 13(12): 4329.

European Food Safety Authority (EFSA). 2017. Guidance on the use of the weight of evidence approach in scientific assessments. EFSA Journal, 15(8): 4971.

EFSA. 2015. Development of a risk ranking toolbox for EFSA BIOHAZ Panel. EFSA Journal, 13(1): 3939.

EFSA. 2013a. Scientific Opinion on the public health hazards to be covered by inspection of meat (bovine animals). EFSA Journal, 11(6): 3266.

EFSA. 2013b. Scientific Opinion on the public health hazards to be covered by inspection of meat (solipeds). EFSA Journal, 11(6): 3263.

EFSA. 2013c. Scientific Opinion on the public health hazards to be covered by inspection of meat from farmed game. EFSA Journal, 11(6): 3264.

EFSA. 2013d. Scientific Opinion on the public health hazards to be covered by inspection of meat from sheep and goats. EFSA Journal, 11(6): 3265.

EFSA. 2012a. Scientific Opinion on the development of a risk ranking framework on biological hazards. EFSA Journal, 10(6): 2724.

EFSA. 2012b. Scientific Opinion on the public health hazards to be covered by inspection of meat (poultry). EFSA Journal, 10(6): 2741.

EFSA. 2012c. Scientific Opinion on public health risks represented by certain composite products containing food of animal origin. EFSA Journal, 10(5):2662, pp. 132.

EFSA. 2011. Scientific Opinion on the public health hazards to be covered by inspection of meat (swine). EFSA Journal, 9(10): 2351.

Evers, E.G. & Chardon, J.E. 2010. A swift quantitative microbiological risk assessment (sQMRA) tool. Food Control, 21(3): 319-330.

FAO (Food and Agriculture Organization of the United Nations). 2017. Food Safety Risk Management: evidence-informed policies and decisions, considering multiple factors. Rome, FAO.

FAO and WHO (World Health Organization). 2012. Multicriteria-Based Ranking for Risk Management of Foodborne Parasites. Report of a Joint FAO/WHO Expert Meeting, September 3–7, 2012. Rome, FAO.

FAO and WHO. 2009. Environmental Health Criteria 240, Principles and Methods for the Risk Assessment of Chemicals in Food. Geneva, Switzerland.

FAO and WHO. 2008. Microbiological hazards in fresh leafy vegetables and herbs: Meeting Report. Microbiological Risk Assessment Series No. 14. Rome, Italy. pp. 151.

FAO and WHO. 2006. Food safety risk analysis: a guide for national food safety authorities. FAO Food and Nutrition Paper No. 87. Rome, FAO.

FAO and WHO. 1997. Risk Management and Food Safety. FAO Food and Nutrition Paper No. 65. Rome, FAO.

Food and Drug Administration (FDA). 2014. A Quantitative Assessment of the Net Effects on Fetal Neurodevelopment from Eating Commercial Fish. [Cited 12 August 2020]. https://www.fda.gov/media/88491/download.

Food Safety Authority of Ireland (FSAI). 2014. Risk-Based Approach to Developing the National Residue Sampling Plan (For Veterinary Medicinal Products and Medicated Feed Additives in Domestic Animal Production). Report of the Scientific Committee of the Food Safety Authority of Ireland. [Cited 12 August 2020]. http://www.fsai.ie/publications_riskbased_residue_sampling.

United States Government Accountability Office (GAO). 2016. Imported Food Safety: FDA’s Targeting Tool Has Enhanced Screening, but Further Improvements Are Possible. Report to the Chairman Subcommittee on Europe, Eurasia, and Emerging Threats, Committee on Foreign Affairs House of Representatives. GAO-16-399. [Cited 12 August 2020]. https://www.gao.gov/assets/680/677538.pdf.

Groth III, E. 2010. Ranking the contributions of commercial fish and shellfish varieties to mercury exposure in the United States: Implications for risk communication. Environmental Research, 110: 226–36.

Guillier, L., Thebault, A., Gauchard, F., Pommepuy, M., Guignard, A. & Malle, P. 2011. A Risk-Based Sampling Plan for Monitoring of Histamine in Fish Products. Journal of Food Protection, 74(2): 302–10.

Hald, T., Aspinall, W., Devleesschauwer, B., Cooke, R., Corrigan, T., Havelaar, A.H., Gibb, H.J., Torgerson, P.R., Kirk, M.D., Angulo, F.J., Lake, R.J., Speybroeck, N. & Hoffmann, S. 2016. World Health Organization Estimates of the Relative Contributions of Food to the Burden of Disease Due to Selected Foodborne Hazards: A Structured Expert Elicitation. PLoS ONE 11(1): e0145839.

Havelaar, A.H., Haagsma, J.A., Mangen, M., J.J., Kemmeren, J.M., Verhoef, L.P.B., Vijgen, S.M.C., Wilson, M., Friesema, I.H.M., Kortbeek, L.M., van Duynhoven, Y.T.H.P. & van Pelt, W. 2012. Disease burden of foodborne pathogens in the Netherlands, 2009. International Journal of Food Microbiology, 156: 231–38.

Health Canada. 2007. Risk Categorization Model for Food Retail/Food Service Establishments. 2nd Edition. Federal/Provincial/Territorial Committee on Food Safety Policy. [Cited August 12 2020]. https://www.canada.ca/en/health-canada/corporate/about-health-canada/reports-publications/health-products-food-branch/risk-categorization-model-food-retail-food-service-establishments-second-edition.html.

Hill, A.B. 1965. The Environment And Disease: Association Or Causation? Proceeding of the Royal Society of Medicine, 58:295–300.

Hoffmann, S., Maculloch, B. & Batz, M. 2015. Economic Burden of Major Foodborne Illnesses Acquired in the United States, EIB-140, U.S. Department of Agriculture, Economic Research Service, May 2015. [Cited 12 August 2020]. https://www.ers.usda.gov/webdocs/publications/43984/52807_eib140.pdf.

Hoffmann S, Fischbeck P, Krupnick A, McWilliams M. 2007. Using Expert Elicitation to Link Foodborne Illnesses in the United States to Foods. Journal of Food Protection, 70(5): 1220–1229.

International Programme on Chemical Safety (IPCS). 2019. Chemical safety information from intergovernmental organizations (INCHEM).

Institute of Medicine and National Research Council (IMNRC). 2010. Enhancing Food Safety: The Role of the Food and Drug Administration. Washington, DC: The National Academies Press.

Jakobsen, L.S., Granby, K., Knudsen, V.K., Nauta, M., Pires, S.M. & Poulsen, M. 2016. Burden of disease of dietary exposure to acrylamide in Denmark. Food and Chemical Toxicology, 90: 151–59.

Kemmeren, J.M., Mangen, M.-J.J., van Duynhoven, Y.T.H.P & Havelaar, A.H. 2006. Priority setting of foodborne pathogens Disease burden and costs of selected enteric pathogens. RIVM report 330080001/2006. [Cited 12 August 2020]. https://www.rivm.nl/bibliotheek/rapporten/330080001.pdf.

Low, F., Lin, H.M., Gerrard, J.A., Cressey, P.J. & Shaw, I.C. 2004. Ranking the risk of pesticide dietary intake. Pest Manag Sci, 60: 842–48.

Mangen, M.-J.J., Bouwknegt, M., Friesema, I.H.M., Haagsma, J.A., Kortbeek, L.M., Tariq, L., Wilson, M., van Pelt, W. & Havelaar, A.H. 2015. Cost-of-illness and disease burden of food-related pathogens in the Netherlands, 2011. International Journal of Food Microbiology, 196: 84–93.

Mataragas, M., Skandamis, P.N. & Drosinos, E.H. 2008. Risk profiles of pork and poultry meat and risk ratings of various pathogen/product combinations. International Journal of Food Microbiology, 126: 1–12.

Morgan, M. G., Florig, H.K., DeKay, M. L. & Fischbeck, P. 2000. Categorizing risks for risk ranking. Risk Analysis, 20: 49–58.

New Zealand Food Safety Authority (NZFSA). 2006. Risk Ranking: Development of a Single Metric for Risk Ranking by the NZFSA. Lake, R. Client Report FW06109. Christchurch, New Zealand. [Cited 12 August 2020]. https://www.mpi.govt.nz/dmsdocument/25946/direct.

Painter, J.A., Hoekstra, R.M., Ayers, T., Tauxe, R.V., Braden, C.R., Angulo, F.J. & Griffin, P.M. 2013. Attribution of Foodborne Illnesses, Hospitalizations, and Deaths to Food Commodities by using Outbreak Data, United States, 1998–2008. Emerging Infectious Diseases, 19(3): 407–15.

Pires, S.M., Vieira, A.R., Hald, T. & Cole, D. 2014. Source Attribution of Human Salmonellosis: An Overview of Methods and Estimates. Foodborne Pathogens and Disease, 11(9): 667–76.

Pires, S.M. 2013. Assessing the applicability of currently available methods for attributing foodborne disease to sources, including food and food commodities. Foodborne Pathogens and Disease, 10(3): 206–13.

Pires, S.M., Evers, E.g. van Pelt, W., Ayers, T., Scallan, E., Angulo, F.J., Havelaar, A.H. & Hald, T. 2009. Attributing the human disease burden of foodborne infections to specific sources. Foodborne Pathogens and Disease, 6(4): 417–424.

Ruzante, J.M., Davidson, V.J., Caswell, J., Fazil, A., Cranfield, J.A.L., Henson, S.J., Anders, S.M., Schmidt, C. & Farber, J.M. 2010. A Multifactorial Risk Prioritization Framework for Foodborne Pathogens. Risk Analysis, 30(5): 724–42.

Scallan, E., Hoekstra, R.M., Angulo, F.J., Tauxe, R.V., Widdowson, M.A., Roy, S.L., Jones, J.L. & Griffin, P.M. 2011. Foodborne Illness Acquired in the United States– Major Pathogens. Emerging Infectious Diseases, 17(1): 7–15.

Sumner, J., Ross, T., Jenson, I. & Pointon, A. 2005. A risk microbiological profile of the Australian red meat industry: Risk ratings of hazard–product pairings. International Journal of Food Microbiology, 105: 221–32.

Sumner, J. & Ross, T. 2002. A semi-quantitative seafood safety risk assessment. International Journal of Food Microbiology, 77: 55–59.

United States Department of Agriculture (USDA) & Food Safety and Inspection Service (FSIS). 2015. Foodborne Illness Source Attribution Estimates for Salmonella, Escherichia coli O157 (E. coli O157), Listeria monocytogenes (Lm), and Campylobacter using Outbreak Surveillance Data. Interagency Food Safety Analytics Collaboration (IFSAC) Project. [Cited 12 August 2020]. https://www.fsis.usda.gov/wps/wcm/connect/df7eebf6-911c-47c4-aee6-36b60ea04663/IFSAC-Project-Report-022415.pdf?MOD=AJPERES.

USDA & United States Environmental Protection Agency (US EPA). 2012. Microbial Risk Assessment Guideline: Pathogenic Organisms with Focus on Food and Water. FSIS Publication No. USDA/FSIS/2012-001; EPA Publication No. EPA/100/J12/001. [Cited 12 August 2020]. https://www.epa.gov/sites/production/files/2013-09/documents/mra-guideline-final.pdf.

USDA & FDA. 2003. Quantitative Assessment of Relative Risk to Public Health From Foodborne Listeria Monocytogenes Among Selected Categories of Ready-to-Eat Foods. Listeria monocytogenes Risk Assessment. [Cited 12 August 2020]. https://www.fda.gov/media/124721/download.

US EPA. 2011. Vocabulary Catalog. Integrated Risk Information System (IRIS) Glossary. [Cited 12 August 2020]. https://iaspub.epa.gov/sor_internet/registry/termreg/searchandretrieve/termsandacronyms/search.do.

US EPA. 2019 Integrated Risk Information System (IRIS) (available at http://www.epa.gov/iris).

van der Fels-Klerx, H.J., Asselt, E.D. van, Raley, M., Poulsen, M., Korsgaard, H., Bredsdorff, L., Nauta, M., Flari, V., d’Agostino, M., Coles, D. & Frewer, L. 2015. Critical review of methodology and application of risk ranking for prioritisation of food and feed related issues, on the basis of the size of anticipated health impact. EFSA Supporting Publication 2015: EN 710.

van Asselt, E.D., van der Spiegel, M., Noordam, M.Y., Pikkemaat, M.G. & van der Fels-Klerx, H.J. 2013. Risk ranking of chemical hazards in food–A case study on antibiotics in the Netherlands. Food Research International, 54: 1636–42.

WHO. 2015. WHO Estimates of the Global Burden of Foodborne Diseases. Foodborne Disease Burden Epidemiology Reference Group 2007-2015.

WHO. 2019 Metrics: Population attributable fraction (PAF).

GLOSSARY

Benchmark Dose: A dose of a substance associated with a specified low incidence of risk, generally in the range of 1–10 percent, of a health effect; the dose associated with a specified measure or change of a biological effect (FAO/WHO, 2009).

Disability-Adjusted Life Year (DALY): A health gap measure that combines the years of life lost due to premature death (YLL) and the years lived with disability (YLD) from a disease or condition, for varying degrees of severity, making time itself the common metric for death and disability. One DALY equates to 1 year of healthy life lost (WHO, 2015).

Dose-Response Assessment: The determination of the relationship between the magnitude of exposure (dose) to a chemical, biological, or physical agent and the severity and/or frequency of associated adverse health effects (response) (CAC, 1999).

End-point: Qualitative or quantitative expression of a specific factor with which a risk may be associated as determined through an appropriate risk assessment. (FAO/WHO, 2009).

Excess Lifetime Risk: The additional or extra risk of developing cancer due to exposure to a toxic substance incurred over the lifetime of an individual (US EPA, 2016).

Exposure Assessment: The qualitative and/or quantitative evaluation of the likely intake of biological, chemical, and physical agents via food as well as exposures from other sources if relevant (CAC, 1999).

Foodborne Disease: A foodborne disease (FBD) can be defined as a disease commonly transmitted through ingested food. FBDs comprise a broad group of illnesses, and may be caused by microbial pathogens, parasites, chemical contaminants, and biotoxins (WHO, 2015).

Hazard: A biological, chemical, or physical agent in, or condition of, food with the potential to cause an adverse health effect (CAC, 1999).

Hazard Characterization: The qualitative and/or quantitative evaluation of the nature of the adverse health effects associated with the hazard. For the purpose of Microbiological Risk Assessment, the concerns relate to microorganisms and/or their toxins (CAC, 1999).

Hazard Identification: The identification of biological, chemical, and physical agents capable of causing adverse health effects and which may be present in a particular food or group of foods (CAC, 1999).

Likelihood: The probability that an event (e.g. illness) will happen. It can be expressed either qualitatively or quantitatively.

Lowest-Observed-Adverse-Effect Level (LOAEL): Lowest concentration or amount of a substance, found by experiment or observation, that causes an adverse alteration of morphology, functional capacity, growth, development or lifespan of the target organism distinguishable from normal (control) organisms of the same species and strain under the same defined conditions of exposure (FAO/WHO, 2009).

No-Observed-Adverse-Effect Level (NOAEL): Greatest concentration or amount of a substance, found by experiment or observation, that causes no adverse alteration of morphology, functional capacity, growth, development or lifespan of the target organism distinguishable from those observed in normal (control) organisms of the same species and strain under the same defined conditions of exposure (FAO/WHO, 2009).

Population Attributable Fraction: The proportional reduction in population disease or mortality that would occur if exposure to a risk factor were reduced to an alternative ideal exposure scenario (WHO, 2019).

Prioritization: The systematic analysis and ordering of foodborne hazards or food safety issues based on a consideration of public health impacts resulting from risk ranking, and other factors such as social, economic, and political considerations.

Qualitative Risk Assessment: A Risk Assessment based on data which, while forming an inadequate basis for numerical risk estimations, nonetheless, when conditioned by prior expert knowledge and identification of attendant uncertainties permits risk ranking or separation into descriptive categories of risk (CAC, 1999).

Quantitative Risk Assessment: A risk assessment that provides numerical expressions of risk and indication of the attendant uncertainties (stated in the 1995 Expert Consultation definition of Risk Analysis) (CAC, 1999).

Quality-Adjusted Life Year (QALY): This is a unit of health care outcomes that adjusts gains (or losses) in years of life subsequent to a health care intervention by the quality of life during those years. QALYs can provide a common unit for comparing cost-utility across different interventions and health problems. Other units for measuring health outcomes include DALYs and healthy-years equivalents (HYEs) (USDA and US EPA, 2012).

Reference Dose (RfD): An estimate of the daily exposure dose that is likely to be without deleterious effect even if continued exposure occurs over a lifetime. Related terms: Acceptable daily intake, Health-based guidance value, Provisional maximum tolerable daily intake, Tolerable daily intake (FAO/WHO, 2009).

Risk: A function of the probability of an adverse health effect and the severity of that effect, consequential to a hazard(s) in food (CAC, 1999).

Risk Analysis: A process consisting of three components: risk assessment, risk management, and risk communication (CAC, 1999).

Risk Assessment: A scientifically based process consisting of the following steps: (i) hazard identification, (ii) hazard characterization, (iii) exposure assessment, and (iv) risk characterization (CAC, 1999).

Risk Characterization: The qualitative and/or quantitative estimation, including attendant uncertainties, of the probability of occurrence and severity of known or potential adverse health effects in a given population based on hazard identification, hazard characterization, and exposure assessment (CAC, 1999).

Risk Communication: The interactive exchange of information and opinions throughout the risk analysis process concerning risk, risk-related factors and risk perceptions, among risk assessors, risk managers, consumers, industry, the academic community and other interested parties, including the explanation of risk assessment findings and the basis of risk management decisions (CAC, 1999).

Risk Estimate: Output of risk characterization (CAC, 1999).

Risk Management: The process, distinct from risk assessment of weighing policy alternatives, in consultation with all interested parties, considering risk assessment and other factors relevant for the health protection of consumers and for the promotion of fair trade practices, and, if needed, selecting appropriate prevention and control options (CAC, 1999).

Risk Profile: A description of the food safety problem and its context (FAO/WHO, 1997).

Risk Ranking: The systematic analysis and ordering of foodborne hazards and/or foods in terms of the public health risk based on the likelihood and severity of adverse impacts on human health in a target population.

Screening: High-level screening of hazards based on predefined criteria to rapidly sort those that should be further included in the ranking exercise, reducing the total number being evaluated. It can also be called risk screening.

Sensitivity Analysis: A method used to examine the behaviour of a model by measuring the variation in its outputs resulting from changes to its inputs (CAC, 1999).

Severity: The magnitude of the potential harmful consequences from an event (e.g. illness). It can be expressed either qualitatively or quantitatively.

Slope Factor: An upper bound, approximately a 95 percent confidence limit, on the increased cancer risk from a lifetime exposure to an agent. This estimate, usually expressed in units of proportion (of a population) affected per mg/kg-day, is generally reserved for use in the low-dose region of the dose-response relationship, that is, for exposures corresponding to risks less than 1 in 100 (US EPA, 2016).

Source Attribution: The partitioning of the human burden of a particular disease to specific sources. With regard to foodborne diseases, source attribution can be conducted at various points along the food distribution chain, from the animal reservoir to the point of consumption (WHO, 2015).

Statement of Concern: Problem statement that summarizes the foods, hazards, and populations that will be addressed in the risk ranking.

Statement of Purpose and Objectives: Concise paragraph describing the management goals of the risk ranking effort.

Toxicity: The potential of a substance to cause injury (adverse reaction) to a living organism (FAO/WHO, 2009).

Transparent: Characteristics of a process where the rationale, the logic of development, constraints, assumptions, value judgements, decisions, limitations, and uncertainties of the expressed determination are fully and systematically stated, documented, and accessible for review (CAC, 1999).

Uncertainty Analysis: A method used to estimate the uncertainty associated with model inputs, assumptions, and structure/form (CAC, 1999).

Weight of Evidence: A process in which all of the evidence considered relevant to a decision is evaluated and weighted (FAO/WHO, 2009).

ANNEX A

RISK RANKING EXPERT BIOGRAPHICAL SKETCHES

JOHN BASSETT

John Bassett has over 20 years of experience in risk assessment and risk management in both industry and government roles and is a skilled communicator on risk and food safety. A veterinarian by training, he brings a “farm-to-fork” perspective on food safety challenges. He undertook this work while running his own consultancy, John Bassett Consulting Ltd, which worked with clients in the commercial food industry as well as governments and inter-governmental agencies. He is currently the Food Safety Microbiology Director at Danone.

MICHAEL BATZ

Michael Batz, MSc, is an operations research analyst in the Office of Analytics and Outreach within the Center for Food Safety and Applied Nutrition of the U.S. Food and Drug Administration. Mike has 20 years of experience developing quantitative analyses to improve public health decision making, with a focus on risk ranking, risk prioritization, and foodborne illness source attribution. From 2007-2016, Mike was head of food safety programmes of the Emerging Pathogens Institute at the University of Florida. During that time, he was also Executive Director of the Food Safety Research Collaboration (FSRC), a joint effort of seven institutions working on improving food safety policy and practice. He was previously at the University of Maryland School of Medicine and at Resources for the Future, a non-profit research organization in Washington, DC. Mike has a Master's of Science in Electrical and Computer Engineering, and a dual Bachelor's of Science in Electrical and Computer Engineering and Engineering and Public Policy, both from Carnegie Mellon University.

HENRY CHIN

Dr Henry Chin retired in 2013 as Senior Director, Global Scientific and Regulatory Affairs at The Coca-Cola Company. At The Coca-Cola Company, he was responsible for scientific and regulatory policy on food safety issues, including the safety of ingredients and packaging. He has made numerous presentations on food safety and on managing food safety risks in both national and international settings. He is an expert on the analysis and risk assessment of food contaminants including heavy metals, pesticide residues, other environmental contaminants, and on the chemical composition of foods. Prior to joining Coca-Cola in 2005, he was with the National Food Processors Association (NFPA) for nearly 30 years, providing scientific and technical advice to most of the major food companies in the United States. At NFPA, Henry held positions as Vice President of the Laboratory Centers, with responsibility for analytical chemistry, food microbiology, and process development, and as Vice President of Toxicology and Food Chemistry, with responsibility for food safety programmes related to food composition and chemical contaminants.

Henry is a past President of AOAC International (the Association of Official Analytical Chemists) and has been a member of several government and academic advisory panels on various aspects of food safety including food additives and pesticide residues. He was a Task Force leader in the 2004 JIFSAN Workshop on Acrylamide in Foods and a participant in the Codex Committee on Contaminants in Food. He has also served on the Board of Trustees of the Health and Environmental Sciences Institute (HESI) and the International Life Sciences Institute (ILSI), co-chaired the Food Safety and Defense Committee of the International Food Information Council, chaired the Chemicals Management Committee of Grocery Manufacturers Association, was a task force leader on food safety policy at the Institute of Food Technologists (IFT), and was a member of many other professional and trade organizations.

He is a Guest Faculty Scholar in the Center for Health and Risk Communication at George Mason University, where his interest is in developing better ways to communicate about food safety and understanding the impact of social advocacy on regulatory policy. He is also Chair of the U.S. Pharmacopeial Convention’s Hazard Identification and Food Adulteration Expert Panels. Henry also assists several organizations as an independent consultant and advisor on food safety and regulatory affairs.

Henry received his doctorate in Chemistry from the University of Southern California, and bachelor’s degree from the University of California, Berkeley. After completing his doctoral studies, Dr Chin returned to Berkeley as a post-doctoral research associate in Chemistry.

JESSICA COX

Jessica Cox is a Lead Chemist and the Chemical Threat Characterization Project lead for the Chemical Security Analysis Center (CSAC), established under the Department of Homeland Security. This Center is collocated with the Department of Defense assets at Aberdeen Proving Ground, MD and provides a scientific basis for the awareness of chemical threats and the assessment of risk to the American public due to chemical hazards. It is a key interagency resource for chemical threat agent information and has ongoing interagency collaboration with many key departments and agencies. Through the Chemical Threat Characterization efforts Jessica leads a group of scientific experts to characterize chemical hazards to improve decisions, policies and activities designed to protect, prepare, mitigate, and respond to chemical events. Jessica also managed the Chemical Terrorism Risk Assessment for the past ten years which provides a comprehensive end-to-end assessment of the chemical risk to the nation. This risk assessment covers Indoor, Outdoor, Food, Water, and Dermal target areas and spans across all potential chemical attack scenarios for these targets. Jessica is the DHS CSAC lead for all food and medical mitigation efforts.

ARIE HAVELAAR

Dr Arie Hendrik Havelaar is a Preeminent Professor of Global Food Safety and Zoonoses in the Animal Sciences Department, the Institute for Sustainable Food Systems and the Emerging Pathogens Institute of the University of Florida, Gainesville, FL, USA. Before moving to the United Sates in 2014, Arie worked at the Dutch National Institute for Public Health and the Environment, Bilthoven, the Netherlands in various scientific and management roles, most recently as Principal Scientist in the Center for Zoonoses and Environmental Microbiology. He is an emeritus professor of Microbial Risk Assessment at the Institute for Risk Assessment Sciences of the Faculty of Veterinary Medicine, Utrecht University, the Netherlands.

Arie holds an MSc degree in Chemical Engineering with a major in Microbiology from the Delft University of Technology, a PhD in Microbiology from Utrecht University and an MSc in Epidemiology from the Netherlands Institute for Health Sciences at the Erasmus University, all in the Netherlands.

His research focuses on epidemiology and risk assessment of foodborne and zoonotic diseases and their prevention. He has published extensively on the global burden of foodborne disease, including in his role as chair of the WHO Foodborne Disease Burden Epidemiology Reference Group. He contributes to the Feed the Future Innovation Lab for Livestock Systems, leads the “Campylobacter Genetics and Environmental Enteric Dysfunction (CAGED)” project and participates in several other projects focusing on food safety in low- and middle-income countries. His current research is funded by the Bill & Melinda Gates Foundation, the UK Department for International Development and the Florida Department of Health.

GREG PAOLI

Greg Paoli serves as Principal Risk Scientist and Chief Operating Officer at Risk Sciences International. He has been providing advice to regulatory agencies and regulated industry in Canada, the United States and at the international level for over 25 years. He holds a Bachelor’s and Master’s Degree in Engineering from the University of Waterloo.

Greg’s career has spanned a wide spectrum of public risk management domains. This has included the safety of food, drinking water, air quality, consumer products, drugs, medical devices and the blood supply, engineered devices, transportation of dangerous goods, museum collections, emergency management for natural and man-made disasters, and climate change impacts on infrastructure.

Greg has served on a number of expert committees devoted to the risk sciences. He has served on numerous expert committees convened by the World Health Organization and the Food and Agriculture Organization of the United Nations. He was a member of the U.S. National Academy of Sciences committee that issued the 2014 report, A Framework to Guide the Selection of Chemical Alternatives, and the 2009 report, Science and Decisions: Advancing Risk Assessment. He was invited to serve as a member of an expert peer review panel for the US EPA’s Framework for Human Health Risk Assessment to Inform Decision Making.

Greg completed a term as Councillor of the Society for Risk Analysis (SRA) and served two terms as a member of the Editorial Board of the journal Risk Analysis. In 2011, he was awarded the Distinguished Lectureship Award by the Society for Risk Analysis and the scientific society, Sigma Xi.

FERNANDO SAMPEDRO

Dr Fernando Sampedro is a researcher in the School of Public Health at the University of Minnesota. Fernando is a food scientist with a Ph.D. in Food Technology from the Institute of Agrochemistry and Food Technology in Valencia (Spain). His expertise includes food processing, risk-based inspection and quantitative risk assessment tools applied to food and feed safety. He has been involved in international projects with USAID, IICA, PAHO and FAO related to the implementation of national risk-based food inspection and surveillance programmes in several countries in Latin America. He also coordinates an international network in food safety risk analysis (FSRisk) for Latin America. He has also participated in numerous in-person and learning capacity building programmes in risk analysis in East Africa, Southeast Asia and Latin America regions.

NGA TRAN

Dr Nga Tran is a Principal Scientist at Exponent’s Health Sciences Center for Chemical Regulation and Food Safety in Washington, DC. Nga has more than 20 years of experience in dietary exposure and safety assessment. She has extensive experience in evaluating the safety of foods and food ingredients, additives and contaminants, food contact materials, cosmetics, and consumer care products. Nga also has extensive experience in dietary exposure assessment and public health risk modeling, including analysing national food consumption surveys to assess consumption pattern, developing models to integrate food frequency surveys and 24-hour dietary recall data to ascertain usual intake, and implementing models to apportion risk of diseases to dietary and lifestyle factors such as dietary cholesterol and coronary heart diseases. She has also worked extensively on risk ranking methodologies for a wide range of risk management purposes. Her work in the risk ranking arena has included the development of tools to prioritize food risks (both chemical and microbial), risk-based site selection model to prioritize U.S. pharmaceutical manufacturing sites for cGMP inspection, and exposure and risk screening methodologies for consumer personal care products. She has provided technical support and prepared a variety of reports and submissions to regulatory authorities including the USFDA, Health Canada, JECFA, and EFSA. She also has extensive experience in conducting scientific review for the substantiation of health claims.

ANNEX B

EXAMPLES OF FOOD SAFETY RISK RANKING EFFORTS

ANNEX C

LIST OF POTENTIAL MICROBIAL AND CHEMICAL HAZARDS TO BE CONSIDERED IN FOOD SAFETY RISK RANKING EFFORTS

Below is a selected list of microbial and chemical foodborne pathogens that could be considered in food safety risk ranking efforts. This potential list was based on several sources, including WHO’s recent burden of disease study (WHO, 2015), EFSA’s scientific opinions on the risks associated with several types of meat (EFSA 2013a; EFSA 2013b; EFSA 2013c; EFSA 2012b; EFSA 2011), and other studies published internationally (ECDC 2015; Scallan et al., 2011; Hoffmann et al., 2015; Kemmeren et al. 2006) and as such already meets most of all the Bradford Hill criteria described in Section 2.1.3.1. The list presented here are not meant to be comprehensive; some contaminants relevant in certain countries may not be listed. Consequently, it is important to critically evaluate it and modify or edit as needed. We recommend that the potential list of hazards for each ranking be reviewed by in-country food safety experts to ensure a critical hazard has not been omitted.

ANNEX D

POTENTIAL SOURCES OF INFORMATION FOR CONDUCTING RISK RANKING EFFORTS

HAZARD INFORMATION AND SURVEILLANCE–MICROBIAL HAZARDS

> WHO/FERG Online Tool https://extranet.who.int/sree/Reports?op=vs&path=/WHO_HQ_Reports/G36/PROD/EXT/FoodborneDiseaseBurden

> Joint FAO/WHO International Food Safety Authorities Network (INFOSAN) http://www.who.int/foodsafety/areas_work/infosan/en

> New Zealand Pathogen Data Sheets http://www.foodsafety.govt.nz/science-risk/hazard-data-sheets/pathogen-data-sheets.htm

> Risk Profiles:

> New Zealand: http://www.foodsafety.govt.nz/science-risk/risk-assessment/risk-profiles

> US FDA: http://www.fda.gov/Food/FoodScienceResearch/RiskSafetyAssessment/default.htm

> ESFA Biological Hazards Data http://www.efsa.europa.eu/en/data/biological-hazards-data

> European CDC Surveillance http://ecdc.europa.eu/en/activities/surveillance/Pages/index.aspx

> PulseNet International https://www.cdc.gov/pulsenet/participants/international/index.html

> FoodNet Canada http://www.phac-aspc.gc.ca/foodnetcanada/index-eng.php

> Center for Science in the Public Interest (CSPI) Outbreak Alert http://www.cspinet.org/foodsafety/outbreak_report.html

> CDC Foodborne Illness Surveillance, Response, and Data Systems http://www.cdc.gov/foodborneburden/surveillance-systems.html, and:

> Calicinet (Norovirus Surveillance) http://www.cdc.gov/norovirus/reporting/calicinet/index.html

> National Outbreak Reporting System (NORS) http://www.cdc.gov/nors

> National Environmental Assessment Reporting System (NEARS) http://www.cdc.gov/nceh/ehs/nears/index.htm

> Foodborne Outbreak Online Database (FOOD Tool) http://www.cdc.gov/foodborneoutbreaks

> CDC A-Z Index for Foodborne Illness http://www.cdc.gov/foodsafety/diseases/index.html

> CDC Viral Hepatitis Surveillance Program http://www.cdc.gov/hepatitis/statistics

> Bad Bug Book http://www.fda.gov/downloads/Food/FoodborneIllnessContaminants/UCM297627.pdf

> USDA AMS Microbiological Data Program (MDP) https://www.ams.usda.gov/datasets/mdp/mdp-program-data-and-reports

> USDA FSIS Microbiological Baseline Data http://www.fsis.usda.gov/wps/portal/fsis/topics/data-collection-and-reports/microbiology/baseline/baseline

HAZARD INFORMATION AND SURVEILLANCE–CHEMICAL HAZARDS

> International Programme on Chemical Safety (IPCS) Chemical Safety Information from Intergovernmental Organizations (INCHEM) database http://www.inchem.org/pages/ehc.html

> Global Platform for Food Safety Data and Information (FOSCOLLAB) Chemical and Pesticide Dashboards https://extranet.who.int/sree/Reports?op=vs_html&path=/WHO_HQ_Reports/G7/PROD/EXT/chemical_overview&userid=G7_ro&password=inetsoft123

> Joint FAO/WHO Meeting on Pesticide Residues (JMPR) Database–basic information and publications http://apps.who.int/pesticide-residues-jmpr-database

> Global Environmental Monitoring (GEMS)/Food contaminants database https://extranet.who.int/gemsfood

> FDA Index of Chemical Contaminants http://www.fda.gov/Food/FoodborneIllnessContaminants/ChemicalContaminants

> EPA Integrated Risk Information Surveillance System (IRIS) http://www.epa.gov/iris

> EFSA Chemical Contaminants http://www.efsa.europa.eu/en/data/chemical-contaminants-data

> EFSA Veterinary Medicines and Pesticide Residues Data http://www.efsa.europa.eu/en/data/chemical-residues-data

> USDA NASS Agricultural Chemical Use Database https://www.nass.usda.gov/Surveys/Guide_to_NASS_Surveys/Chemical_Use

> USDA Pesticide Data Program (PDP) https://www.ams.usda.gov/datasets/pdp

> EPA Health Effects Databases for Pesticides https://www.epa.gov/pesticide-science-and-assessing-pesticide-risks/databases-related-pesticide-risk-assessment

> Pesticide Handlers Exposure Database (PHED) Empirical Exposure Data for Workers Handling Pesticides https://www.epa.gov/pesticide-science-and-assessing-pesticide-risks/occupational-pesticide-handler-exposure-data

> National Pesticide Information Retrieval System (NPIRS) http://ppis.ceris.purdue.edu

> EXtension TOXicology NETwork (ETOXNET) http://extoxnet.orst.edu

> CDC ATSDR Toxicological Profiles http://www.atsdr.cdc.gov/toxprofiles/index.asp

> The Food Animal Residue Avoidance Databank (FARAD): http://www.farad.org

> FDA Pesticide Program Residue Monitoring Program Reports and Data http://www.fda.gov/Food/FoodborneIllnessContaminants/Pesticides/ucm2006797.htm

> USDA/FSIS Residue Chemistry and Residue Testing http://www.fsis.usda.gov/wps/portal/fsis/topics/data-collection-and-reports/chemistry, and:

> National Residue Sample Results (“Red Book”) http://www.fsis.usda.gov/wps/portal/fsis/topics/data-collection-and-reports/chemistry/red-books/red-book

> Dioxin Monitoring in USDA Regulated Meat and Poultry http://www.fsis.da.gov/wps/portal/fsis/topics/data-collection-and-reports/chemistry/dioxin-related-activites

> Global Health Observatory (GHO) Data: Poison Centres http://www.who.int/gho/phe/chemical_safety/poisons_centres/en

> FDA Analytical Results of the Total Diet Study http://www.fda.gov/Food/FoodScienceResearch/TotalDietStudy/ucm184293.htm

> CDC Agency for Toxic Substances and Disease Registry (ATSDR) Toxicological Profiles http://www.atsdr.cdc.gov/toxprofiles/index.asp

INSPECTIONS AND RECALLS

> US Food Safety and Inspection Service (FSIS) Data: Analysis and Reporting http://www.fsis.usda.gov/wps/portal/fsis/topics/data-collection-and-reports/fsis-data-analysis-and-reporting/data-reporting

> Food inspection and control reports from country importing foods from the country of interest:

> e.g. USDA ERA Report on FDA Refusals of Imported Food Products by Country and Category, 2005-2013 https://www.ers.usda.gov/webdocs/publications/44066/57014_eib151.pdf?v=0

> Canadian Food Inspection Agency http://www.inspection.gc.ca

> FDA Import Refusal Report http://www.accessdata.fda.gov/scripts/importrefusals

> US FDA Recalls, Outbreaks & Emergencies https://www.fda.gov/food/recalls-outbreaks-emergencies

> FDA Bioresearch Monitoring (BIMO) Compliance Programs http://www.fda.gov/ICECI/ComplianceManuals/ComplianceProgramManual/ucm255614.htm

> USDA FSIS Eligible Foreign Establishments List http://www.fsis.usda.gov/wps/portal/fsis/topics/international-affairs/importing-products/eligible-countries-products-foreign-establishments/eligible-foreign-establishments

> USDA FSIS Recalls and Public Health Alerts http://www.fsis.usda.gov/wps/portal/fsis/topics/recalls-and-public-health-alerts

> Rapid Alert System for Food and Feed (RASFF)–Food and Feed Safety Alerts http://ec.europa.eu/food/safety/rasff/index_en.htm

> Canadian Food Inspection Agency (CFIA) http://www.inspection.gc.ca/about-the-cfia/newsroom/food-recall-warnings/complete-listing/eng/1351519587174/1351519588221

> Health Canada Recalls and Safety Alerts http://www.healthycanadians.gc.ca/recall-alert-rappel-avis/index-eng.php?cat=1

BURDEN OF DISEASE/DISEASE SEVERITY

> WHO Estimating the burden of foodborne diseases http://www.who.int/foodsafety/areas_work/foodborne-diseases/ferg/en, and:

> Online tool https://extranet.who.int/sree/Reports?op=vs&path=/WHO_HQ_Reports/G36/PROD/EXT/FoodborneDiseaseBurden

> The Lancet Disability weights for the Global Burden of Disease 2013 study http://www.thelancet.com/pdfs/journals/langlo/PIIS2214-109X(15)00069-8.pdf

> WHO Health Data and Statistics–Global Health Observatory, Global Health Estimates, WHO Mortality Database http://www.who.int/healthinfo/statistics/en

> PLoS One World Health Organization Estimates of the Relative Contributions of Food to the Burden of Disease Due to Selected Foodborne Hazards: A Structured Expert Elicitation http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0145839#sec018

FOOD CONSUMPTION INFORMATION

> WHO Food Safety Databases http://www.who.int/foodsafety/databases/en

> World Food Programme (WFP) Comprehensive Food Security and Vulnerability Analysis (CFSVA) Country Reports https://www.wfp.org/publications/comprehensive-food-security-and-vulnerability-analysis-cfsva-guidelines-first-edition

> FAO/WHO Global Individual Food Consumption Data Tool (FAO/WHO GIFT) http://www.fao.org/nutrition/assessment/food-consumption-database/en

> EFSA Food Consumption Data http://www.efsa.europa.eu/en/data/food-consumption-data

> CDC National Health and Nutrition Examination Survey (NHANES) http://www.cdc.gov/nchs/nhanes/index.htm

> USDA ERS Food Consumption and Nutrient Intakes http://www.ers.usda.gov/data/foodconsumption

> Food Commodity Intake Database (FCID) What We Eat in America: http://fcid.foodrisk.org

> Chronic Individual Food Consumption database–Summary statistics (CIFOCOss) https://extranet.who.int/sree/Reports?op=vs&path=/WHO_HQ_Reports/G7/PROD/EXT/CIFOCOSS_Country&userid=G7_ro&password=inetsoft123

RISK RANKING RESOURCES AND TOOLS

> iRISK: https://irisk.foodrisk.org

> Risk Ranger: http://www.foodsafetycentre.com.au/riskranger.php

> A swift Quantitative Microbiological Risk Assessment (sQMRA): http://foodrisk.org/exclusives/sqmra

> FDA’s Fresh Produce Risk Ranking Tool: https://www.foodrisk.org/resources/display/26

FOOD SOURCE ATTRIBUTION

> WHO estimates of the Relative Contributions of Food to the Burden of Disease Due to Selected Foodborne Hazards: A Structured Expert Elicitation http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0145839#sec018

> WHO Source Attribution Task Force http://www.who.int/foodsafety/areas_work/foodborne-diseases/ferg4/en

RISK ASSESSMENT RESOURCES

> FAO Risk Assessments http://www.fao.org/food/food-safety-quality/scientific-advice/jemra/risk-assessments/en

> New Zealand Food Safety Risk Assessment http://foodsafety.govt.nz/science-risk/risk-assessment

> FDA CFSAN Risk & Safety Assessment http://www.fda.gov/Food/FoodScienceResearch/RiskSafetyAssessment

> US Food Safety and Inspection Service (FSIS) Risk Assessments (by food and hazard) http://www.fsis.usda.gov/wps/portal/fsis/topics/science/risk-assessments/risk-assessments

> WHO Risk Assessment http://www.who.int/foodsafety/risk-analysis/riskassessment/en

> EFSA Environmental Risk Assessment https://www.efsa.europa.eu/en/topics/topic/era

> Joint Committee on Food Additives and Contaminants (JECFA)–Chemical exposure information and available publications for each compound evaluated http://apps.who.int/food-additives-contaminants-jecfa-database/search.aspx

PUBLICATION DATABASES

> EFSA http://www.efsa.europa.eu/en/publications

> FAO http://www.fao.org/publications/en

> PubMed http://www.ncbi.nlm.nih.gov/pubmed

> Scientific Electronic Library Online (SciELO) http://scielo.org/php/index.php?lang=en

> Google Scholar https://scholar.google.com

> WHO http://www.who.int/foodsafety/publications/en

> FAO Statistics Division http://faostat3.fao.org/home/E

ANNEX E

EXAMPLES OF FOOD CATEGORIZATION

THE CENTERS FOR DISEASE CONTROL AND PREVENTION (CDC)[6]

This flowchart was produced by the Interagency Food Safety Analytics Collaboration (a tri-agency that includes CDC, FDA, and USDA-FSIS) in the United States (Figure A1). It has been used for several of the food source attribution projects in the United States.[7]

EUROPEAN FOOD SAFETY AUTHORITY (EFSA)

The EFSA developed a hierarchical Food Classification and Description System for Exposure Assessment called FoodEx2 (version 2). The system classifies 20 main food groups (level 1):

1. Grains and grain-based products

2. Vegetables and vegetable products (including fungi)

3. Starchy roots and tubers

4. Legumes, nuts and oil seeds

5. Fruit and fruit products

6. Meat and meat products (including edible offal)

7. Fish and other seafood (including amphibians, reptiles, snails, and insects)

8. Milk and dairy products

9. Eggs and egg products

10. Sugar and confectionary

11. Animal and vegetable fats and oils

12. Fruit and vegetable juices

13. Non-alcoholic beverages (excepting milk-based beverages)

14. Alcoholic beverages

15. Drinking water (water without any additives except carbon dioxide; includes water ice for consumption)

16. Herbs, spices, condiments

17. Foods for infants and small children

18. Products for special nutritional use

19. Composite food (including frozen products)

20. Snacks, desserts and other foods

Within each main food group, there are several subgroups (levels 2–4). Figure A2 shows a screenshot of the system. To access the tool, see: https://www.efsa.europa.eu/en/data/data-standardisation.[8]

FOOD AND AGRICULTURE ORGANIZATION OF THE UNITED NATIONS (FAO)

FAO provides the definitions and classifications of commodities as used by FAO. There are 20 FAO Commodity Groups, listed below. For each commodity group, there are primary agricultural commodities, which appear in capital letters (e.g. WHEAT), as well as processed or derived products, (e.g. Flour of Wheat). For definitions and classifications of commodities within the 20 groups, see here: http://www.fao.org/ES/faodef/FAODEFE.HTM.

1. Cereals and Cereal Products

2. Roots and Tubers and Derived Products

3. Sugar Crops and Sweeteners and Derived Products

4. Pulses and Derived Products

5. Nuts and Derived Products

6. Oil-Bearing Crops and Derived Products

7. Vegetables and Derived Products

8. Fruits and Derived Products

9. Fibres of Vegetable and Animal Origin

10. Spices

11. Fodder Crops and Products

12. Stimulant Crops and Derived Products

13. Tobacco and Rubber and Derived Products

14. Vegetable and Animal Oils and Fats

15. Beverages

16. Livestock

17. Products from Slaughtered Animals

18. Products from Live Animals

19. Hides and Skins

20. Other Livestock Products

In addition, the FAO International Network of Food Data Systems (INFOODS) maintains various databases of food compositions across the globe that could be used to categorize foods. The FAO/INFOODS Analytical Food Composition Database is a global compendium of analytical data for commonly consumed foods. There are 15 groups and subgroups:[9]

1. Cereals

2. Starchy Roots and Tubers

3. Legumes

4. Nuts and Seeds

5. Vegetables

6. Fruits

7. Meat and Poultry

8. Eggs

9. Fish and Shellfish

a. Finfish

b. Crustaceans

c. Mollusks

10. Milk

11. Herbs and Spices

12. Miscellaneous

FAO also maintains links to country-specific Food Composition Databases around the world here: http://www.fao.org/infoods/infoods/tables-and-databases/en.

WHAT WE EAT IN AMERICA (WWEIA), NATIONAL HEALTH AND NUTRITION EXAMINATION SURVEY (NHANES), UNITED STATES

What We Eat in America (WWEIA), National Health and Nutrition Examination Survey (NHANES), is an integrated national food survey conducted as a partnership between the U.S. Department of Health and Human Services (DHHS) and the U.S. Department of Agriculture (USDA). WWEIA represents the integration of two nationwide surveys: USDA’s Continuing Survey of Food Intakes by Individuals (CSFII) and DHHS’ NHANES. Under the integrated framework, DHHS is responsible for the sample design and data collection. USDA is responsible for the survey’s dietary data collection methodology, development, and maintenance of the food and nutrient databases used to code and process the data, and data review and processing. The two surveys were integrated in 2002 and the integrated dataset is released every two years. There are 15 WWEIA food categories:

1. Milk and Dairy

2. Protein Foods (Meat, Eggs, Soy Products, Beans, Legumes, Nuts, and Seeds)

3. Mixed Dishes (e.g. Pizza, Sandwiches)

4. Grains

5. Snacks and Sweets

6. Fruit

7. Vegetables

8. Beverages, Nonalcoholic

9. Alcoholic Beverages

10. Water

11. Fats and Oils

12. Condiments and Sauces

13. Sugars

14. Infant Formula and Baby Food

15. Other (Protein and Nutritional Powders)

The below example shows the WWEIA food categories, subcategories and individual foods. For more information, see https://www.ars.usda.gov/northeast-area/beltsville-md/beltsville-human-nutrition-research-center/food-surveys-research-group.

Milk and Dairy

Milk

> Milk, whole

> Milk, reduced fat

> Milk, lowfat

> Milk, nonfat

Flavored Milk

> Flavored Milk, whole

> Flavored Milk, reduced fat

> Flavored Milk, lowfar

> Flavored Milk, nonfat

Dairy Drinks and Substitutes

> Milk shakes and other dairy drinks

> Milk substitutes

Cheese

> Cheese

> Cottage/ricotta cheese

Yogurt

> Yogurt, whole and reduced fat

> Yogurt, lowfat and non fat[10]

U.S. FOOD AND DRUG ADMINISTRATION (FDA) REPORTABLE FOOD REGISTRY (RFR) CATEGORIES

The Reportable Food Registry (RFR) is an electronic portal for Industry to report when a food could cause serious adverse health consequences. The Registry helps the FDA better protect public health by tracking patterns and targeting inspections. The RFR applies to all FDA-regulated categories of food and feed, except dietary supplements and infant formula. The FDA produces an RFR annual report summarizing all the reports for the past calendar year. The FDA is considering using these categories for risk ranking efforts to meet the Food Safety Modernization Act mandate that requires the identification of high-risk foods for which enhanced traceability efforts are needed.[11] The 28 RFR Categories and their definitions (if available) are listed below. For more information, see: http://www.fda.gov/Food/ComplianceEnforcement/RFR/default.htm.

1. Acidified/Low-Acid Canned Foods (LACF): Acidified foods mean low-acid foods to which acid(s) or acid food(s) are added. They have a water activity (aw) 0.85 and a finished equilibrium pH <4.6. Low acid [canned] foods means any foods, other than alcoholic beverages, with a water activity (aw) 0.85. T and a finished equilibrium pH 4.6.

2. Animal Feed/Pet Food

3. Bakery: Baked goods including fresh, refrigerated, and frozen products that are ready-to-eat or ready-to-bake products and mixes that require preparation before serving.

4. Beverages: Beverages and beverages bases, both alcoholic and non-alcoholic

5. Breakfast Cereals: Ready-to-eat and instant and regular hot cereals

6. Chocolate/Confections/Candy

7. Dairy: Milk and milk products

8. Dressings/Sauces/Gravies

9. Eggs: Does not include dried egg powders or pasteurized liquid eggs

10. Frozen Foods: Does not include bakery, seafood, pasta, or dairy items

11. Fruit and Vegetable Products: Does not include fresh or frozen produce or Acidified/Low-Acid Canned Foods

12. Game Meats: Does not include USDA-regulated meat

13. Meal Replacement/Nutritional Food and Beverages

14. Multiple Food Products: Use when an RFR has multiple products from different categories

15. Nuts, Nut Products, and Seed Products: Does not include sesame seeds, poppy seeds, or other spices

16. Oil/Margarine

17. Pasta: Can be fresh, refrigerated, frozen, dried, or filled

18. Prepared Foods: Includes ready-to-eat salads, closed-faced sandwiches, appetizers, and side dishes

19. Produce: Fresh cut–Bagged leafy greens and fresh-cut fruits and vegetables

20. Produce–Raw Agricultural Commodities (RAC): Includes fresh produce and herbs

21. Seafood

22. Snack Foods

23. Soup: Includes refrigerated soup, dry mixes, ramen, and bouillon cubes

24. Spices/Seasonings

25. Stabilizers, Emulsifiers, Flavors, Colors, and Texture Enhancers

26. Sweeteners: Includes natural and artificial sweeteners

27. Whole and Milled Grains and Flours

28. Other: For example–non-dairy cheese

ANNEX F

OVERVIEW OF RISK RANKING METHODS

There is a wide range of methods and tools for food safety risk ranking that have been described and evaluated elsewhere (van der Fels-Klerx et al., 2015; EFSA 2015; EFSA, 2012a). Therefore, the goal of this Annex is to provide a brief description of selected methods that were identified by the experts as being the most relevant and applicable to the goals of this guidance. As with the risk assessments methodology, we categorize and describe the risk ranking methods as: qualitative (outcomes without numerical values); semi-quantitative (intermediate format where scores are assigned to express the relative ranking), and quantitative (numerical outcomes with specific units) (FAO/WHO, 2006).

QUALITATIVE METHODS–OUTCOMES WITHOUT NUMERICAL VALUES

Qualitative risk ranking methods produce outcomes without numerical values (i.e. low, medium, high). Qualitative risk ranking methods are most applicable in situations where time is a critical factor and resources and data are limited; they can be conducted relatively quickly with limited resources and data. Ideally, they would be used as an initial step in a continuous long-term risk ranking strategy that incorporates and evaluates more robust data and information over time using more complex risk ranking methods. For example, decision flowcharts can be used as a screening tool for identifying parameters to be included in more complex risk ranking models. Another advantage is that output can be easily used by risk managers or decision makers. The main disadvantages of qualitative risk ranking methods are that they may often not be based on quantitative scientific values and that there may be greater degrees of uncertainty in the output/results of qualitative methods than in quantitative methods.

Decision flowcharts and deliberative processes are two qualitative methods that have low resource and data needs. These methods are included in the qualitative methods presented in van der Fels-Klerx et al., (2015), although the nomenclature in this guidance has been modified to differentiate between a qualitative use of experts to rank risks (herein referred to as “deliberative process”) versus the quantitative approach to elicit expert input to fill data gaps, described as “standard expert elicitation.”

DECISION FLOWCHARTS

Decision flowcharts, also known as decision trees, provide an objective approach for conducting risk rankings using qualitative information on the severity of illness and the likelihood of contamination. A more general flowchart can be developed for microbial and/or chemical food safety hazards, while more specific flowcharts can be developed to address the inherent differences present between the two types of hazards. The outputs from flowcharts are categorical risk bins (e.g. high, medium, or low) that can be used by risk managers or decision makers. Flowcharts can also be used during the screening stage to clearly communicate the choices made to include or exclude a certain hazard or food from your ranking evaluation (van der Fels-Klerx et al., 2015).

All steps in the decision flowcharts need to be well documented so that the flowcharts are reproducible and transparent, especially if the decision flowcharts will be used for decision support. It is also important for each node of the flowchart to result in a clear yes or no decision (i.e. product supports pathogen growth; food is meant to be cooked; the chemical hazard will or will not degrade during processing or cooking). Whenever possible, it is recommended that data are used to quantify the nodes of the flowcharts. For example, a node for “Is the prevalence of hazard X in product Y high?” is vague and open to interpretation. A decision flowchart developed by EFSA to specifically rank risks associated with poultry and inform decision-making around poultry inspection practices in the European Union is provided in Figure A3 as an example, but note, in this example “high” is not clearly defined in the decision flowcharts. Ideally, “high” would be defined in the node or in the narrative using existing data (or expert elicitation); for instance, a clearer statement would be “Is the prevalence of hazard X in the product more than 25 percent?” While still subjective, using a cut-off provides transparency and context for interpretation of results.

DELIBERATIVE PROCESS

Deliberative process, also known as reasoned opinion or expert sorting/synthesis, provides a relative risk ranking based on discussions among food safety experts, risk assessors, and/or risk managers. Through the deliberative process, experts rank risks by placing them into categorical bins (i.e. high, medium, low) and the output is often a narrative that captures the rationale used by experts to categorize the foods and/or hazards. Deliberative processes can result in the elicitation of individual assessments or in consensus agreements on values or rank order. Descriptive tables are a good mechanism to capture some of the parameters guiding the expert’s decisions, which may include disease fact sheets, summary risk profiles or, in the absence of data, may only be the expert’s background and scientific expertise. The selection of experts is critical as they must have a deep understanding of the hazards and/or foods being ranked. Depending on the availability of experts, the number of hazards and foods that can be ranked at a time may be limited. This approach is a subjective process, often not transparent, and may be difficult to replicate.

However, when decisions need to be made quickly and data are lacking, deliberative processes can provide a starting point to help inform the risk manager’s decision. An example of the deliberative process method is FAO’s 2008 Meeting Report on Microbiological Hazards in Fresh Leafy Vegetables and Herbs, where experts considered six criteria (i.e. frequency and severity of disease, size and scope of production, diversity and complexity of the production chain/industry, potential for amplification of foodborne pathogens through the food chain, potential for control, and extent of international trade and economic impact) to prioritize specific food commodities.[12] Experts reviewed the available information in light of these criteria, which enabled the identification of commodities to be ranked into three priority groups (level 1, 2, and 3) (FAO/WHO, 2008).

SEMI-QUANTITATIVE–NUMERICAL OUTCOMES WITHOUT A UNIT OF MEASUREMENT

Semi-quantitative risk ranking methods produce outcomes with numerical values without measurement units (i.e. ranking score, risk ratio). Such methods require moderate resources and data availability. Scores allow for items to be ranked, but do not provide an actual measure of risk or burden of illness, such as with quantitative methods. Thus, the outputs from these methods are considered to provide relative risk ranking outcomes.

Two common semi-quantitative risk ranking methods are presented below (i.e. Risk Matrix and MCDA); details on other risk ranking methods (e.g. risk ratio, scoring) have been documented elsewhere (van der Fels-Klerx et al., 2015).

RISK MATRIX

The Risk Matrix can be both a qualitative and a semi-quantitative method that considers a wide variety of data to rank risks. Risks are categorized into bins according to their relative severity and likelihood. For practical use, 5×5 risk matrices are often used and based on the assumption that risk equals severity multiplied by likelihood, as shown in Figure A4.

There is no widely recognized guidance on how to aggregate the different qualitative and semi-quantitative scores for likelihood and severity into different scoring bins, which are represented by individual cells of the matrix. It will be up to the analysts conducting the risk ranking in discussion with the risk manager to derive the meaning of those bins, which will also help with the interpretation of results and transparency of the process. It might be difficult and subjective to determine that, for instance, “low likelihood” times “high severity” is a “low risk.” Therefore, if a risk matrix is used, binning should be conducted as the last step, and binning categories should be clearly tied to risk outputs.

Larger risk matrices (e.g. 5x5) are often preferable to simplified risk matrices (e.g. 2×2) that may require more subjective binning, lack resolution, contain errors, and make risk decision-making more difficult due to an oversimplified approach.

It should be noted that it may be challenging to include diverse hazard types in the same risk matrix if the hazards are evaluated using different risk metrics (e.g. toxicity endpoints, exposure metrics, exposure pathways). For example, data availability and types between chemical and microbial hazards in food may differ, which could create challenges for including them both in the same risk matrix. Nonetheless, efforts should be made to use the same metrics, data types, endpoints, and other factors to make comparisons between hazard types more reliable.

Advantages–Semi-quantitative Risk Matrices

Risk matrices have several advantages. As with qualitative methods, they can be implemented relatively quickly and with limited availability of resources and data. Risk matrices are also easy to communicate and allow for different types of risk and risk levels to be considered in the same relative ranking scheme. Risk matrices can be especially useful for certain chemical hazards where exposure values can be multiplied by toxicity values to obtain meaningful outputs. In these cases, risk matrices are both transparent and understandable.

Disadvantages–Semi-quantitative Risk Matrices

Drawbacks to risk matrices include concerns that binning quantitative risk information decreases the resolution of the results and that they create a false sense of precision. For example, if bins are not clearly defined, it is difficult to interpret the results and there is a loss of information. Additionally, the use of risk matrices is more controversial for microbial hazards because the mathematical processes may not be valid in certain cases (for example, multiplying Escherichia coli count to deaths is not a linear function). As noted by Cox (2008), the qualitative risk matrix output matches the underlying quantitative risk ranking results only if certain principles are adhered to during the risk ranking process. Overall, the main limitations of risk matrices are: (1) poor resolution, (2) errors, (3) suboptimal resource allocation, (4) inability to account for uncertainty, and (5) ambiguous inputs and outputs based on the user and the type of risk matrix employed.

Applicability–Semi-quantitative Risk Matrices

Risk matrices are particularly applicable in instances where qualitative or quantitative data need to be further categorized to facilitate risk management, prioritization efforts, or risk communication efforts. Risk matrices can provide an effective visualization tool for communicating the results of quantitative risk ranking efforts. For example, for risk management purposes, it might be helpful to have a set of hazards and/or foods categorized broadly into risk groups using information on the magnitude and the actual risk to public health obtained from a quantitative risk assessment.

MULTI-CRITERIA DECISION ANALYSIS

Multi-criteria Decision Analysis (MCDA) is a collection of decision analytic techniques that has been used for the prioritization of foodborne hazards and/or food safety issues where multiple criteria (or factors), in addition to public health, need to be incorporated to inform decisions (FAO/WHO, 2012; Ruzante et al., 2010). MCDA offers a transparent and objective approach to accounting for all the criteria influencing the risk manager’s decision and for providing a structured approach to identifying decision alternatives. It is also able to incorporate a variety of data types, from qualitative to quantitative measures, as well as expert opinion and judgment. MCDA is not a semi-quantitative method but has been placed in this section because it is able to aggregate qualitative and quantitative variables into a single metric that allows the options (e.g. foods and/or hazards) being ranked to be ordered. Because of these capabilities, its conceptual approach can also be adapted for risk ranking. In the context of this guidance the “decision alternatives” are food and/or hazards to be ranked according to several public health metrics. More specifically, it is possible to integrate, simultaneously, several public health criteria that are relevant for the decision maker into a single output.

MCDA generally involves the following steps:

> Define the scope of the problem with risk manager inputs.

> Define a set of alternatives or options to be ranked (e.g. food, hazards, or food-hazard pairs).

> Define independent decision criteria to assess each alternative.

> Score each alternative against the decision criteria.

> Select criteria weights to allow for user judgement and preferences.

> Computation of results and identification of decision outcomes.

The outputs of MCDA can be used to select the “best” decision alternative. In the case of evaluating the public health risks of foodborne illnesses, MCDA does not produce a public health risk estimate, but rather a “measure of concern.” The public health criteria will largely be based on the two dimensions of risk, likelihood and severity, but often include other criteria that can be relevant to public health such as the “potential for a multistate outbreak.” MCDA is typically used to combine several criteria; however, the score produced is not directly related to the actual probability of illness. The score gives an ability to order the foods, hazards or their pairs in a way they can be ranked. Additive models, where scores are assigned and later added across alternatives to yield a total, are widely used but not overtly recognized as MCDA. Nonetheless, MCDA offers a structured framework for risks to be ranked when several factors (i.e. criteria) need to be considered – this is often referred to as “scoring methods,” however, in this guidance, we propose that one uses the MCDA methods and best practices when several public health factors need to be taken into consideration. One example of a simplified MCDA focusing only on public health criteria is the FDA’s produce risk ranking tool (Anderson et al., 2011) where an additive model was used to aggregate the different criteria impacting public health. In this example, the public health criteria include outbreaks (e.g. number, frequency), hospitalization rates, mortality rates, likelihood of contamination, growth potential of foodborne hazards, shelf life of food, cross-contamination, consumption, and economic impact (expressed as the cost of illness).

Advantages–MCDA

The main advantage of MCDA is that it offers a transparent and systematic approach to incorporate multiple public health factors that are a concern to the decision maker. It is typically less resource intensive than quantitative risk ranking methods. The flexibility inherent in MCDA approaches allows for different types of data to be used and the addition or removal of criteria. MCDA also has the ability to rank a large number of hazards and/or foods.

Disadvantages–MCDA

MCDA approaches can become complex due to the use of multiple criteria, data aggregation, scoring, weighting, scaling, and binning considerations. If possible, data should be normalized for different criteria and risk outputs to have objective meaning without distortion.

Applicability–MCDA

MCDA is a well-recognized and accepted way to simultaneously evaluate and weigh multiple public health criteria, hazards, foods, and data types. MCDA offers the ability to use quantitative data that may or may not be limited at this time to prioritize risk management activities. MCDA outcomes can be later incorporated into a quantitative risk ranking methodology.

Best Practices–MCDA

The scoring methodology for MCDA is dependent on the underlying process being estimated. In cases where different domains are being considered (e.g. there is not a single value to combine one criterion/metric), MCDA is appropriate in incorporating multiple different types of criteria (e.g. for chemical hazards, both acute and chronic non-cancer toxicity benchmark values could be aggregated and averaged for one non-cancer toxicity value per chemical to use as a proxy for severity criteria).

It is imperative to consider what level of precision, accuracy, and uncertainty is acceptable for the risk ranking exercise prior to deciding to use MCDA. Combining qualitative and quantitative data can introduce subjectivity from the risk assessor or experts conducting the risk ranking. To the degree possible, the evaluation should be mostly quantitative and rely on raw data rather than surrogate data. Binning should be done after the ranking is completed and, to ensure transparency, a database should be maintained to preserve the underlying data. Scoring and weighting algorithms should also be well documented.

SEMI-QUANTITATIVE RISK RANKING TOOL: RISK RANGER

Risk Ranger is a spreadsheet-based risk ranking tool, developed using Microsoft Excel. Users can select from qualitative statements or can provide quantitative data concerning 11 factors that affect the food safety risk of a specific population for selected product-hazard combinations. It is a bottom-up approach, evaluating risk from harvest to consumption. A total of 11 inputs are grouped into three general categories (susceptibility and severity, probability of exposure to food, and probability of food containing an infectious dose). The spreadsheet converts the qualitative inputs to numerical scores, and using three different multiplicative algorithms, provides a risk ranking score (scaled logarithmically from 0 to 100) that approximates probabilities of disease or death. Risk estimates include predicted annual illnesses or probability of illness per day in the target population. Risk Ranger is simple to use and publicly available as a free download. However, it only ranks microbial risks. Uncertainty is also not addressed, but users could run different scenarios to explore the different results. The tool was carefully developed and maintains the theoretical model of risk as defined by Codex, being an excellent choice if the goal is to focus on microbial hazards and the number of food categories is manageable (Sumner and Ross, 2002) Available here: http://www.foodsafetycentre.com.au/riskranger.php

QUANTITATIVE METHODS–NUMERICAL OUTCOMES WITH SPECIFIC UNITS

Quantitative risk ranking methods produce numerical estimates of the likelihood of foodborne illness and severity of outcomes with measurement units. Examples include disability adjusted life years (DALY), quality-adjusted life year (QALY), cost-of-illness (COI), and the number of illnesses, hospitalizations, and deaths (total and per serving of a certain food). Quantitative methods require the development of mathematical models that are either deterministic (outputs are single values[13] or point-estimates) or stochastic/probabilistic (outputs are characterized by probability distributions to represent the uncertainty and variability inherently associated with them). In stochastic models, calculations are made using computer simulations such as Monte Carlo (EFSA, 2012a).

Quantitative methods are robust, can provide estimates of risk, and the magnitude of the differences between each item being ranked can be more evident if probabilistic methods are used. However, quantitative methods are typically more complex and require greater technical expertise, resources, and data than qualitative or semi-quantitative methods. Therefore, quantitative methods are not applicable to very broad risk ranking questions, involving, for example, the evaluation of over 50 hazards and 50 foods at once. Quantitative methods are more efficient when the scope is narrower, and data are available. Expert opinion can be incorporated into quantitative risk ranking to fill specific data gaps; however, quality data must be the core of the model. Without quality data, the accuracy and validity of the risk ranking are compromised. Quantitative risk ranking methods can be based on epidemiological or quantitative risk assessments methods.

BURDEN OF DISEASE METHODS

Top-down approaches use epidemiological data, such as the number of illness reported to national health authorities and detected by surveillance systems, to estimate likelihood and severity.

The proportion of cases that is foodborne as well as the food vehicle that caused the illness (food source attribution) are critical data for this approach. Since reported cases are also just a small percentage of all illness, when using those quantitative epidemiological approaches, the rate of cases that are underreported and underdiagnosed will also need to be determined. Data from other countries and published literature might be helpful in certain cases but, given the differences in surveillance, culture, and health care systems, should be carefully evaluated to ensure representativeness. For example, if a country has a much higher incidence of a certain hazard than others (i.e. incidence of Campylobacter infections in New Zealand is 1.5 to 3 times higher than reported incidence rates in Australia, England and Wales, and several Scandinavian countries), or if consumption and handling practices for a certain food is unique to the country (i.e. consumption of raw cheese are more common in France than in the United States), data from other countries or regions might not be appropriate. WHO global estimates for the burden of foodborne disease for 31 major foodborne pathogens is a great example of an epidemiological approach to risk ranking that can inform risk ranking efforts in countries with limited resources and data. WHO is also working on publishing estimates for underreporting and underdiagnoses multipliers for the different sub-regions. Annex G presents the data available from WHO FERG and can be of great value to countries starting risk ranking efforts.

QUANTITATIVE RISK ASSESSMENT

Quantitative risk assessment (QRA) uses a bottom-up (“source to effect”) approach to rank risks. The likelihood and severity of a risk are estimated by modeling exposure (i.e. consumption data, prevalence, and contamination levels) and dose-response. Normally, the model mirrors the production chain from farm to fork and accounts for how the hazard changes (or not) through the production chain. While QRA provides robust estimates, it is labor intensive and requires a significant amount of data. Its applicably is also limited to a certain number of hazards and foods. The Listeria monocytogenes risk ranking in ready-to-eat foods conducted by the US FDA and USDA/FSIS (USDA and FDA, 2003), and the FDA risk assessment for methylmercury in fish (FDA, 2014) are two good examples of probabilistic risk assessment that targeted one type of hazard in a selected food. Two risk ranking tools that could be extremely helpful using this approach are presented below. These tools require users to have a certain understanding of probabilistic risk assessment; however, users do not need to be experts or have a specific software to conduct the simulations.

The first tool, iRISK, is a robust quantitative probabilistic risk ranking tool that integrates data and assumptions from seven components: the food, the hazard, the population of consumers, the process models describing the introduction and fate of the hazard up to the point of consumption, the consumption patterns, the dose-response curves, and the health effects (Chen et al., 2013). iRISK calculates several metrics: total DALYs or COI, DALY or COI per eating occasion or consumer, total illnesses, and total illnesses per eating occasion or consumer. iRISK is time consuming and data intensive, making it impractical for ranking a large number of hazards and foods. Therefore, iRISK is recommended for more targeted risk management questions. iRISK can be used to evaluate both microbial and chemical risks and is available online free of charge at irisk.foodrisk.org.

The second tool is the swift quantitative microbial risk assessment (sQMRA), which uses a simple, deterministic approach to compare the risk of pathogen-food product combinations (Evers and Chardon, 2010). sQMRA, which is implemented using Microsoft Excel, models pathogen growth and reduction from retail to consumption using data on 11 parameters and an intermediate model for contamination, cross-contamination and preparation. It consists of consecutive questions for values of each of the 11 parameters, always followed by intermediate model output broken down into categories of contamination, cross-contamination and preparation. Model outputs are compared with results from a full-scale QMRA of Campylobacter on chicken fillet, providing a relative risk measure (EFSA, 2012a). sQMRA’s narrow focus is a limitation, as users might need to look at other segments of the production chain, and chemical hazards are also outside the scope. sQMRA is also available online free of charge at http://foodrisk.org/exclusives/sqmra.

ANNEX G

WHO ESTIMATES OF THE GLOBAL BURDEN OF FOODBORNE DISEASES

The WHO FERG estimates are a valuable resource that can be utilized by countries with little data. Data from WHO (2015) are available for 31 hazards for the 14 subregions and for the following variables:

> foodborne illnesses, 2010

> foodborne deaths, 2010

> foodborne years lived with disability, 2010

> foodborne years of life lost, 2010

> foodborne disability-adjusted life years, 2010

> foodborne illnesses per 100 000, 2010

> foodborne deaths per 100 000, 2010

> foodborne years lived with disability per 100 000, 2010

> foodborne years of life lost per 100 000, 2010

> foodborne disability-adjusted life years per 100 000, 2010

> foodborne years lived with disability per case, 2010

> foodborne years of life lost per case, 2010

> foodborne disability-adjusted life years per case, 2010

> illnesses, 2010

> deaths, 2010

> years lived with disability, 2010

> years of life lost, 2010

> disability-adjusted life years, 2010

The median estimate as well as the 95 percent uncertainty interval can be downloaded into an Excel file through their online tool (Figure A5). It is important to note that FERG data are not country specific, so it is important to evaluate the appropriateness of using the regional or international estimates.

1 RTI Project No. 0215184; 23/01/2017.

2 Dr Kowalcyk was affiliated with RTI International when this document was developed.

3 Mr Batz was affiliated with University of Florida when this document was produced.

4 The Codex Alimentarius was established by FAO and the World Health Organization in 1963 to develop harmonized international food standards, which protect consumer health and promote fair practices in food trade. Codex standards ensure that food is safe and can be traded. The 188 Codex members have negotiated science-based recommendations in all areas related to food safety and quality. For more information, please see http://www.fao.org/fao-who-codexalimentarius/en/.

5 The Food Safety Modernization Act (FSMA) makes risk-based initiatives a core feature of the U.S. FDA’s activities and shifts the agency’s mode of operation from reactive to preventive.

6 http://www.cdc.gov/foodsafety/ifsac/projects/food-categorization-scheme.html.

7 http://www.fao.org/documents/card/en/c/ca5758en.

8 http://www.efsa.europa.eu/sites/default/files/scientific_output/files/main_documents/215e.pdf; https://www.efsa.europa.eu/en/efsajournal/pub/1970; and http://www.efsa.europa.eu/sites/default/files/scientific_output/files/main_documents/2489.pdf.

9 http://www.fao.org/3/a-i5399e.pdf.

10 https://www.ars.usda.gov/ARSUserFiles/80400530/pdf/1112/food_category_list.pdf.

11 http://www.fda.gov/downloads/Food/GuidanceRegulation/FSMA/UCM380212.pdf.

12 Note that since criteria other than public health were considered in this example, it is technically a prioritization effort rather than risk ranking. Still, it is a valid example, and the methodology could be applied to just public health criteria.

13 Average, median, highest level, ninety-fifth percentile, are examples of single values often used in determinist models.