This document presents an interoperability information framework for emergency management. This provides a reference model for information interoperability across the stakeholder functions in emergency management.
Discussion of this document is invited on the public mailing list firstname.lastname@example.org (public archives). Public comments should include "[EIIF-Framework]" as the subject prefix .
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of Final Incubator Group Reports is available. See also the W3C technical reports index at http://www.w3.org/TR/.
This document was developed by the W3C Emergency Information Interoperability Framework Incubator Group, part of the W3C Incubator Activity.
Publication of this document by W3C as part of the W3C Incubator Activity indicates no endorsement of its content by W3C, nor that W3C has, is, or will be allocating any resources to the issues addressed by it. Participation in Incubator Groups and publication of Incubator Group Reports at the W3C site are benefits of W3C Membership.
Incubator Groups have as a goal to produce work that can be implemented on a Royalty Free basis, as defined in the W3C Patent Policy. Participants in this Incubator Group have made no statements about whether they will offer licenses according to the licensing requirements of the W3C Patent Policy for portions of this Incubator Group Report that are subsequently incorporated in a W3C Recommendation.
The management of emergencies is an endeavour that is characterised by involvement from a multitude of stakeholders, including numerous government agencies, military groups, non-government and charitable organisations, private enterprise and community groups. Some jurisdictions have attempted to integrate government response under a single emergency response agency, but although this can help to manage the logistics of interagency communication, the problem remains, particularly for non-governmental participants. Of the four commonly identified phases of emergency management – prevention/mitigation, preparation, response and recovery – response poses the clearest immediate need for efficient communication between agencies. However, each phase offers opportunities for improved communications, and indeed, the languages used and the problems faced have significant commonalities across all phases.
The proliferation of participants poses challenges when trying to build information technology solutions to support the management of emergency operations. Without agreement on how stakeholders’ information technology solutions can intercommunicate, the use of IT threatens to complicate rather than simplify the processes. The general consensus in IT is that the co-operation of disparate systems is best addressed through the use of standards, agreed-upon interfaces and protocols of communication that, when adhered to, should guarantee successful interaction with other systems. Although encouraging stakeholders to use standardised structures can make and has made great strides in garnering agreement on the structures of information being exchanged, the values for data that are being exchanged, the vocabularies being used by the different agencies, present a much greater challenge. There are numerous reasons that different stakeholders use different vocabularies. Different spoken languages, different universes of discourse, different concerns, can each lead to differing terminologies that make it very difficult for stakeholders to exchange information efficiently.
This report on Emergency Information Interoperability Frameworks looks at the issues facing the wider emergency management community and outlines some potential paths forwards via a number of informal and formal information models, scenarios, use cases, and ontology directions.
Given the many stakeholders and contexts, there are numerous ways in which to view a framework for information interoperability in emergency management. The EIIF XG has not attempted to create an all encompassing single Framework. Rather, we looked at a number of different view points to showcase both the expansive impact and complexity of information interoperability for emergency management.
The Conceptual Mind Map (as shown in Figure 1) was a result of the EIIF XG Face-2-Face meeting in Washington DC (May 2008) in which the participants brain-stormed the various information entities that they deal with in their particular emergency management context. As a result, the Conceptual Mind Map contains many entities and relationships, and many of which overlap in their semantics and behaviour.
The Conceptual Mind Map shows 21 primary entities (each with many properties) with some explicit relationships between them. This is far from complete but shows the intricate inter-relationships that exist in emergency management information. The Mind Map shows common entities (such as People, Organisations, and Resources) as well as the more esoteric (such as Animals and Policy). All of these are important in different contexts to different stakeholders in emergency management.
We also are experiencing a new Web 2.0 world where mass user participation has resulted in the need for simpler shared vocabularies utilising tag clouds and Wordle, for example. The Web 2.0 user is becoming an integral part of the set of emergency management stakeholders with both their demand and supply of pertinent information during incidents. Figure 2 shows a Wordle output from the textual analysis from a large corpus of emergency management documentation (the Emergency Management Australia Manuals). These results can assist on determining common terms and phrases that will form part of a common shared ontology.
The Phased Framework takes a different more formal approach from the previous examples in that it views the four key phases of emergency management as separate (but related) activities and looks at how key entities (Organisations, People, Activities, Resources, Information) evolve through these phases (over time). The phases include:
Figure 3 below shows an example of the Phased Framework (as the figure is not complete in showing all the evolutions of the key entities).
The Phased Framework shows how the key entities evolve and undertake different roles at different stages. For example, the People entity has a number of different roles, such as:
Similarly for the other key entities, their roles and tasks will be dictated by the incident phases and direct requirements. For example, the Information entity needs include:
The Phased Framework is also useful when several overlapping emergencies occur (eg. a storm followed by a flood followed by a health epidemic). The phases allow you to calculate the changing demands and needs - allowing resources to be better staged and managed with appropriate planning support. Overall, how resilient a community is can be directly attributed on how well it approaches the four phases and how well it can deal with the consequences.
One of key questions in emergency response is Who is doing What, Where, and When. We use this as a framework in our brainstorm session to highlight some of data requirements and challenges in emergency response.
We need to keep track of all organizations and coordination centers involved in an incident. Information about the mission of the organizations - overall and in the context of an incident, their capabilities, and their current responsibilities can support situational awareness and time-boxing required for emergency operations. In this context, we identified the following organizational contact information:
Of note, although the focus here is on organizational entities, local efforts by persons affected are not often captured in existing data models.
Other issues include:
The activities that the organizations perform are mainly emergency support functions, transportation, and evacuation. The key point in such activities is the ability to assess the needs, and to address and integrate available resources versus capabilities. The activities can be characterized as following:
The type of disaster/incident will also determine (or give a good indication of) the range of Whats that should be provided. These activities would be applied to respond and recover from the incident.
Geospatial location is a fundamental property for understanding emergences in a coherent and intuitive way. In emergencies all parties can relate to where they are on a map, follow directions to or from a place, grasping the spatial context of their route etc. Handling the “where”may mean using information from a map or an image, data encoded as an address, zip code, or phone number, or landmark or events described in a text message, However, there is no single frame of reference to locate people, areas and/or resources. For example, some use identifiers (such as place names e.g. ISO 19112 - Spatial referencing by geographic identifiers), coordinate reference system (CRS e.g. ISO 19111 - Spatial referencing by coordinates), or jurisdiction for this purpose. Some humanitarian information centers use universal indicators such as p-code instead. Areas involve different types of geospatial concepts and may involve basic geometric constructs. The approach taken here is to leverage existing ISO and Open Geospatial Consortium (OGC) standards for spatial objects and relations in support of the Framework Concepts and harmonize these as needed.
We start with the ISO 19100 series of geographic information standards is a set of interlocking object-based standards consistent with principles and methodology of ISO/TC211 that can be used to define, describe, and manage geographic information about objects, events or phenomena that are located relative to the Earth. For location the most relevant are the ISO 19112 (Spatial referencing by geographic identifiers), and ISO 19111 – (Spatial referencing by coordinates) previously mentioned. But the nature of spatial characteristics and geographic features is also important since geographical objects can be defined in different ways by different people for different purposes. This geospatial objects can be referenced like location, by name, number or description. The ISO 19107 standard for Spatial schema has a conceptual schemas to describe the spatial characteristics of geographic features needed to support understanding and usage of geographic information. For example, the standard can support representing the geometry of a road or wall needed to communicate information about a supply or escape route and barriers to transport.
Also of interest is the ISO 19123 Geographic information schema for coverage geometry and functions coverages include digital orthoimages, gridded elevation data sets, and thematic classification maps such as soils maps.
The Geography Markup Language standard original GML of OGC model was based on the World Wide Web Consortium's Resource Description Framework (RDF). Subsequently, the OGC introduced XML schemas into GML's structure to help connect the various existing geographic databases, whose relational structure XML schemas more easily define. The resulting XML-schema-based GML retains many features of RDF, including the idea of child elements as properties of the parent object (RDFS) and the use of remote property references.
The OGC features model (see Figure 4) is a useful way to organize several aspects of geospatial information. More broadly this is a start on a geo-spatial continuum. This starts with spatial things like arrangements of parts of a natural object which we might see in an image, them there are geo-spatial mixtures such as building designs and more abstract, geographic rendering in maps and models.
As shown in the Figure 4, GML contains a rich set of related primitives which are used to build application specific schemas or application languages. These primitives include:
These issues include:
The incident can be described by the following properties:
The information comes from data inputting and external sources. Validity of such information is an important issue. Some assign a degree of confidence to the data and often use different frames of reference to assign probabilities. Veracity of the information is sometimes questionable as well. Interoperability is another concern. As mentioned before, there are different geographical frameworks to locate things. The other factors affecting interoperability are as following:
The previous scenario gave a broad overview of one facet of emergency operations. We now look at two Use Cases that the EIIF XG community felt lacked sufficient attention and posed significant interoperability issues facing current emergency management stakeholders. These use cases look at real-world operations systems and review and analyse their information models and semantics.
This use case information model represents the concepts and relationships that define an overall context for sharing of coordination information in an emergency. The model uses "Who (organizations or people) does What (activity) Where" scenario as a basis to derive high-level concepts and relationships and it is developed based on data schemas from two existing Emergency Information Systems [OCHA and Sahana]. We have reviewed their operational information models/standards and provided a harmonized view of the overarching W3 constructs shown in Figure 5.
Organizations represent any local/national/international, government/non-government organizations or local agencies and offices such as local churches. Organizations normally present a set of capabilities. In an emergency, they provide services based on their capabilities and current emergency needs, and in return they may need services from other organizations or people. They normally assign a person as a point of contact to each service. The organizations have general contact and location information as well.
Persons represents the individuals (Affliated_Person) who are affiliated with the organizations involved in an emergency, volunteers (Unaffiliated_Person), or the ones affected by the emergency (Affected_Person) who are the primary beneficiaries of emergency services. Person, similar to an organization, provides a set of capabilities or needs. An Affected_Person may need emergency services and an Unaffiliated_Person may be the one who volunteers to provide such services or to be available as a resource to other organizations which provide services to Affected_Person or Affected_Group.
Party is a general concept representing Organization and Person.
Contact_Details may represent general contact information for an organization or specific information for a contact person in that organization. It may represent contact information for any person involved in, or affected by, an emergency as well. Contact_Details can be published or private and each may have different access strategies for various locations of the object.
Unaffiliated_Person represents any person that is not related to an Organization.
Capability represents a set of activities that a person or an organization can potentially undertake (represented by Working_Sector) or the resources they can provide (represented by Resource). For example, organization X may potentially be able to provide medical services plus 3 ambulances and 5 shelters. Resources can be Equipment (vehicles, communication facilities, etc.), People (human force), Fund (any financial support), or Supplies. Resources can have Location_Information for tracing purpose in emergency operations.
Service is a set of activities that is carried out by an organization or a person. An example for a service could be a humanitarian project that is to improve education facilities in a given jurisdiction, or medical care that a volunteered physician provides to the victims of an earthquake. Services use the available capabilities to respond to an emergency. Location_Information represents the location where the service takes place.
Emergency represents the actual incident that is being coordinated. Emergencies can happen unexpectedly, such as an earthquake, or can be the result of significant vulnerabilities in the society such as HIV/AIDS. The services address the needs in different phases of an emergency. Location_Information represents the location where the emergency takes place. Related Emergencies should also be linked to enable shared resources and improved lessons-learnt outcomes
Location_Information represents the geographical location of the objects. There are different frames of reference for locating things: Address, Position, Place_Identifier, and Place_Indicator. Route represents a set of specific locations that the object is assigned to traverse, where the route is known in advance.
Address represents a physical location identified using typical address-like characteristics.
Position is normally used to identify the current location of moving objects.
Place_Indicator is a unique identifier which is assigned to a settlement based on a hierarchical division of settlements from national to village level.
Place_Identifier is the name used to identify places such as refugee camps.
Route represents a set of locations and estimated arrival and departure time to/from those locations.
The W3 and Missing Person Use Cases are two examples of two very prevalent coordination issues during a disaster. However there are many other use-cases that need to be explored in the emergency and disaster management domain for the sake of effective interoperability of systems.
The estimation of damages is critical to deciding the amount of resources and aid that needs to be requested and applied to a region. The assessments are made by field representatives of each relief agency (Gov, NGOs) and often with Government includes the registration of people. Effective assessments provide a balanced distribution of aid as it arrives. However each agency often does their own assessment and each system is isolated from each other. This means that there is redundant work being done and often victims are found entering multiple similar forms from different agencies before aid can be provided. This can also result in an imbalance of aid distribution.
Tracking families and displaced and making sure they are accounted for is also another requirement during disasters. Exchanging information of people displaced with relief agencies is needed to coordinate relief effort to rehabilitate families. Data captured on displaced people is slightly different from that of missing people, where in the latter all data on the individual appearance and identification marks need to be captured. Often with displaced families they are registered with the head of family or group with a break down of the composition of the group (e.g. babies, children, adults, elders) with the data needed for sustaining the family in the displaced location and information for rehabilitation.
Volunteers in their hundred thousands can descend into a disaster response effort in apply their labour, skills and goodwill to help those in need. By definition of a disaster and the lack of capacity of emergency response agencies, often there is no option but to accept even the untrained resources as part of the carder to respond to the relief effort. Tracking and checking especially spontaneous volunteers is a challenge and fine balance needs to be made. Some volunteers might not be utilized for by a particular organization, but needed for an activity in another. A quick mechanism of exchanging volunteer information including skills, availability, credentials and references would be valuable between agencies to minimize the volunteer registration effort.
Donors provide pledges and relief organizations make damage estimates and requests for aid quantities. However matching the two effectively is not a straightforward process due to the lack of integration between systems. Donations that were made to one org might not be utilized, but required by another relief grouping. This results in wastage and the ineffective distribution of aid. Standards are needed for the exchange of meta data around aid provided from the donor, with a tracking of it's final destination. The effective exchange of aid meta data can improve efficiencies in this regard.
The above Use Cases has focused on the use-cases in the disaster phase of Relief. There are other such use cases in the phases of disaster preparedness, recovery and rehabilitation that also need to be looked into.
In a very general dictionary sense, ontology refers to efforts to represent knowledge by categorizing and characterizing concepts, and the relationships between them. From a more practical sense, the term is used for the practice of setting down the concepts and relationships used in a domain, which is then used to allow reasoning over the objects in the domain based on these concepts and relationships.
We have shown in this report that the need to move towards a common ontology methodology is the major goal to meet in order to address the need for information interoperability in emergency management. Having stated this, it is also one of the hardest goals to meet as the consensus process across all the stakeholders will be a significant challenge.
However, having a single domain ontology shared by various applications may not be feasible in most cases. This is due to the fact that domain ontologies do rely on the particular task at hand and on the organization that develops them. This distributed nature of ontology development has led to a large number of ontologies covering the same or overlapping domains. Various organizations develop their own ontologies without fully understanding each other. Hence ontology heterogeneity becomes the first problem that needs to be solved while designing an ontology-based system. As such, ontology engineers face the problem of integrating different ontologies, either to support communication amongst the existing and new domains, or to enable interoperability across heterogeneous systems. Ontology mapping is the process of identifying the correspondences (mappings) between the concepts of two ontologies. It aims to solve the syntactic and semantic heterogeneity problem and can be done (semi-)automatically or manually with the help of ontology experts.
One of the key challenges in creating ontologies is where to begin the collection of the semantics. The US-based National Information Exchange Model has collected all the current XML-based standards and collated them to provide a comprehensive set of of semantics not only for emergency management, but also, immigration, infrastructure protection, intelligence, international trade, justice, and person screening. The disadvantage of this model is that it is simply a union of a large overlapping set of semantics with no overarching model or abstract framework to guide interoperability.
Conceptual models available today are generally built bottom-up by analyzing domain specific notions. The conceptualization process consists in introducing classes, properties, individuals, as well as axioms such as class inclusion, domain/range/cardinality restrictions on properties, and so on. From a formal standpoint, analysts use a variety of tools whose logical expressiveness ranges from simple taxonomic structures to rich description logics. UML (Class Diagrams) is commonly used as conceptual modeling language due to the wide acceptance within software developers, the high degree of standardization, and the availability of tools. In contrast with the high standardization of conceptual modeling in its formal aspects, there are not widely accepted ontological standards to be readily used as a basis for analyzing common, cross domain notions. Also, there’s a general tendency at capturing commonsense concepts as they appear in natural languages and to represent them with minimal analytic effort. Since natural language semantics is fuzzy, and varies among cultures, this situation leads to a great amount of heterogeneity in domain specific ontologies when they are developed by independent and culturally different organizations.
To reduce such heterogeneity, thus relieving the effort of integrating information systems, a viable practice could be that of adopting foundational ontologies, i.e. conceptual models of common, cross-domain notions such as spatial-temporal ones. However, at the time being, this practice is not very diffuse, and one of the reasons is the lack of a well assessed reference foundational ontologies. In fact, the process of standardizing a set of basic and commonly accepted ontological distinctions is very far from trivial. Nevertheless, a number of proposals are available today that can be used to guide the development of domain models basing on ontological foundations rather than linguistic intuitions. Moreover, existing domain models can be revised and enhanced by aligning their concepts to ontological categories coming form some selected foundational layer, with the aim of clarifying ontological commitments and better structuring hierarchies and relationships.
What follows shows the results of aligning the current draft version of the W3 Coordination model (as shown in Figure 5) with respect to DOLCE ontology . DOLCE (Descriptive Ontology for Linguistic and Cognitive Engineering) is based on a fundamental distinction between events (called perdurants) which have temporal parts, objects (called endurants) which have spatial parts, and abstract things without spatial-temporal qualities. These distinctions are inspired to philosophical literature and are generally accepted within ontology standardization initiatives. Also, DOLCE introduces qualities as concepts that inhere to entities in that they exist as long as their host entities exist, and regions as spatial, temporal, and conceptual dimensions. For the sake of simplicity, we’ve adopted a reduced version of the DOLCE core called DOLCE-Lite (See Figure 6), but we’ve included some notion of an additional module called DnS (Descriptions and Situations ).
In particular, concepts related to social processes, such as “Social Role”, are very important for modeling the domain of emergency management, but many aspects of these notions are currently under investigation by the scientific community.
What follows is a summary of what resulted by framing W3 concepts under DOLCE conceptualization. The results (see Figure 7) show that most of the concepts where smoothly positioned under DOLCE categories, but some concepts are not clearly placed and requires further analysis.
Service, in a concrete sense, can be seen as a Process, i.e. a perdurant (event) whose temporal parts may have different qualities (e.g. agreement, delivery, and conclusion). By looking at the attributes of the W3 class, however, it seems that the concept aims at modeling abstract and informative qualities such as Title and Description. To represent both informative properties and spatial-temporal ones under DOLCE’s conceptualization, Service might be split in two different classes: “ServiceDescription” (InformationObject) and “ServiceProcess” representing the concrete processes of service’s execution.
Capability is used in W3 for representing the kind of actions Persons and Organization should be able to perform. This should be represented in DOLCE by an AbstractQuality (qualities inherent in non-physical endurants) whose value should range over a suitable abstract region, to be introduced. According to DOLCE, however, this would limit the ascription of (instances of) this class to non-physical endurants.
Organizations can be collocated under the DOLCE’s notion of Collective (collections with only agents as members).
In terms of DOLCE, Emergency can be seen as a stative perdurant. The presence of the attribute “phase” confirms that the concept is intended to capture a temporal notion. However, like in the case of Service, other attributes (e.g. “type”) seem to be related to the description of classes of emergencies situations.
Person can be straightforwardly collocated under DOLCE’s AgentivePhysicalEntity.
Classes related to encoding and exchanging information can be famed under DOLCE InformationObject, socially constructed objects which play the role of “expressions” in communication processes.
It is not immediately clear what Resource could be in terms of DOLCE categories. The class looks like the union of three other classes Equipment, People, and Fund. Intuitively, Resource stands for any concrete thing that can be instrumental to the process of delivering a Service. It is questionable, however, whether a specific class is really needed here.
The lack of shared vocabulary is acknowledged as one of the causes for knowledge disconnectedness on the web, and is a common, major problem in all sectors. Using different terms, definitions and concepts is one of the central causes of lack of semantic integration and divergence, therefore one of major obstacles to leveraging synergy and allowing collective intelligence to be catalyzed. Lexical and semantic distance may arise from differences among:
As an example, one of the documented terms in emergency information systems is victim. Victim is a widespread English language term in use in emergency management operations worldwide. There is also evidence some people affected by adverse events resent being called 'victim', as this conveys an image of passivity, helplessness and impotence. While many would agree that people impacted by adversity and in need of emergency aid have higher priorities than disputing preferred naming conventions, it could be argued that ‘victim’ in itself does is not necessarily a meaningful word, and that terminological correctness and sensitivity is desirable where possible. Therefore the term victim can be semantically mapped to a preferred context neutral term, such as ‘Affected Person’, which is the current naming convention for this entity in W3 Information Model in Section 4.1.
Therefore, a Semantic cluster identifier for "person" may include:
Similar processes apply for many terms used conventionally by the emergency management sector, for example, ‘disaster’. During open community discussion, it emerged that the term ‘disaster’ is not necessarily representative of the range of adverse events that constitute an emergency, and it may have some undesired connotations. Therefore, a Semantic cluster identifier for 'event' may include:
For global interoperability we need some harmonized vocabularies and glossaries. One example is the Institute for Crisis, Disaster, and Risk Management Emergency Management Glossary of Terms [ICDRM, 2009]. We can see part of the challenge by looking at the ICDRM terms as there is no entry for "victim". Instead we have some related terms such as "actor" which mentions the victim role in a simulation context. It's defined as:
On the other hand the ICDRM Glossary has any number of informational items that we might add such as Alert, which discusses related terms (e.g. “advisory”) and a supertype "notification" :
Such terms could be used to map to (or harmonize) the "warning" term used in the Phased Framework Model (see section 2.3). This will lead to greater interoperability in the longer term, with the proviso that the semantics of the ICDRM terms are consistent with the intended semantics of the Phased Framework Model.
Looking at the USA National Incident Management System glossary [FEMA], they also have a specific definition for Emergency:
Clearly, this definition will not work outside the USA.
However, other useful terms include:
In a broad sense, it is envisaged that there is a top level universal interoperability requirement, which is to support the communication and mutual reinforcement of all potential agents able to provide capability to deliver aid and act as first responders during an emergency to interoperate. Responders need the ability to easily and fluidly share information, voice data and video. That is not possible with most deployed systems [ShipCom, 2008].
In such unconstrained scenario responders may be a mixed bag of organisations and individuals whose operations are regulated by different protocols, who communicate in different languages and following different rules, each providing a contribution to overall emergency relief capability. The above is likely to be a chaotic 'open world' scenario, where resources and decision making are distributed, and coordination is the key strategic requirement. More detailed interoperability requirements can be more tightly defined by specifying scenarios, use cases and circumstances.
Loosely speaking, it can be said that the interoperability gap is generated by a disparity (difference) between interoperability requirements, interoperability standards and interoperability implementation. An interoperability gap can be envisaged as a factor resulting from the combinatorial disparity between the different logical representation layers: syntactic, semantic and pragmatic [Carlile, 2004] as shown in Figure 8.
The syntactic gap in communication is caused by different language schema notation, where the schemas are not compatible. This gap is generally bridged by 'mapping' elements of a schema to another syntactic representation.
The semantic gap characterizes the difference between two descriptions of an object by different linguistic representations, for instance languages or symbols. In computer science, the concept is relevant whenever ordinary human activities, observations, and tasks are transferred into a computational representation.
The pragmatic gap results from the difference in organisational and social context of the communication layer, which contributes to different operational and information models, and therefore can be viewed as the result of the combinatorial explosion of the 'context' to knowledge on the web. Pragmatically challenges to knowledge reuse, and relevant contextual dependencies, are considered not merely technical, but belong to the realm of social and organisational systems design and management, and extend well into the boundaries of what is designated as 'policy' management.
Each aspect of the interoperability gap should be tackled with a targeted approach, leading to the development of an 'integrated' approach. For example, Figure 8, the syntactic, semantic and pragmatic gaps are identified 'at the boundary' level (where they intersect), which is where change/transformation/innovation tends to take place, and corresponding solutions suggested.
A strategy for filling the pragmatic gap should focus on:
Specific detailed instruments should be developed in more detail and are likely to include:
Decision making during emergencies is characterized as mission-critical and time-critical. When a catastrophe occurs, no single organization has all the necessary resources to alleviate the damage. Collaborative efforts between various agencies is required in sharing information. In this report we have looked at various Frameworks and Use Cases that showcase this issue. The future effort towards sharable semantics for ontologies will provide significant enhancements to the current emergency management information interoperability challenges.
The editor would like to thank all the members of the W3C EIIF Incubator Group for their valuable input into this document with special thanks to Rebecca Curzon and Craig Hubley for detailed reviews and comments.