Analysis of Business Contract Scenario

Detailed analysis of the Business Contract Scenario, which is scenario 3 of our group's requirements report.

Goals/Scope

This is where we identify anchor/entry points that people outside the provenance group can use to understand how to map their provenance needs into general aspects of the scenarios.

• Checking whether past actions comply with stated obligations
• Understanding how one product is derived from another
• Filtering the information revealed in provenance by privacy and ownership concerns
• Discovering where two sources have a hidden mutual dependency
• Resolving apparent inconsistencies in multiple accounts of the same event
• Verifying that those who performed actions had the expertise or authority to do so

The above general functions are realised as the specific user requirements listed below.

User Requirements Specific to Business Contract Scenario

Below we enumerate specific (provenance-related) user requirements from the scenario as described here, and give reference to the relevant general user requirements listed in the requirements list.

• Content
  • Record adequate information to prove contract compliance (instance of C-JUST-UR2)
  • Store information to prove contract compliance for several years (instance of C-JUST-UR3)
  • Document multiple versions of a product, such that the version is identifiable in the provenance (instance of C-Vers-UR 1b)
  • Show, where something is asserted to be occurring at the same time as something else, that this was an inference on the basis of two timestamps with same the semantics and source clock (instance of C-Entail-UR 5)
  • Ensure that design reviews can be traced to the experts who produce them, to verify that they were independent (instance of C-Attr-UR 2)
  • Determine the involvement of untrained personnel on security-related parts of the website (instance of C-Proc-UR 1)
  • Check the experience of a developer by searching for the number of websites they have been involved in developing (instance of C-Proc-UR 4)
  • Determine and record when a design was changed from one version to another (instance of C-Vers-UR 2b)
  • Determine and record which developer was responsible for creating a new version of a previously designed website (instance of C-Vers-UR 3b)
  • View and extract some or all of the provenance records regarding a particular version of a website design (instance of C-Vers-UR 5b)
  • Link a completed website design the prior designs, specifications and intermediate version used to produce it (instance of C-JUST-UR1)
  • Verify that the less detailed record (removing IP-sensitive information, for example) is consistent with the full record with regards to the claims being made against the manufacturer (instance of C-Entail-UR 1)
  • Provide awareness of who removed information from the detailed provenance trace (instance of C-Entail-UR 3)
  • Given a claim about the inconsistency of timing information regarding quality checks, identify the claimant (instance of C-Entail-UR 6)
  • Given a claim about the inconsistency of timing information regarding quality checks, identify the timestamps used to derive that conclusion and the assumptions made in doing so, i.e. about the semantics of the timestamps (instance of C-Entail-UR 9)

• Management
  • Query for the provenance of a given product (instance of M-Acc-UR 1)
  • Determine whether a provenance trace contains information extraneous to proving compliance with a contract (instance of M-Diss-UR 1)
  • Determine whether provenance information reveals information about another customer (instance of M-Diss-UR 5)
  • Store many kinds of data as part of a provenance record, such as images, source code etc. (instance of M-Scale-UR 1)
  • Provide a representation format for provenance information which is capable of conveying the proof of compliance with a contract (instance of M-Pub-UR3)
  • Query across multiple department records to provide a combined record of every stage of development for a website (instance of M-Acc-UR 1)
  • Query the records of independent experts to find corroborating information of their activity in checking quality (instance of M-Acc-UR 3)
  • Provide a way for a provenance record to retain the ability to prove contract compliance even where information is anonymised to protect other clients (instance of M-Acc-UR 4)

• User
  • Prove that a contract was executed correctly for a given product (instance of U-Acct-UR 1)
  • Convey signed assurance that a derived, simplified provenance trail is consistent with a signed, detailed provenance trail (instance of U-Acct-UR 1)
  • Allow users to generate a summarised version of the provenance record required to succinctly explain the proof of contract compliance (instance of U-Imper-UR 2)
  • Demonstrate that a design was approved in independent checks by multiples experts (instance of U-Debug-UR 1)

Technical Requirements for Business Contract Scenario

Here we list the general technical requirements (a subset of those in the list gathered here) which are required to fulfil the user requirements listed above specifically for the Business Contract Scenario.

• Content
  • C-Attr-TR 2.1: There should be a notion of the creator of some data
    • e.g. it should be possible to express that Bob's Website Factory is the creator of each website produced
  • C-Attr-TR 2.2: There should be a schema which will lead users to fulfill requirements for correct management of provenance metadata
    • e.g. it should be possible to express at what point and in what way each developer was involved in the creation of the website
  • C-Attr-TR 2.3: A system should track which portions of a document were produced by an entity
    • e.g. it should be possible to express which components of a website design were developed by which developers
  • C-Proc-TR1.1: The provenance metamodel should be expressive enough to support reasoning about the process which occurred
    • e.g. it should be possible to use the provenance of a design to reason about whether the contract was complied with over the whole course of development
  • C-Proc-TR4.1: It should be possible to compare two or more processes documented in the provenance
    • e.g. it should be possible to determine that the checks performed by the two independent experts were not based on a common source
  • C-JUST-UR1.1: A justification derived from the provenance should distinguish between source and derived data, and show how intermediate results were obtained
    • e.g. the proof of contractual compliance should distinguish between the original customer specification and designs derived from it
  • C-JUST-UR1.2: There should be allowance for links to both public and confidential materials stored both online and offline
    • e.g. the proof of contractual compliance will refer to the specification of Customers Inc. and also other customers whose privacy should be maintained
  • C-JUST-TR3.1: The data and supporting information should be archived securely in a stable long-term preservation format
    • e.g. the records used to create the proof of compliance need to be maintained for a long period to satisfy legal requirements
  • C-JUST-TR3.2: Changes to the source data or justification over time need to be recorded to keep the records consistent
    • e.g. wherever an event occurs which could affect the proof of contractual compliance, e.g. a quality check, this should be recorded such that it becomes part of the proof
  • C-Vers-TR 1.1: A system should identify and record changes that are made to a document or data
    • e.g. all changes made to a website under design should be expressible
  • C-Vers-TR 1.2: A system should define what constitutes a different version of a document or data
    • e.g. each version of the website, and that version's characteristics, should be expressed in the provenance
  • C-Vers-TR 2.1: A system should identify and record when content was changed
    • e.g. all edits made within a webpage (a component of the site) should be expressible
  • C-Vers-TR 3.1: A system should identify and record who and/or what changed the version, document or data
    • e.g. the developer who edited a webpage, and who marked its generation into a new version, should be expressible
  • C-Vers-TR 5.1: A system should allow anonymised records (or parts of those records) to be viewed or extracted as required
    • e.g. where the proof of contractual compliance concerns another customer's private information, this should be anonymised in the proof and, where possible, in the provenance record
  • C-Vers-TR 5.2: A system should be capable of anonymizing the personal data held in those records before viewing or extraction
    • e.g. where the proof of contractual compliance is obtained from the records, customer's identifying data should be anonymised
  • C-Entail-TR1.1 The provenance should be able to express the same events at multiple levels of granularity
    • e.g. the provenance should express, at a coarse granularity, that a website design was derived from a previous design and, at a fine granularity, the steps by which one design was adapted to form the other

• Management
  • M-Acc-TR 1.1: Applications should support specialized provenance query infrastructure (e.g. query operators)
    • e.g. it should be possible to query for the provenance of a website design
  • M-Acc-TR 1.4: Applications should provide efficient access mechanism over large provenance datasets and be scalable
    • e.g. where there is a record of a large number of website design processes performed by Bob's Website Factory, it should be possible to extract only that relevant to the proof of contract compliance for a single instance in a timely manner
  • M-Diss-TR 1.3: Provide mechanisms to examine data's provenance to check for correct usage according to pre-stated purpose.
    • e.g. it should be possible to verify that a summarised report of the design was not used for quality control checks, as it is only the full design which should be used for this purpose
  • M-Diss-TR 5.1: Represent the provenance of a resource such that it is apparent from what it is not derived, as well as from what it is derived
    • e.g. it should be possible to determine that a website design was not derived from one produced for another customer, but created from scratch
  • M-Scale-TR 1.1: Tools to allow provenance tracking at fine level of granularity that scales with increasing size of data.
    • e.g. where there is a record of a large number of website design processes performed by Bob's Website Factory, it should be possible to extract the full details of all edits on a single website

• Use
  • U-Acct-TR 1.1: Contracts should have a machine-understandable representation in terms of data to be used, methods to be performed, time constraints, constraints on agent, and other obligations
    • e.g. to extract a proof of contractual compliance from the provenance, we should have a version of the contract expressed as a query for relevant information from the provenance
  • U-Acct-TR 1.2: Provenance should be expressed in terms of workflow such that it can be matched against the contract
    • e.g. it should be possible to extract the provenance of a website design as a process starting from the specification to which the contract applied
  • U-Acct-TR 1.4: Should be possible to suppress certain details in provenance such as certain processes or identity of samples
    • e.g. it should be possible to remove details of other customer's in presenting the proof of a website design's contractual compliance drawn from the provenance
  • U-Acct-TR 1.5: Should be possible to use (signed) statements from trusted third parties in place of portions (either secret or lengthy) of provenance
    • e.g. where an independent auditor, Audit.com, looks at BWF's full provenance record and makes a signed statement that the original signed provenance of the two independent experts' reports do not share a common source design summary report (and the experts' signatures were intact and valid when Audit.com looked).
  • U-Acct-TR 1.6: Requires mechanisms for establishing trustworthiness between parties and verifying the integrity of signed statements
    • e.g. it should be possible to verify that the record used to prove contractual compliance has not been tampered with
  • U-Debug-TR 1.1: Representing the full record of how each actor in a chain of sources depended on others
    • e.g. it should be possible to determine whether independent experts based their assessments on information produced by a non-independent, and possibly suspect, source within BWF
  • U-Debug-TR 1.2: Provide a mechanism to analyse a record of how some information was produced to detect where there is reliance on single sources for information critical to producing that information
    • e.g. it should be possible to determine whether the two independent experts actually based their assessments on a single source produced by BWF

Diagrams

The diagrams below illustrate the scenario requirements from different viewpoints.

Contract 'content' block diagram (Customers Inc. view)

From the high-level perspective as seen by client being provided a website, Customers Inc., the content of the provenance consists of a record of BWF receiving a specification for the website (1) and an agreed contract for how it would be developed (2), and producing (3) a website supposedly fulfilling the specification (4) in accordance with the contract.

Contract 'content' block diagram (BWF view)

The development process (3) can be expanded into more detail when seen from BWF's perspective. Here they record how one or more existing designs (5), developed for previous customers, are transformed into a design meeting Customers Inc.'s specification. The design will go through multiple distinct versions (6, 7). Then, before being implemented, the design will be verified to be of adequate quality by two independent experts (8). On these checks being passed, the design is implemented (9).

Contract 'management' block diagram

The management concerns are as follows. First, BWF has an issue that the designs created for previous customers (10) are confidential and should not be released to Customers Inc., even if their own website is based on those earlier designs. Second, Customer Inc. is concerned that the provenance records regarding the website's production (11) may have been altered by BWF before release, so as to hide their malpractice (12). Third, there is a dispute over the semantics of the information in the records, specifically where BWF and Customers Inc. disagree over the timestamps used on recording the independent experts' checks. BWF asserts that the timestamps are compatible with the checks being carried out before implementation (13), while Customers Inc. claim they are not, and so the implementation must have gone ahead regardless of the quality checks (14).

Contract 'use' block diagram

The primary use of provenance in this scenario is for each side in the dispute to generate legal proofs backing up their claims (15, 16). A particular disagreement which the provenance may resolve is whether the experts were truly independent in their assessment of the design (17), or if they actually both relied on the same potentially misleading summary of the design produced by BWF (18).

Relevant State of the Art

Here we describe the research conducted in relation to, and the technology available to fulfil, the above described requirements with specific regard to the Business Contract Scenario.

Here is a summary of related work brought up in the original use cases.

Existing Solutions Used Today for Business Contract Provenance

A summary of research, with referenced papers, which consider issues similar to the scenario-specific user requirements above, or are applied in a similar domain.

The Business Contract scenario envisages mechanisms to explain the details of procedures, in this case instances of design and production, such that it can be compared with the normative statements made in contracts, specifications or other legal documents. It further assumes the ability to filter this evidence on the basis of confidentiality. While we are not aware of any system providing exactly the functionality described, there are many which address particular aspects.

Tracking Design

It is critical to track which products a design decision or production action ultimately affect, so that it is possible to both show that what was done in producing an individual product fulfilled obligations and determine which set of products a decision affected. As an example of the need for the latter, Toyota recently had to recall millions of vehicles due to a particular manufacturing issue [BBC10Toyota]. Without knowing the connection between the manufacturing actions and products, many vehicles unaffected by the problem may be recalled, so costing a company a great deal more than necessary.

A key feature of the VisTrails workflow system [Scheidegger08Vistrails] is its ability to capture not only the provenance of data produced by executing workflows, but changes to the design of the worklow itself. The users can then employ this information to replay all or part of their workflow, revert decisions, etc.

Computer-Aided Design

Computer-aided design (CAD) systems can include features to capture what is occurring as a design is created. Aside from storing a history of changes to a design, some systems allow the rationale behind design choices to be captured [Bracewell08Rationale], which may be an essential part of the record in explaining how contractual obligations were aimed to be met (particularly where a contract states that some factor must be 'taken account of').

The provenance of a design goes beyond just the changes made through a single CAD system, both because one design may be based on another previously developed for another customer and because the design is just a part of a larger manufacturing and retail process. Ways in which the interconnection between parts of a process is eased include the use of common formats for describing designs and standardised archiving mechanisms used by all stages of a process [Patel08CAD].

Electronic Notebooks

Using the web for business processes allows multiple remotely deployed and regularly updated tools and services to be integrated through a single portal. One beneficial effect of this is that the different activities involved in a complex design process can be tracked together, so providing a single coherent record of how something was produced. These systems are sometimes called electronic notebooks [Reimer04Notebook].

Current Provenance Research Relevant to the Scenario

Here, we aim to discuss how the technical requirements drawn from the Business Contract Scenario are addressed by existing technology and research, focusing on surveys and papers that introduce key ideas ro conclude lines of research, not comprehensive coverage of all related work. We proceed by discussing each dimension in turn.

Content

Justification

In the scenario, BWF has many customers over time (and re-uses earlier designs in satisfying later customers). In order to ensure that adequate proof of contract fulfilment can be provided if needed, BWF and a customer may agree up front as to the form this, i.e. the provenance data, would take. It is infeasible for such a format to be reinvented for each customer, so it is preferable to use a commonly accepted suitable provenance model. Attempts at shared exchange language for provenance (of the form required in the scenario) are the Open Provenance Model [Moreau10OPM] and Proof Markup Language [Pinheiro06Proof].

Management

Dissemination

Nagappan and Vouk [Nagappan08Privacy] directly consider the issues of preserving an audit trail, suitable for later providing as evidence, and access control to ensure confidential provenance information is not exposed. This approach is applied within the context of a database accessed via a web query interface. More recently, Davidson et al. [Davidson07Workflow] have considered how to characterise and measure the privacy of information revealed within provenance data (in the form of an executed workflow).

Publication

A noticeable point in the scenario is that BWF do not know in advance that the request to prove contract compliance will be made, or what particular part of the design process will be queried, e.g. the competence of a particular engineer. In general, it is impossible to know exactly what information about current events, and to what detail, will be required by those later querying the provenance. On the other hand, it is also impossible to keep records of everything which occurs in full detail. This problem was noted by Bose and Frew [Bose05Survey], and attempts have been made to give engineers guidance in tackling it by Chapman and Jagadish Chapman07Issues and Miles et al. [Miles10TOSEM].

Use

Debugging

Analogous to the scenario, Miles et al. [Miles07Validation] conducted research into checking compliance of a past procedure to given constraints, where the procedure was a bioinformatics experiment. Ontology-based reasoning was used to allow the constraints (which concerned debugging and checking use of licensed material) to be expressed at a coarser granularity to the description of the process in the provenance.

The question "Were two experts' assessments truly independent, or based on the same ultimate source?" is one tackled, in general form, by other existing work. As the sources from which an assessment are derived may not all be apparent from the assessment, especially where information is second- or third-hand, the provenance of the assessments can help determined whether there is commonality. Analogous research has been conducted by Townend et al. [Townend05Fault] in the area of debugging systems which provide multiple independent checks for fault tolerance in service-based applications.

Imperfections

A question is raised in the scenario about the quality of the information given, and a dispute occurs over the semantics of time information. There exists work by Hartig and Zhou [Hartig09Quality] on using provenance of web data to assess its quality, in particular its timeliness. Given the reliance on correct engineering of a secure website for Customers Inc., deciding contract compliance in their or BWF's favour may be a question of the risk caused by BWF's engineering approach. Gandhi and Lee [Ghandi08Regulatory] propose a methodology for providing a chain of evidence to produce insights for risk assessment.

Comparison

Also using provenance to determine contract compliance, Groth et al. [Groth09Trust] used OPM along with an electronic contract representation to determine whether a new contract proposal should be accepted based on it similarity with past contracts and the success of those contracts (as determined from the provenance).

Understandability

With regards to design of software, such as BWF's websites, the provenance of a manufactured product is closely related to the traceability of the design. Jahnke et al. [Jahnke02History] proposed automatically capturing the design process by recording the changes in the models, so allowing a designer to relate a later design to an earlier one from which it was derived. They extended this idea to view re-design as a workflow-like process [Jahnke02Iterations], so tying traceability even more closely to many approaches to provenance.

Gap Analysis

Given the analysis of the state of the art above, what are the gaps in technology which need to be addressed to achieve what the scenario proposes?

Overall, there is a gap in practice: provenance technologies are simply not being used for the described purpose. Even with encouragement, the existing state of the art does not make it a simple task to achieve the scenario, because of a lack of standards, guidelines and tools. Specifically, we can consider the gaps with regards to content, management and use, as modelled above.

First, there is no established way to express what needs to be expressed regarding the provenance of an engineered product, in such a way that this can be used by both the developer and the customer (and any independent third party). Each needs to first identify and retrieve the provenance of some product, then determine from the provenance where one item is a later version of another, where implementation activity was triggered by a successful quality check etc. Moreover, even if a commonly interpretable form for the latter provenance information was agreed, there is not an established way to then augment the data so that we can verify the quality of the process it describes, e.g. how to make it comparable against contractual obligations, or to ensure it cannot be tampered with prior to providing such proof. While provenance models, digital signatures, deontic formalisms, versioning schemes and so on provide an adequate basis for solutions, there is a substantial gap in establishing how these should be applied in practice. Without common expectations on how provenance is represented, a producer cannot realistically document its activities in a way which all of its customers can use.

Assuming that common representations are established, there is a gap in what the technology provides for storing and accessing that information. This goes beyond simply using existing databases due to the form of information conveyed in provenance. For example, if an independent expert documents, in some store local to themselves, that they checked a website design, this must be connected to rest of the website's manufacturing process to provide a full account. While web technologies could be harnessed for the purpose, there is no established approach to interlinking across parts of provenance data. Moreover, the particular characteristics of provenance data may affect how other data storage requirements must be completed: how to scale up to the quantities of interlinked information we can expect with automatic provenance recording, how to limit access to only non-confidential information etc. Finally, the provenance, once recorded, has to be accessible and there are no existing standards for exposing provenance data on the web.

With regards to use of provenance, the scenario requires various sophisticated functions, and it is non-obvious how existing technologies would realise them. For example, the various parties need to be able to understand the engineering processes at different levels of granularity, to resolve apparent conflicts in what appears to be expressed in the provenance data, to acquire indications and evidence of which parts of the record can be relied on, determine whether the provenance shows that the engineering process conformed to a contract, check whether two supposedly independent assessments rely on the same, possibly faulty, source, and so on. To encourage mass adoption of provenance technologies, it must be apparent how to achieve these kinds of function, through guidelines and standardised approaches.

In summary, although there are many proposed approaches and technology solutions that are relevant, there are several major technology gaps to realizing the Business Contract scenario. Organized by our major provenance dimensions, they are:

• With respect to content:
  • No mechanism to refer to the identity/derivation of an information object as it is published and reused on the Web
  • No common standard for exposing and expressing provenance information that captures processes as well as the other content dimensions
  • No standard techniques for versioning and expressing provenance relationships linking versions of data
  • No standard formats which can characterise whether provenance is of adequate quality for proof, e.g. signatures
  • No pre-specified way to ensure provenance content can be compared against electronic contracts
• With respect to management:
  • No well-defined standard for linking provenance between sites
  • No guidance for how existing standards can be put together to provide provenance (e.g. linking to identity)
  • No guidance for how application developers should go about exposing provenance in their web systems
  • No proven approaches to manage the scale of the provenance records to be recorded and processed
  • No standard mechanisms to find and access provenance information for each item that needs to be checked
  • No well-defined means of ensuring only essential non-confidential provenance is released when querying
• With respect to use:
  • No clear understanding of how to relate provenance at different levels of abstraction, or automatically extract high-level summaries of provenance from detailed records.
  • No general solutions to understand provenance published by another party
  • No standard representations that support integration of provenance across different sources
  • No standard representations that support comparison between provenance and business process specifications
  • No broadly applicable approaches for dealing with imperfections in provenance
  • No broadly applicable methodology for making trust judgments based on provenance when there is varying information quality
  • No standard methods for validating whether provenance is of adequate quality for proof
  • No mechanism to query whether provenance data shows that laws, regulations or contracts have been complied with
  • No standard mechanism to assess whether two supposedly independent assessments rely on the same, possibly faulty, source
  • No means to resolve conflicts in (possibly inferred) provenance data

References

[BBC10Toyota] BBC. Toyota recalls 2.3m US vehicles. http://news.bbc.co.uk/1/hi/8473789.stm

[Bose05Survey] Bose, R. and Frew, J. 2005. Lineage retrieval for scientific data processing: A survey. ACM Computing Surveys 37, 1 (Mar.), 1–28.

[Bracewell08Rationale] Rob Bracewell, Ken Wallace, Michael Moss & David Knott (2008). Capturing Design Rationale. Computer-Aided Design 41. 173–186. DOI: 10.1016/j.cad.2008.10.005

[Chapman07Issues] Chapman, A. and Jagadish, H. V. 2007. Issues in building practical provenance systems. Bulletin of the Technical Committee on Data Engineering 32, 4 (Dec.), 38–43.

[Davidson07Workflow] Davidson, S., Cohen-Boulakia, S., Eyal, A., Ludaescher, B., McPhillips, T., Bowers, S., Anand, M. K., and Freire, J. 2007. Provenance in scientific workflow systems. Bulletin of the Technical Committee on Data Engineering 32, 4 (Dec.), 44–50.

[Ghandi08Regulatory] Gandhi, R. A. and Lee, S.-W. 2008. Regulatory requirements-driven risk assessment. Tech. rep., University of North Carolina.

[Groth09Trust] Groth P, Miles S, Modgil S, et al. Determining the Trustworthiness of New Electronic Contracts. In: Proceedings of the Tenth Annual Workshop on Engineering Societies in the Agents' World, (ESAW-09). Utrecht, The Netherlands; 2009.

[Hartig09Quality] Hartig O, Zhao J. Using Web Data Provenance for Quality Assessment. In: Proceedings of the 1st Int. Workshop on the Role of Semantic Web in Provenance Management (SWPM) at ISWC. Washington, USA; 2009.

[Miles10TOSEM] S. Miles, P. Groth, S. Munroe and L. Moreau. PrIMe: A Methodology for Developing Provenance-Aware Applications, ACM Transactions on Software Engineering and Methodology, 2010 (to appear).

[Miles07Validation] S. Miles, S. Wong, W. Fang, P. Groth, K. Zauner and L. Moreau. Provenance-based Validation of E-Science Experiments, Journal of Web Semantics 5(1), pp. 28-38, 2007.

[Moreau10OPM] Moreau L, Clifford B, Freire J, et al. The open provenance model core specification (v1.1). Future Generation Computer Systems. 2010. Available at: http://eprints.ecs.soton.ac.uk/21449/.

[Nagappan08Privacy] Meiyappan Nagappan and Mladen Vouk. A Privacy Policy Model for Sharing of Provenance Information in a Query Based System. Proceedings of the Second International Workshop on Provenance and Annotations (IPAW 2008).

[Patel08CAD] Manjula Patel, Alexander Ball & Lian Ding. 'Strategies for the Curation of CAD Engineering Models'. International Journal of Digital Curation 4(1), 2008

[Pinheiro06Proof] Paulo Pinheiro da Silva and Deborah L. McGuinness and Richard Fikes. A Proof Markup Language for Semantic Web Services. Information Systems. Volume 31, Issues 4-5, June-July 2006, Pages 381-395.

[Reimer04Notebook] Reimer Y, Douglas S. Implementation Challenges Associated with Developing a Web-based E-notebook. Journal of Digital Information. 4(3). Available at: http://journals.tdl.org/jodi/article/view/106.

[Scheidegger08Vistrails] CE Scheidegger, HT Vo, David Koop, Juliana Freire. Querying and Re-Using Workflows with VisTrails. ACM SIGMOD 2008.

[Townend05Fault] Paul Townend, Paul Groth, Jie Xu (2005) A Provenance-Aware Weighted Fault Tolerance Scheme for Service-Based Applications. In Proc. of the 8th IEEE International Symposium on Object-oriented Real-time distributed Computing (ISORC 2005)