Section 2 Stage 01: Technology verification

2.1.1 The certification process commences with a verification of the technology, followed by its validation. This part of the certification process begins with an agreement between LR and the technology developer as to the precise scope of the certification involved.

2.1.2 Ordinarily, the process of verification consists of a desktop design appraisal activity to determine whether the technology meets the minimum requirements of a set of standards or codes.

2.1.3 As existing codes and standards are often inadequate to assess novel technologies, an alternative approach is adopted through a TQ process.

2.1.4 The performance requirements and scope of work should be agreed as these will become the terms of reference for the remainder of the assessment process. The sequence activities related to technology verification are illustrated in Figure 3.2.1 Technology verification activities.

To commence the certification process, as a minimum, the technology developer will be required to submit the following documents for review. If the developer has an overall document register (e.g. MDR), this should be submitted with the documents clearly referenced. During the design review process, if additional documents are requested, they should be submitted to complete the initial assessment of the technology. As a minimum, the documents required are:

• Basis of design including the context of use and intended operating environment;
• List of any recognised codes and standards, if applicable;
• Equipment and components list;
• General arrangement, layout drawings, block diagrams or schematics;
• Detailed designs of the technology’s subsystems; Process flow diagrams/piping and instrumentation drawings, if applicable;
• Cause and Effects diagram, if applicable;
• Material specifications, including documentation related to the applicable operating environment;
• Design calculations and load analysis, if applicable;Performance assessments, if applicable;
• Electrical single-line diagrams, if applicable;
• Electrical power supplies and analysis, if applicable;
• Definition of the operational environment (e.g. ATEX, SIL, etc.);
• Risk analysis (e.g. FMEA, FMECA, etc.), if conducted;
• Control, safety, and protection systems, including any functional safety system, if applicable; and
• Quality assurance plan for manufacturing, transit, installation, commissioning, and operations.

2.3.1 LR recognises that in order to deliver a certificate based upon a series of assessment activities, its scope must be locked-in and agreed between the technology developer and LR from the outset. This is to avoid scope creep during the assessment, which can potentially cause delays and additional costs.

2.3.2 A typical scope for certification through technology qualification includes a statement that outlines the business goals and requirements, as well as a list of stakeholders and their contribution towards various activities throughout the process. The scope must also include major project objectives, major constraints, key milestones, assumptions, limitations, exclusions, deliverables, and benchmarks to help measure success. A well refined scope ensures that the certification process goes smoothly and that every qualification activity adds value to the end goal. Based upon the design review conducted earlier, LR may wish to clarify and confirm the final scope with the technology developer before proceeding with the rest of the verification process.

2.4.1 As stated earlier, in conventional certification processes, a mature product or technology is assessed against a set of recognised standards, codes or normative rules. This is usually a straightforward process, to confirm whether or not there is compliance with defined specifications.

2.4.2 As technologies characterised as being innovative or novel cannot ordinarily conform to recognised codes and standards without shoehorning them to fit their respective requirements, a new set of design specifications to replace the role of recognised standards needs to be established. This collective detailed set of technical specifications is referred to as the ‘Definition of the technology’. Aside from the technical specifications, it must also contain the metrics for measurement to avoid any opportunity for ambiguity during the rest of the assessment activities. Once confirmed by the technology developer and accepted by LR, this set of specifications (and their metrics) will become the terms of reference for every activity or sub-activity until the issuance of the technology qualification certificate.

2.4.4 Once confirmed, the definition of the technology shall not be modified, unless it is clear from objective tests that the technology is incapable of meeting one or more specifications without a fundamental change to its design.

2.4.5  If a modification to the definition of the technology is required, LR shall re-assess the full scope of the modification alongside the original scope of the certification process, prior to deciding whether the technology can still merit a certificate. LR will inform the technology developer of their decision in writing.

2.5.1 Within this certification process, the acceptance criteria are a set of statements, each with a clear pass or fail result, and related to a defined specification (see Ch 3, 2.4 Definition of the technology) . These statements are to reflect the minimum verifiable requirements that must be achieved to confirm whether conditions are met for a respective specification. If appropriate, the acceptance criteria may be documented in performance standards that include additional technical safety barrier systems as well as operational and organisational elements.

2.6.1 While the preceding activities were part of a pre-verification stage, formal verification commences with an interactive technology appraisal workshop. As the objective of verification is to understand how the technology is likely to perform in the intended environment, LR encourages the workshop participants to comprise of experts from a variety of backgrounds with knowledge and experience related to the design or operation of the technology. Each of these experts can support this process by posing relevant questions related to the deployment of the technology over its planned lifecycle. To this end, the participants should include technology developers, consultants providing targeted services for the technology, LR assessors with relevant domain knowledge, and a host of other stakeholders, such as representatives from regulators, insurers, financiers, end users, suppliers, etc.

2.6.2 LR recommends that since the workshop is interactive wherein several experts participate in detailed technical analysis of the technology, it should ordinarily be an in-person (face-to-face) event. Based upon recent experience, LR has learnt that remote contribution by the main participants especially across several time-zones, extended display-screen fatigue, domestic circumstances, language barriers and unreliable internet connections, does not provide the best experience for both assessors and developers. Remote attendance (e.g. using phone or video communication) should be avoided, other than for a small number of specialists expected to provide limited input.

2.7.1 This is the first activity conducted during the technology appraisal workshop. Also referred to as a system decomposition, the objective is to identify distinct components and subsystems, each of which can subsequently be assessed for their maturity. The decomposition process stops at the lowest component/subsystem that delivers a function for the technology.

2.7.3 Structural decomposition is based upon the physical identification of the major subsystems and interfaces of the technology with other systems. It progressively breaks these subsystems down until the lowest component for which a maturity level can be assigned is identified. The outcome of this process is a list of subsystems and components, each of which will need to be analysed in detail during the remainder of the technology assessment activities. Information available from this process is required to drive the technology assessment process (see Ch 3, 2.8 Technology assessment and Ch 3, 2.9 Integration assessment.

2.7.4 Functional decomposition follows structural decomposition and focuses upon understanding the impact on a component of functioning under dynamic operational conditions. These operational conditions should also include systemlevel functions and structure, human-machine interface, roles and responsibilities of operational personnel and other support systems (e.g. IT systems). The outcome of this process is the production of a significant amount of information which is required to drive the risk assessment process (see Ch 3, 2.12 Risk assessment ).

2.7.5 As part of the technology decomposition process, it can be useful to map the interdependence of each technology subsystem by using a network diagram. This can assist significantly during the integration assessment of the technology (see Ch 3, 2.9 Integration assessment ).

2.7.6 As it is possible that multiple components or sub-systems can be identical to each other, it will be beneficial to assign a unique identifier to each of them to assist with traceability throughout the certification process.

2.8.1 This qualitative analysis of the technology begins with an assignment of a readiness level for each identified component based on a universally recognised technology readiness level (TRL) framework (see Ch 5, 1 Technology assessment scale). In the interest of conducting an objective assessment process founded on a consistent datum of reference to provide an accepted understanding of the technology status, LR does not recommend the use of nonstandardised technology readiness levels.

2.8.2 While it is generally accepted that the TRL of the overall technology is considered the benchmark of maturity for commercial reasons, from the point of view of the certification process, each component of the technology must be analysed individually to identify the extent of its criticality within the system (see Ch 3, 2.11 Classification of a technology component).

2.8.3 The assignment of a TRL should consider a component in isolation, without regard to any other component or subsystem it might interact with. Its application under dynamic operational conditions or its influence on another subsystem should therefore be disregarded. The TRL of a component should instead be assessed against the definition of the technology (see Ch 3, 2.4 Definition of the technology).

2.9.1 Components within a technology can often pose uncertainties when integrated into a system. To provide system engineers with a tool to evaluate the quality of such uncertainties based on the characteristics of a system’s configuration, an integration readiness level (IRL) between interfacing components is applied. This metric is based on a universally accepted integration readiness level (IRL) framework (see Ch 5, 2 Integration assessment scale).

2.9.2 This framework provides a systematic measurement for integration between components and must be performed for each identified component during the technology decomposition activity (see Ch 3, 2.7 Technology decomposition).

2.10.1 It must be recognised that the TRL of a technology does not necessarily equate to its appropriateness for its required application. Factors such as the quality of interaction with other components and the operational environment have a cumulative influence on its performance.

2.10.4 The calculation of the mTRL for a component is based upon the mathematical equation:

• mTRL_(component) = reduction factor_(component) x Original TRL_(component)

For details of how the reduction factor for a component is derived, see Ch 5, 3 Derivation of a reduction factor for a component.

For example, to calculate the mTRL for Component E in Figure 3.2.2 Non-linear system with the minimum IRL = 3,

the equation is given as:

from Ch 5, 3.1 Derivation of component's reduction factor 3.1.3, we have the reduction factor (y) for IRL = 3 calculated as:

hence, for Component E with a TRL = 2, the mTRL is calculated as:

As the TRL of a component has to be rounded to the nearest integer, the mTRL_{(Component E)} = 1.

2.11.1 The final activity of the technology assessment process is the determination of the technology class for a component. This refers to its state of maturity in relation to both, its calculated mTRL, and experience of use in the operating environment. The outcome of this activity determines the intensity of further qualification activities required for the respective component to comply with the expected requirements as stated in the definition of the technology.

2.11.3 Components classified as being in the red, amber, or yellow areas on the matrix (see Figure 3.2.4 Technology classification matrix) are considered to exhibit a degree of criticality and so are referred to as Critical Technology Elements (CTE). A CTE is a component which, upon failure (or partial failure), is likely to impede the minimum operational requirements of a technology or can potentially prevent the system (or interfacing technologies) from returning to a safe state.

2.11.4 Components classified in the green areas on the matrix are normally regarded as being mature or unlikely to pose a risk to the technology. These should be screened out from any further testing or other qualification activities. However, during the technology appraisal workshop a discussion must still be held to determine whether any component belonging to this class is regarded as critical. If so, a determination must be made to reclassify the component accordingly.

2.11.5 The final list of CTE must be based upon consensus between all workshop participants before progressing with a risk assessment of the technology. The CTE determined during this classification process, should form the basis for the risk assessment activities.

2.12.1 The purpose of a risk assessment is to provide further information on how the technology will perform under dynamic conditions within the operational environment. A risk assessment is not a substitute for a technology assessment but complements good engineering judgement and expertise, and hence follows a technology assessment.

2.12.2 The inputs to this assessment are the findings from the technology assessment which identified the CTE and the extent to which further qualification activities may be required. A secondary input to the risk assessment is the outcome of the functional decomposition of the technology, which provided information of how various components may respond under dynamic operational conditions (see Ch 3, 2.7 Technology decomposition 2.7.4). Additionally, inputs from any previous risk assessment work conducted by a technology developer should also be considered.

2.12.3 Provided the risk assessment technique(s) follow international or industry accepted standards, this document does not mandate any particular scheme (see Ch 5, 4 Risk assessment techniques). Depending upon the technology under assessment, workshop participants should determine which risk assessment technique would be most appropriate. Whatever technique is applied, the objective is to identify reasonably foreseeable risks and potential risk reduction measures that should be considered to mitigate them.

2.12.5 ISO/IEC 31010 provides guidance on the selection and application of systematic risk assessment techniques. As a starting point, in LR’s experience, a qualitative analysis such as a simplified FMEA tends to be effective in highlighting basic failure modes and those of their consequences which can challenge the achievement of the objectives as stated in the definition of the technology (see Ch 3, 2.6 Technology Appraisal Workshop).

2.12.6 IEC 60812 provides a technique for systematically analysing each possible failure mode and identifying its effect on safety, operations, asset costs and the environment. The guidance provided in this standard should be used to redefine the various levels of probability of failure and the consequences to the technology. LR recommends that during the risk assessment process, a risk ranking is considered based upon a risk priority number (RPN), which is a function of the probability of failure of a component and the consequences of this.

2.12.7 If findings from the initial risk assessment advise that specific components need to be analysed further, workshop participants should decide on the scope, objectives and assessment technique that would be used for such an evaluation. Participants should also determine if any additional assessment activity can be carried out during the technology appraisal workshop, or if a secondary risk analysis workshop with a limited scope should be held later to conclude the risk assessment process. For example, for a novel chemical process, a HAZOP (see Ch 5, 4.3 Hazard and Operability study (HAZOP)) risk assessment may be an appropriate secondary assessment technique to further understand process operational risks.

2.13.1 Throughout the technology and risk assessment processes, the operational impact on individual components was assessed against the definition of the technology (see Ch 3, 2.4 Definition of the technology). This provided assurance that each CTE was identified and investigated, and that any further qualification activities were suitably determined.

2.13.2 The final activity in the technology verification process consists of merging the findings from the technology and risk assessment activities. During this system assessment activity, the technology should be evaluated in relation to its potential integration with other technologies or systems it is expected to operate with.

2.13.3 If the consideration of any relevant standards or codes is beneficial for providing general guidance to the technology developer, these should be discussed as part of this activity to enable the developer to re-evaluate their goals, especially if the technology is intended to be deployed alongside mature technologies or destined to operate in a regulated environment (e.g. hazardous areas).

2.13.4 During a system assessment activity, non-technical stakeholders should also be invited to participate in the discussions, as the findings can impact commercial decisions with respect to further development of the technology.

2.14.1 This is a deliverable from LR which follows the conclusion of the technology verification stage. This document forms the basis of the next stage of the certification process. It also serves as the body of evidence required by the technology developer to demonstrate that the technology has undergone an independent objective assessment process.