G1111-9 VTS 시스템·장비 인수 체계 (요약·한영 대역)

IALA GUIDELINE G1111-9
FRAMEWORK FOR ACCEPTANCE OF VTS SYSTEMS

Edition 1.0, December 2022
urn:mrn:iala:pub:g1111-9:ed1.0

International Association of Marine Aids to Navigation and Lighthouse Authorities
Association Internationale de Signalisation Maritime
10, rue des Gaudines, 78100 Saint Germain en Laye, France
Tél. +33 (0)1 34 51 70 01 · [email protected] · www.iala-aism.org

Revisions to this document are to be noted in the table prior to the issue of a revised document.

1. Introduction
  1.1 Document purpose
  1.2 The G1111 Guideline series
2. Acceptance requirements
3. Acceptance process
  3.1 Acceptance Process Framework
  3.2 Acceptance Steps
    3.2.1 Design review
    3.2.2 Factory acceptance
    3.2.3 Site acceptance
    3.2.4 Final handover
  3.3 Acceptance documentation management
    3.3.1 Requirement traceability matrix
    3.3.2 Test plan
    3.3.3 Test procedure
4. Definitions
5. Abbreviations
6. References

List of Tables
Table 1 An example of the simple requirement traceability matrix

List of Figures
Figure 1 VTS system breakdown structure and order of acceptance
Figure 2 Acceptance steps within the establishment of a VTS system

This sub-Guideline presents a common source of information to assist competent authorities and VTS providers in the framework for acceptance of VTS systems and VTS equipment. Tailoring is required to capture the specific and relevant acceptance framework from the generic information included within this document.

This Sub-Guideline is associated with Recommendation R0128 Operational and Technical Performance of VTS Systems and VTS Equipment [1], a normative provision of IALA Standard S1040 Vessel Traffic Services (VTS). To demonstrate compliance with the Recommendation, the provisions of this Sub-Guideline should be taken into account.

IALA Guideline G1111 Establishing Functional & Performance Requirements for VTS Systems provides a framework to assist these authorities in preparing requirements for a VTS system(s) and VTS equipment.

This document provides a framework for the acceptance process such that the specified system:

• is working according to the agreed requirements (e.g., verification); and
• is suitable for the intended services (e.g., validation).

As a result, there will be a common understanding between the VTS provider and the supplier about the set requirements and the procedures that demonstrate compliance.

This document's suggested steps can be tailored depending on the system's size and/or complexity.

This Sub-Guideline provides high-level guidance and doesn't replace existing internationally recognized standards such as ISO 21502:2020 [2], nor does it seek to replace individual suppliers' project management and acceptance methods.

The G1111 series of guideline documents present system design, sensors, communications, processing, and acceptance, without inferring priority. New sub-guidelines can be introduced to the G1111 series in future. The guideline documents are numbered and titled as follows:

• G1111 Establishing Functional & Performance Requirements for VTS Systems
• G1111-1 Producing Requirements for the Core VTS System
• G1111-2 Producing Requirements for Voice Communications
• G1111-3 Producing Requirements for Radar
• G1111-4 Producing Requirements for AIS
• G1111-5 Producing Requirements for Environment Monitoring Systems
• G1111-6 Producing Requirements for Electro Optical Systems
• G1111-7 Producing Requirements for Radio Direction Finders
• G1111-8 Producing Requirements for Long Range Sensors
• G1111-9 Framework for Acceptance of VTS Systems

IALA Guideline G1150 [3] advises on the planning and implementation of a VTS, including the purpose and timing to define the acceptance plan. The VTS provider should establish an acceptance plan at the same time as they establish the documents in the planning phase, such as project plan, functional requirements, risk plan, communications plan and procurement plan.

VTS provider should prepare an acceptance plan to define the acceptance requirements, which could include:

• Scope of the project
• Role and responsibility
• Levels of testing (VTS system and/ or VTS equipment, integration, regression)
• Types of testing (design review, factory, site, functional, performance, etc.)

The acceptance requirements should be defined in the planning phase of the overall VTS project and prior to the implementation of the VTS system.

The acceptance process should demonstrate the compliance of the VTS system to the agreed requirements before operation. For an acceptance process to work effectively, the VTS provider must present the operational, functional and performance requirements clearly and completely so that the supplier can address each requirement during the acceptance testing process. Where requirements are not precise, the parties should discuss the details in order to arrive at a common understanding. Acceptance testing can only be performed on requirements that have been clearly defined by the VTS provider and provided in writing to the supplier.

This section provides a general framework to manage an acceptance process and suggests possible acceptance steps, focusing on the possible steps and documentation in the acceptance process.

A VTS system may include many different technologies at single or multiple sites to support the VTS operation. These technologies include the core VTS system, communications systems and sensors, described as VTS equipment of the VTS system in Figure 1.

The acceptance process is intended to demonstrate that the delivered VTS system complies with the operational and functional requirements as agreed between the VTS provider and the supplier.

The acceptance process of VTS system and VTS equipment comprises the following elements:

• A set of agreed requirements
• A (physical) implementation of the requirements
• A group of acceptance tests to evaluate the fulfilment of the requirements

It should be noted that, in some cases, a VTS system can be delivered from a supplier on a turnkey basis. This means that the supplier has complete responsibility for the delivery and implementation of the VTS system. Before presenting the complete VTS system to the VTS provider, the supplier will accept VTS equipment on behalf of the VTS provider under the acceptance process that has been agreed upon, as illustrated in Figure 1.

In some cases, it could be agreed that the VTS provider may be involved in VTS equipment testing where this testing can be better performed at the premises of a VTS equipment supplier. This should be agreed upon before the start of the project.

The acceptance process should follow an agreed acceptance plan that clearly defines the acceptance stages. Each acceptance stage should be supported with an agreed acceptance test procedure that shows how the acceptance stages align with the defined and agreed operational, functional and performance requirements.

Figure 1 VTS system breakdown structure and order of acceptance

A VTS system may be acquired as an off-the-shelf solution or customized to meet specific requirements.

Where the VTS system is planned to be delivered as an "off-the-shelf" solution, the acceptance process will typically comprise steps as follows:

• Factory acceptance
• Site acceptance
• Final handover

Where the VTS provider has specific requirements for VTS system and/ or VTS equipment, it may need to agree on a tailored test plan to ensure that the special features will be implemented correctly. Before implementing the VTS system and/or VTS equipment, VTS provider and the supplier should clearly understand how the operational, functional and performance requirements will be implemented by the proposed VTS system.

If it is discovered that the agreed specific requirements and the proposed VTS system are not aligned while proceeding with the acceptance steps, both parties need to agree upon how to proceed. Figure 2 shows the typical steps to establish a VTS system and illustrates acceptance steps as:

• Acceptance at design review(s)
• Factory acceptance
• Site acceptance
• Final handover

This acceptance process can apply equally well to the different VTS equipment.

The VTS provider should review a test readiness to assess if the testing system is ready to proceed into the acceptance testing. Reviewing test readiness is normally conducted at each acceptance stage before executing the test, defined in the test plan. The test readiness for each acceptance step can be ensured by checking the following aspects, including but not limited to:

• The test procedures including test scripts, are completed, agreed and fit the test purpose.
• The test demonstrates one or more agreed operational, functional or performance requirements.
• The presence of authorized and qualified personnel (both VTS provider and supplier).

Figure 2 Acceptance steps within the establishment of a VTS system

The acceptance steps can include design review(s) in the test plan. The design review confirms that the designed VTS system and VTS equipment align with the agreed specific requirements. Early involvement of relevant stakeholders in the process of reviewing the system architecture and design development:

• ensures the fulfilment of the specific requirements before implementation;
• helps to identify possible risks (e.g., first-time development risk, delay of delivery) and prepare the approach to them;
• develops a mutual understanding of the system design and architecture;
• develops a mutual understanding of the way, steps and timing to accept the system; and
• helps VTS provider to be aware of design, performance, and legality issues.

Typically, in a design review, a "walk through" might be done to check if VTS system and VTS equipment are designed correctly to form a coherent and integrated system. In this stage, issues such as flexibility, expandability, security, etc., might be clarified. This is also a base for a more detailed VTS equipment design, upon which further subsequent orders might be placed.

The VTS provider can set an acceptance step at the factory level (factory acceptance) for VTS system and/ or VTS equipment to demonstrate the fulfilment of the functional and technical performance and their interaction with the agreed requirement before installation on-site.

The factory acceptance is unique in the acceptance steps because the specific and specialized test equipment (e.g., measuring equipment and test jig) and the controlled environment are available, which makes factory acceptance:

• Methodical
• Efficient
• Precise
• Repeatable

Depending on the importance and criticality of the system, the VTS provider can select to conduct factory acceptance by:

• documentation, which the supplier publishes (e.g., a quality assurance document and factory test report); or
• VTS provider or VTS provider's representative witnessed the factory acceptance test (FAT).

The above way of factory acceptance could be decided at an earlier stage (e.g., planning phase, design phase and design review).

The VTS provider can set an acceptance step on-site (site acceptance) to demonstrate VTS system and/or VTS equipment to evaluate against the agreed requirements after installation. Site acceptance should not be conducted to repeat the FAT. The VTS provider can select to set site acceptance as a single test or multiple tests depending on delivering VTS system or VTS equipment complexity. The VTS provider and the supplier may need to test the site conditions, such as environment, access and infrastructure, to fit the purpose before installing VTS system and/ or VTS equipment.

Site Acceptance may be the only opportunity to test fulfilling the requirements in the operational environment before the operation launches. Site Acceptance test includes the following:

• demonstrating the compliance and suitability of the delivering system to the mutually agreed requirement;
• observing specific dynamic performance measurements;
• observing live data in the representative environment (e.g., geography);
• evaluating the interaction and integration:
  • of the developed VTS system or VTS equipment; and
  • between the developed VTS system or VTS equipment and the present system and infrastructure; and
• evaluating the ergonomics.

Final handover is a VTS provider's final acceptance step. The VTS provider can start VTS operations supported by the delivered VTS system after the final handover.

The final handover process includes the followings:

• Completing documentation
  • User manuals
  • Technical manuals
  • Project documentation (e.g., test documentation and test results)
• Completion of training
  • Operator training
  • Maintenance training
• Handover certificate

The purpose of a proper acceptance process is that the system to be delivered, when set operational, meets all the requirements. It should be considered to organize the acceptance process and the documentation belonging to that process to control the alignment between agreed requirements and the delivered VTS system. During the acceptance process, both supplier and VTS provider develop a mutual understanding of the requirements to be fulfilled. All the performed acceptance processes, e.g., planning, set-up and result, should be documented.

The acceptance documentation may include:

• Test plan for the entire project
• Factory and site acceptance test procedure
• Factory and site acceptance test report

It is essential to record, map and trace the link between the requirements and implementation of a VTS system. A requirement traceability matrix (RTM) may be included in the test procedures to show how the test achieved the agreed requirements. The RTM may include the following:

• The agreed requirements - which should each be uniquely identified
• Test criteria for each uniquely identified requirement (pass/fail criteria)
• Test procedure - which describes the tests to be conducted and the relationship to the requirements

The test plan describes an overview of acceptance steps and how the supplier intends to fulfil the requirements according to the agreed acceptance requirements. The VTS provider should approve the test plan before executing tests.

There are the following discussions and practices in the process of producing the test plan:

• The number and level of the acceptance steps should be appropriate for the type of VTS system.
• The test plan should be supported by test procedures based on the agreed requirements.
• The expected location, date and time of the testing should be established and agreed.

It should be noted that if specific acceptance steps are required, these should be included in acceptance requirements and agreed upon between the VTS provider and the supplier before the start of the project.

The test plan of a VTS system should describe how the overall acceptance steps are organized, and this may include:

• Scope
• List of items to be tested: which can be a plan to design, implement, integrate and/or interface a VTS system or VTS equipment
• Test approach
• Test readiness criteria
• Resources and Schedule
• Test Criteria (may be included in the RTM)
• Regression testing
• Role and responsibility of the stakeholders (e.g., the personnel involved or conducting the acceptance test)
• Dependencies between process and steps
• Logistics arrangements
• Key milestones

The VTS provider or representatives of VTS provider personnel who accept the VTS system should be:

• familiar with the set-up and operation of the VTS system; and
• appropriately qualified to review test reports and accept the system and/or VTS system.

The supplier personnel who test the VTS system should be:

• familiar with the set-up of the system to be tested;
• appropriately qualified to perform the test; and
• appropriately qualified to decide the direction in case of discrepancies.

Test procedures should describe how to execute an acceptance test for a VTS system. The test procedure typically includes:

• Scope and method of testing
• Test script
• Test report

The scope of acceptance testing may include one or more methodologies. The basic test methodologies applicable to system acceptance may include the following:

Inspection: Inspection determines acceptance by whether the system is in proper condition and right quantity involving examination and observation (e.g., paint colour, weight, physical dimensions, etc.).

Similarity: Similarity determines acceptance by whether the system is modified or similar to an existing accepted system. It needs to be a similar system and/or system architecture.

Analysis: Analysis determines acceptance by showing theoretical compliance under defined conditions between the proposed system and the agreed requirement. Analysis (including simulation) is used where testing to realistic conditions cannot be achieved or is not cost-effective. This is specific case such as wind resistance testing of a radar antenna.

Demonstration: Demonstration determines acceptance by a practical or qualitative exhibition and explains how the system works or performs functionally. Demonstration may be conducted by using a set of system stimuli or test jig. Demonstration can be used to show that the system responses to the stimuli are suitable. Demonstration may be appropriate when requirements or specifications are given in statistical or practical terms (e.g., mean time to repair, average power consumption, etc.) or test item is a partial system of a larger system (e.g., replace or repair of system module).

Operational Trial: Operational Trial is a period to demonstrate system performance and reliability.

Certification: Certification is a written assurance that the product can perform its assigned functions by legal or industrial standards (e.g., CE certification, UL certification).

The test script is the most detailed documentation of the acceptance testing within the test procedure. It is a line-by-line description of the action required to execute the acceptance tests. The test script typically has steps that describe the planned testing in as much detail as possible and specifies the required results.

A test script typically includes:

• Test ID from requirement traceability matrix
• Type of tests, methodology and scenarios
• Test conditions and environment
• Test equipment
• A (detailed) description of the test set-up (if not described elsewhere yet)
• A detailed description of each test
• An expected outcome of the test
• A pass/fail indication

After each test is performed according to the Acceptance Test procedure, the supplier should produce a test report. The purpose of the test report is to record and have a reference on:

• References to the project name
• Test date and time
• Test environment (e.g., weather conditions)
• List of instruments and their calibration status
• Tested requirement(s)
• Test outcome (e.g., Pass/fail, measurement result, log file)
• List of items to be tested (including software revisions and hardware serial numbers)
• Remarks and report of discrepancies, if applicable
• Person(s) who performed/witnessed the test and signatories

The terms in this document are explained as follows:

Supplier: The organisation providing a VTS system or VTS equipment.

system: A system is an arrangement of parts or elements that together exhibit behaviour or meaning that the individual constituents do not (as defined in [4]). This can be a group of items or devices working together.

Test procedure: A (detailed) sequence of steps to be executed to demonstrate compliance to a requirement.

VTS system: within the G1111 guidelines, the VTS system is the VTS software, hardware, communications and sensors. This excludes personnel and procedures.

VTS equipment: within the G1111 guidelines, VTS equipment refers to the individual items of software, hardware, communications and sensors, which make up the VTS system.

Other definitions of terms used in this Guideline can be found in the International Dictionary of Marine Aids to Navigation (IALA Dictionary) and were checked as correct at the time of going to print. Where conflict arises, the IALA Dictionary should be considered as the authoritative source of definitions used in IALA documents.

1. IALA. Recommendation R0128 Operational and Technical Performance of VTS Systems.
2. ISO. (2020) ISO 21502:2020 Project, programme and portfolio management — Guidance on project management.
3. IALA. Guideline G1150 Establishing Planning and Implementing a VTS.
4. INCOSE. Definition of system: General Case. Retrieved 25 April 2022, from https://www.incose.org/about-systems-engineering/system-and-se-definition/general-system-definition.