G1195 VTS 영어 의사소통 역량 평가 (요약·한영 대역)

IALA GUIDELINE G1195
VTS ENGLISH COMMUNICATION COMPETENCY TESTING

Edition 1.0, December 2025
urn:mrn:iala:pub:g1195:ed1.0

Revisions to this document are to be noted in the table prior to the issue of a revised document.

1. Introduction
2. Document purpose
  2.1 Relationship to key documents
3. Benefits of assessing competency in VTS English communications
4. Elements of quality competency test
5. Evaluation framework
  5.1 Evaluation criteria (5.1.1 Message compilation · 5.1.2 Message delivery · 5.1.3 Message interpretation · 5.1.4 Standard phraseology)
  5.2 Rating scale
6. Development of a competency test
  6.1 Defining test objectives and competencies required
  6.2 Testing methods (6.2.1 Traditional tests involving human interactions · 6.2.2 Computer-based testing · 6.2.3 AI-based evaluation systems · 6.2.4 Simulation-based evaluation)
  6.3 Test duration
  6.4 Scoring allocation
  6.5 Test questions (6.5.1 Applying evaluation criteria · 6.5.2 Selecting question types · 6.5.3 Levels of difficulty · 6.5.4 Utilisation of media)
  6.6 Trial test
  6.7 Test implementation (6.7.1 Test environment · 6.7.2 Instructions and preparations · 6.7.3 Monitoring and supervision · 6.7.4 Use of technology)
  6.8 Test evaluation, review and update
7. Assessment process
  7.1 Collection and verification of results
  7.2 Evaluation, review, and summary of results
  7.3 Recording and retention of results
  7.4 Notification of results and feedback delivery
  7.5 Handling of issues
  7.6 Ongoing enhancement of the assessment process
8. Skills and qualifications for test design and assessment

The use of standard messaging structure and phrases when interacting with participating ships is essential for Vessel Traffic Services (VTS) to ensure clear, concise, and unambiguous communications and minimize potential misunderstanding of the intent of messages and reduce the time required for effective communication.

Ensuring the competency of VTS operators in the use of standard messaging structure and phrases is essential to for a VTS to:

• Identify the strengths and areas where VTS operators may need to improve;
• Monitor the progress and growth of VTS operators by providing appropriate feedback and support; and
• Evaluate the effectiveness of training programs (e.g. model courses, OJT at the VTS).

Noting the diversity of linguistic backgrounds and cultural experiences worldwide, this guideline has been prepared to facilitate global harmonization in the delivery of VTS messages and their intent.

The purpose of this guideline is to provide a framework to assess the competency of VTS operators in the use of standard messaging structure and phrases to:

• Facilitate clear, concise, and unambiguous VTS communications that are timely and effective; and
• Minimize misunderstanding of the intent of messages and reduce the time required for effective VTS communication.

The Guideline may assist:

• VTS providers to monitor that VTS operators continue to conform with the practices for VTS communications as outlined in IMO SMCP and IALA G1132. This framework may be incorporated into periodic performance assessments, On-the-Job and revalidation training programmes to assess competency with standard message structure and phrases; and
• VTS training organizations to assess the competency of how students use standard messaging structure and phrases in the delivery of model courses.

This Guideline is associated with IALA Recommendation R1012 VTS Communications, which is a normative provision of IALA Standard S1040 Vessel Traffic Services, and shall be observed if compliance with this Standard is claimed. To demonstrate compliance with the Recommendation, the practices described in the Guideline should be taken into account.

This Guidance should be read in conjunction with:

• IALA Recommendation R1012 VTS Communications
• IALA G1132 VTS Voice Communications and Phraseology
• IALA G1141 Operational Procedures for Delivering VTS
• IALA G1089 Provision of a VTS
• IMO A.918(22) Standard Marine Communication Phrases (SMCP)
• IMO A.954(23) Proper Use of VHF Channels at Sea
• ITU publication Radio Regulations Volume 1 (2020) Chapter VII - Distress and Safety Communications.
• National procedures or documents on the use of VTS Communications

The benefits of assessing competency of VTS interactions and communications include:

1 Enable individuals to improve their communication skills

The competency assessment offers a comprehensive evaluation of an individual's ability to provide VTS by identifying their current level of proficiency and allowing individuals to focus on specific areas that require improvement, such as message structures, standardized vocabulary and phrases in verbal communication. This targeted approach helps individuals gain confidence and competence in using VTS communications effectively.

It also provides valuable insights into their strengths and weaknesses, enabling them to understand where they excel and where they can enhance their skills. This awareness empowers individuals to make informed decisions about where they may need to improve.

2 Improve interactions between VTS, ships and allied services

Effective communication is essential for the coordination of VTS operations. Improved VTS communication reduces misinterpretation and assists in establishing better relationships and coordination between VTS, ships and allied services.

3 Reduce workplace errors

Miscommunication can be minimized by ensuring that VTS operators have the necessary skills to communicate accurately and effectively to interact with vessel traffic and respond to developing situations within a VTS area.

An effective competency assessment must be valid, authentic, sufficient, current, and reliable. The key elements of a quality test are:

• Valid: The test content needs to accurately measure whether VTS operators have the skills they need to effectively communicate. Specifically, the test should include tasks that reflect real-world scenarios, ensuring the results are meaningful and relevant to the practical requirements of the role.
• Authentic: The test ensures that the evidence presented accurately represents the learner's skills, knowledge and abilities. The evidence is directly related to the assessment task and the work of the VTS operators.
• Sufficient: The test sufficiently assesses the VTS Operator's communication skills, against the evaluation criterion. Included in ensuring a sufficient test, this refers to how accurately a test reflects the language and communication demands of VTS operations and situations VTS personnel may encounter.
• Current: The test needs to be up-to-date and reflects the current knowledge and practices.
• Reliable: The test yields consistent results over time and across different assessors, with repeatability where the test produces similar results for an individual regardless of whether they complete it once or multiple times. To ensure credibility, the test must be administered and scored in a standardized way.

Reliability needs to be considered in three different ways:

• How the test is structured
  The structure of the test should enable results to be meaningfully compared between VTS operators. This does not imply that all VTS operators will face identical questions, but rather that the questions and tasks should be comparable in type and difficulty. All individuals should have the same amount of time to complete the test. In this way, results can be compared with confidence, and a high score will reliably indicate a high level of competency in VTS interaction and communications.
• How it is assessed
  Answers should be graded in an identical manner using the same standards, ensuring that each assessment criterion is consistently applied across different levels. This ensures errors are addressed uniformly, leading to accurate and meaningful results.
  Further, comparing the communication skills of different VTS operators or tracking an individual's competence over time can help to refine model courses and tailor other training programs.
• How it is given
  The conditions in which a test is given play can affect the outcomes. To ensure reliable results, tests should be administered in environments with minimal distractions, as certain settings may otherwise make it more difficult for some individuals to perform well in the test.

The evaluation criteria for VTS communication competency testing are based on the framework outlined in IALA G1132, these include:

• Message compilation;
• Message delivery;
• Message interpretation; and
• Standard phraseology.

This is the process of assembling and structuring information into a clear, coherent, and effective message that considers the operational context and communication procedures. The following should be taken into account:

• Message structure: the framework to convey information or instructions unambiguously using a standard format and content structure. Further, there are some general rules for constructing and composing VHF messages to emphasize brevity, clarity, and precision.
• Message markers: IMO Resolution A.918(22) defines eight message markers in the SMCP, seven of which (i.e. information, advice, warning, instruction, question, answer, and request) are frequently used in VTS operations for precise communication to ensure messages are understood as intended.
• Phonetic alphabet and numbers: The use of the phonetic alphabet and numbers enhances clarity by avoiding misunderstanding and preventing potential confusion in the delivery of specific words or letters over in radio communications.
• Standard use of maritime terminology: The correct application of maritime terminology - encompassing positions, bearings, courses, distances, speeds, times, geographical names, and abbreviations - essential for clear and effective communication.

These criteria ensure that VTS communications are professional, clear, concise, consistent, and accurate. The following considerations should be taken into account:

• Preparation when using VHF: The proper use of VHF equipment is essential to ensure that transmissions are successful, and the message's initial content is not lost.
• Tone and volume: Demonstration of calm confidence, professionalism, and respectfulness in tone. The voice volume should be moderated to normal conversation levels to avoid distortion from shouting and prevent inaudibility from soft speech.
• Emphasis on keywords: Highlighting of key words in a message by pronouncing them with a slightly elevated tone and longer duration to ensure they are distinctly accentuated.
• Speech rate: The recommended speech rate is about 120 words per minute (WPM) for routine communication, whereas a slower rate of 100 WPM is advised for emergencies or when dealing with developing unsafe situations, to ensure clarity of speech.
• Word grouping and pausing: The utilization of word grouping (the division of sentences into smaller groups or phrases) and the insertion of brief pauses between each word group both for enhancing listeners' understanding of the message and for facilitating speakers' preparation for delivery, ultimately contributing to mutual intelligibility.
• Questioning techniques: The effective application of questioning techniques (i.e. closed questions, open questions, and funnel questions) to gather and disseminate information based on real-time situations within the VTS area.
• Avoidance of ambiguous language: The avoidance of unambiguous terminology and the use of local terminology can lead to confusion and misunderstandings.

• Confirmation/Clarification of message: The effective application of standard communication protocols involves various actions such as confirmation, correction, reinforcement, and request within maritime exchanges. This includes:
  • "Yes" or "no" followed by the appropriate phrase, and responding affirmatively or negatively
  • "Received" to acknowledge receipt of the message
  • "Stand by" to indicate temporary unavailability of the requested information
  • "Correction" followed by the corrected part of the message
  • "Repeat" to emphasize the corresponding and important part of the message.
  • "Say again" when messages are not properly heard.
• Distress and safety communication: The ability to apply operational procedures to provide accurate, effective communications in response to distress and safety situations, even under the pressure.

Message interpretation focuses on how VTS operators use various communication strategies to accurately understand and interpret messages. This process aims to enhance mutual understanding and effective communication by taking into account:

• Effective listening skills: Observations should be made on aspects of attentive listening, clarity of information, and accurate interpretation of the information provided. Individuals should pay close attention to the speaker's message by:
  • engaging in appropriate communication to clarify information.
  • identifying key issues and confirm accuracy.
  • determining the most appropriate message markers according to the target communicative situations, to effectively convey the intention of the message and facilitate the listener's understanding.
  • identifying the most appropriate communication procedures to respond to the situation. For example, if the situation demands an instruction, then they should be able to respond with the appropriate message marker and standard phrases.
• Closed-loop communication: To reduce the risk of misunderstandings, a closed-loop communication or read-back technique may be used to confirm that VTS messages are correctly received and understood.

Individuals should be familiar with standard phraseology and be able to immediate recall phrases consistent with:

• IMO Resolution A.918(22) - the IMO Standard Marine Communication Phrases
• Part C of IALA Guideline G1132 on VTS Voice Communications and Phraseology

Competency testing should consider the following core areas associated with the provision of VTS, which may include:

• General communications (e.g. Radio Checks, VTS Operational Status, Requesting Reports, Call Requests, Use of Other VHF Channels, Reporting of Ship Identification and Particulars, Speed, Engine)
• Provision of information (e.g. traffic information, weather information, tidal/hydrological information, information broadcasts).
• Management of ship traffic (e.g. navigating in the VTS area, berthing, proceeding from or to an alongside berth or anchorage, anchor operations, pilotage).
• Responding to a developing unsafe situation.
• Applying the phraseology as appropriate for the specific situation.

It is acknowledged that situations may arise where no standard phrases have been identified. Although plain language¹ or locally adopted phrases may be used in these circumstances, as described in IALA G1132 on VTS Voice Communications and Phraseology, it is advisable to design the test in a manner that minimizes the use of plain language or locally adopted phrases.

The VTS communication proficiency rating scale has been developed to assess the competency of VTS operators in the use of standard messaging structure and phrases. Level 1 is designed for beginners, where they may only have a foundational understanding of VTS, while Level 6 indicates a high level of proficiency.

It is expected that qualified VTS operators should achieve at least a Level 4 rating on the scale to be considered competent in VTS communications.

Annex A provides reference level descriptors of how this rating scale may be used to assess competency against these four evaluation criteria elements. Each element is further detailed with six progressive levels, ranging from Basic Competence (Level 1) to Expert (Level 6). These reference level descriptors provide a clear progression of skills, enabling assessors to comprehensively and systematically evaluate competencies.

The following descriptions provide a broad definition of the levels of proficiency in VTS communications.

Competency tests are tools designed to evaluate an individual's knowledge, skills, and abilities in a consistent and objective manner. These tests play a role in ensuring that VTS operators have the necessary communication competencies to perform their duties effectively, both in routine and high-pressure scenarios.

Properly designed competency tests should focus on genuine skill development and practical application. To achieve these objectives, it is essential to adopt a structured and systematic approach to test design. A test matrix serves as a valuable tool by organizing key components of the test and aligning them with clearly defined goals. This includes defining evaluation criteria, setting difficulty levels, balancing question types, and allocating scores to key assessment components. This document provides the framework that can be adapted to consider operational requirements during the design and application of a test matrix.

By integrating the principles of a test matrix, the following advantages can be realized:

• Structural Integrity: Systematic design focuses on evaluating specific competencies rather than assembling unrelated questions.
• Consistency: Standardized criteria enable consistent assessments, regardless of an individual's background.
• Clarity in Test Development: Defined criteria provide a clear structure for creating coherent and well-organized assessments.
• Validity and Reliability: Tests measure intended competencies accurately and produce consistent results across administrations.
• Meaningful Result Interpretation: Precise performance analysis highlights strengths and areas for improvement, supporting actionable feedback and program refinement.

By addressing these factors, the test matrix can be designed flexibly to meet practical needs while maintaining alignment with global standards.

The objective of a competency test is to measure how VTS operators interact and communicate using standard messaging structure and phrases when performing a specific task or activity during VTS operations. Annex B identifies some of the communication and interaction activities regularly performed by VTS operators.

It is important that the objectives are defined in terms of the level of language competencies required and specific outcomes that the test needs to measure. These objectives should be clear, measurable, and focus on what the test is intended to evaluate.

To achieve the objectives, the individual should demonstrate they meet the required VTS communication skills and behaviour competencies as identified in the evaluation criteria.

The structure of the test and assessment methods may vary depending on the operational requirements and assessment tools available.

While traditional face-to-face or group-based assessments have been widely implemented, the increasing integration of technology has highlighted the inherent limitations of these human-dependent methods. Issues such as assessor subjectivity, varying levels of expertise, and the impact of fatigue can lead to inconsistencies in the assessment process which raise concerns about the objectivity and reliability of the results. Further, face-to-face assessments tend to be resource-intensive, making them less suitable for large-scale evaluations or assessments that require precise measurement of specialized language skills. These methods often lack standardized mechanisms for maintaining transparent test records, complicating the process of reviewing or verifying results in cases of disputes or re-evaluation.

In contrast, automated systems, such as computer-based and AI-driven assessments, as well as automated simulation-based evaluations, offer comprehensive solutions by systematically recording responses and evaluation data, thus ensuring transparency and accountability in high-stakes testing environments.

As technology continues to integrate into testing environments, it is important that the most suitable testing method is used to maximize objectivity, efficiency, and scalability. The following sections outline some testing methods, focusing on their specific features and advantages.

Traditional interactive assessments encompass various methods such as interactive interviews, role-play exercises, and listening tasks, each with distinct benefits and challenges. Interactive interviews is a commonly used method that provides a more natural and communicative testing environment, where the assessor can use prescribed scenarios and adapt situations on the spot which minimizes the likelihood of rehearsed responses. It is particularly effective in evaluating verbal communication skills through conversation and immediate feedback throughout the process. However, this approach is resource-intensive and requires strict standardization to minimize bias and can be affected by variations in human interactions.

Listening exercises serve as another option, where individuals listen to prompts and respond by answering questions or interacting based on what they hear. This method ensures consistency by using standardized prompts and allowing assessors to gauge listening comprehension and situational responses.

Another widely used method is role play, which evaluates the verbal and interpersonal skills, such as articulation, persuasion and problem-solving. Unlike interactive interviews, role-play exercises are designed to replicate real-life situations, focusing on routine or emergency communications where individuals need to manage and respond to developing situations. The exercises can be conducted either one-on-one or in group-based formats, providing a dynamic and practical assessment environment. While this method effectively simulates real-world conditions, it may face limitations in capturing the full complexity of such situations in a controlled setting.

Written communication tasks, such as preparing or revising digital messages, may offer insights into how effectively a VTS operator uses standard messaging formats and identifies errors. These tasks can highlight proficiency in structuring clear and accurate messages. However, they fall short in evaluating verbal communication skills and the interactive abilities essential for real-time operational scenarios, making them less suitable as a primary method for assessing communication competence in VTS contexts.

Computer-based testing involves individuals responding to pre-recorded, pre-set questions, with their responses recorded for later evaluation. This method eliminates the need for real-time interaction and ensures test consistency. The consistent nature of the questions and the uniform conditions under which they are administered guarantee objectivity, while the absence of live interaction minimizes the influence of spontaneous variability. All questions and responses are systematically documented, providing a clear record for subsequent review or re-evaluation if required. Additionally, this method supports the inclusion of multimedia elements, such as audio files, images, and videos, that facilitates a broader range of assessment activities.

This approach uses artificial intelligence (AI) to analyse individual responses in real time, assessing key elements such as pronunciation, vocabulary, grammar, and fluency. Results are automatically recorded, and the AI system applies predefined criteria to assign scores. Objectivity is maintained through the consistent application of structured evaluation criteria, thereby reducing the risk of human bias. All response data is securely stored, and AI-generated records are preserved for future verification if required. This system is particularly well-suited for managing large numbers of VTS operators efficiently, ensuring that all responses are recorded and available for subsequent review or analysis.

Simulation-based evaluation uses simulated scenarios to replicate specific situations and responses are recorded for later assessment. For example, a common simulation exercise is presented and the individual responds to a given problem. Standardized scenarios ensure objectivity as all individuals are tested under identical conditions. The entire simulation process including responses and actions is automatically recorded, providing a reliable record for dispute resolution or post-assessment review.

The length of the test should be kept to a maximum of 30 minutes to maintain the individual's focus and ensure a valid and efficient assessment. The rationale for this limitation is:

• For individuals, long tests increase mental fatigue and reduce performance accuracy, particularly in the later stages of the test. Limiting the duration helps to avoid cognitive overload, leading to a more balanced and accurate assessment of language proficiency; and
• For assessors, shorter tests ensure efficient grading and helps prevent inconsistencies in scoring due to fatigue. Focused, shorter tests promote a more consistent assessment for all individuals.

A score allocation should be given to each evaluation criteria to determine its relative importance in the overall assessment. These weighted scores may reflect specific training goals, operational priorities, national regulatory considerations, or the language proficiency of individuals.

Further, scoring allocation is important with the comparability of tests as it ensures that everyone is evaluated according to the same criteria and standards.

Test questions should be designed around the test objectives, while balancing four key elements that will form the framework for the test. The following elements ensure that the test is consistent, comprehensive, and provides a valid assessment of an individual's abilities:

• Evaluation criteria specify the skills being assessed, forming the foundation for structuring and formatting the test questions.
• Question types determine how individuals will respond and which particular skills will be evaluated, shaping the complexity of the assessment.
• Difficulty level to adjusts the depth and challenge of the questions, ensuring that the test accurately measures the required proficiency.
• Media utilization determines how the questions are presented (e.g. text, audio, video, or images) enhancing the realism of the examination and expanding the scope of the evaluation.

Annex B outlines common communication and interaction activities regularly performed by VTS operators during daily VTS operations, which can serve as a basis for developing test questions.

It is recommended to limit the number of questions and scenario exercises to no more than ten, progressing gradually with difficulty and complexity from a low level (N.1) to a high level (N.10). The reasons for this limitation are to:

• Reduce the cognitive load by limiting the number of questions to help prevent mental fatigue, allowing individuals to stay focused and perform at their best.
• Improve the test efficiency by using a focused set of questions to avoid redundancy and ensuring that each question serves a distinct purpose and contributes to the overall assessment.
• Prevent the test from becoming overwhelming for both individuals and assessors by assessing key competencies.

Some example test questions are provided in Annex C to illustrate how VTS communication competencies may be evaluated assessed through practical, scenario-based tasks.

Test questions should be structured around the evaluation criteria described in Section 5.1 (Evaluation Criteria) and Annex A: Reference Level Descriptors which focus on message structure, message delivery, message interpretation, and standard phraseology. This approach enhances the test's validity and reliability, enabling an objective evaluation of the individuals' abilities.

To comprehensively assess the individuals' diverse skills, questions should be presented in various formats that consistently evaluate communication skills. Each question type should target specific competencies while utilizing media such as text, audio, images, and video to ensure a holistic assessment of the individual's performance.

Examples of question types and applications are provided below, but the test should not be limited to these formats.

6.5.2.1. Text-Based Questions

These questions assess the individual's ability to convey information based on written text.

• Message Delivery: Provide a message based on the scenario.
• Message Revision: Correct and rephrase inaccurate written messages.
• Message Summarization: Summarize long texts and convey key points.
• Information Transmission: Relay information from text to a designated recipient.
• Situation Explanation: Analyse a written scenario and explain it.

6.5.2.2. Image-Based Questions

These questions evaluate the individual's ability to interpret visual information and respond appropriately.

• Situation Interpretation: Analyse an image and provide information using Standard VTS phraseology (e.g. SMCP, G1132).
• Warning Message Composition: Compose a warning message, applying an appropriate message marker based on the image provided.
• Situation Transmission: Relay visual information from the image to nearby vessels.
• Problem Analysis: Analyse a problem shown in the image and propose instructions for a vessel.
• Information Confirmation: Review the image, correct inaccurate information, and deliver the corrected information.

6.5.2.3. Audio-Based Questions

These questions test the individual's ability to interpret and respond to auditory information.

• Listening and Responding: Listen to an audio message and provide an appropriate response.
• Message Confirmation: Confirm the content of a message after listening.
• Message Transmission: Summarize auditory information and relay it to a designated party.
• Situation Summarization: Summarize complex situations based on audio information and relay them.
• Correcting Information: Listen to an incorrect message and provide a corrected version.

6.5.2.4. Audio-Visual-Based Questions

These questions test the individual's ability to interpret and respond to auditory information.

• Analysing and Responding: Analyse audiovisual material and provide an appropriate response.
• Problem Solving: Resolve issues presented through audiovisual materials.
• Information Transmission: Relay audiovisual information to a relevant party.
• Situation Summarization: Summarize the audiovisual situation and relay it.
• Verifying and Correcting Information: Review audiovisual materials, correct any inaccuracies, and provide an updated message.

6.5.2.5. Video-Based Questions

These questions require individuals to analyse visual scenarios in video format and respond accordingly.

• Analysing and Responding: Analyse the video and provide an appropriate response.
• Situation Transmission: Relay information from the video to another party.
• Warning Message Composition: Compose a warning message based on the visual information in the video.
• Situation Summarization: Summarize complex situations from the video and relay the information.
• Correcting and Transmitting Information: Watch the video, correct any misinformation, and provide the updated information.

6.5.2.6. Dialogue Simulation Questions

These questions simulate conversational scenarios, requiring individuals to engage in exchanges within a simulated environment.

• Initiating Dialogue: Begin a conversation and exchange instructions within the simulated scenario.
• Message Confirmation: Confirm and respond to the information provided by the other party.
• Requesting Additional Information: Request additional information when the message is unclear.
• Transmitting Information: Relay received information to the relevant recipient.
• Closing the Scenario: Appropriately conclude the conversation at the end of the scenario.

6.5.2.7. Interactive-Based Questions

These questions simulate real-time interactions, requiring individuals to engage in live dialogues and provide immediate responses.

• Real-Time Instructions: Respond to real-time instructions provided by a simulation system or assessor.
• Requesting Additional Information: Request supplementary information when the provided data is insufficient.
• Decision Making: Choose the correct course of action when presented with multiple options.
• Continuing the Conversation: Engage in a simulated conversation by responding appropriately to incoming messages.
• Situation Management: Manage and respond as the situation evolves in real time.

To ensure a comprehensive and accurate assessment of individuals' communication and interaction skills, it is important to progressively increase task difficulty using different media formats. Using a combination of text, images, audio, and video not only reflects real-world VTS communication challenges but also ensures a balanced evaluation of both basic and complex skills.

The test should start with simpler formats, such as text-based or image-based questions, focusing on foundational skills like message delivery or interpreting visual information. As the test progresses, more complex media formats such as audio scenarios or video simulations, should be introduced, requiring individuals to integrate multiple information sources and respond to dynamic situations.

Incorporating varied media formats throughout the test provides the following benefits:

• Enhanced Realism: Different formats simulate the diverse communication scenarios encountered in VTS operations, leading to a practical and relevant evaluation.
• Layered Skill Assessment: Mixing text, images, audio, and video enables a thorough assessment of a range of communication skills, from basic interpretation to complex problem-solving; and
• Adaptability Evaluation: Using different media types challenges individuals to switch between communication modes, which is critical for real-world operational environments.

When incorporating media in a test, it is important to consider key factors such as adjusting task difficulty, defining response scope, using standard phrases, managing scenario complexity, and maintaining practical relevance. These factors play an important role in ensuring the test not only accurately evaluates communication skills but also reflects the practical demands of real-world scenarios. Additionally, applying consistent evaluation criteria across all media formats further enhances the accuracy of the test.

• Adjusting Difficulty to Reduce Cognitive Load: The difficulty of the media used in the test should be adjusted to match the complexity of the tasks. For example, simpler tasks may involve text or images, while more complex tasks could incorporate audio or video content. By progressively increasing the difficulty, individuals are given the opportunity to demonstrate their abilities without being overwhelmed ensuring they can perform optimally.
• Ensuring Consistency in Responses and Use of Standard Phrases: Consistency in responses is critical for maintaining accuracy in the assessment process. Clearly defining the scope and purpose of each question reduces the potential for overly broad or varied answers. This clarity directs individuals toward responses that closely meet the task requirements, making it easier for assessors to evaluate based on predefined criteria. Additionally, using standard phrases is essential for measuring an individual's ability to communicate effectively in real-world VTS scenarios. Structuring test questions to prompt the correct use of these standard phrases ensures that individuals not only demonstrate their technical language proficiency but also reinforces consistency in their responses and aligns them with expected industry communication standards.

• Managing the Complexity of Situations in Media: While video and audio can create a more immersive testing environment, it is important that the complexity of the scenarios does not detract from the primary goal of assessing communication skills. Overly complicated or unclear scenarios can confuse individuals or cause unnecessary cognitive strain. The media presented should reflect real-life scenarios, allowing individuals to focus on effective communication without being overwhelmed by extraneous or irrelevant details.
• Maintaining Consistency in Evaluation Criteria: Regardless of the media format used - whether text, audio, images, or video - the evaluation criteria must remain consistent. This ensures that all individuals are assessed using the same standards, promoting the reliability of the test. Consistency across different media formats is essential for ensuring that the evaluation accurately reflects an individual's communication abilities.
• Ensuring Relevance to Operational Situations: The media used in the test should represent realistic scenarios that VTS operators are likely to encounter. The test should simulate situations such as emergency communications or routine traffic management, allowing individuals to demonstrate their ability to use standard phrases and respond appropriately.

Consideration should be given to conducting a trial test with a small, representative group to identify potential issues and make necessary refinements. This approach ensures a more accurate and effective competency assessment by addressing ambiguities and optimizing the test's structure and content.

The trial testing involves the following steps:

• Conduct Trial Test: An initial trial test with a small, representative group will help to identify any issues or ambiguities within the questions or scenarios. Focusing on these problems or unclear elements is essential for improving the test's overall quality. This process allows for a more thorough evaluation of how the test performs in a real-world context, providing clearer insights into its effectiveness and practical usability.
• Feedback Collection: Gather detailed feedback following the trial test on various aspects such as the clarity of instructions, the relevance of questions, and any issues encountered during the test. This feedback is valuable for identifying specific areas that require improvement and ensuring the test is both user-friendly and effective in assessing the required competencies.
• Result Analysis: Performance data from the representative test group should be analysed to evaluate how effectively the test measures the intended competencies and to identify patterns that may indicate issues with the test's structure or content. This analysis helps determine whether the test effectively reflects the various aspects of communication skills.
• Test Refinement: Based on the analysis and feedback, necessary adjustments should be made to the test. This may include refining questions, revising scenarios, or clarifying instructions to improve its accuracy and reliability. By iterating on the test design, the assessment process will be further strengthened over time.

When implementing a competency test, careful attention should be given to various factors to ensure the process is smooth, consistent, and effective. Proper planning and execution are crucial to achieving reliable and valid results while maintaining consistency and ensuring that all individuals are evaluated under standardized conditions. Key considerations include, but are not limited to, the following:

6.7.1. Test Environment

A controlled and consistent test environment is essential to ensure that all individuals are evaluated under the same conditions.

• Testing Space: The testing space should be designed to minimize distractions, ensuring it is free from noise.
• Individual Personal Space: Everyone should be provided with sufficient personal space to complete the assessment with minimal interference from others.
• Equipment Check: All equipment, including microphones and recording devices, should be thoroughly tested beforehand to ensure proper functionality.
• Time Limits: All individuals should be provided with the same amount of time to respond to each question set.

6.7.2. Instructions and Preparations

Clear instructions and adequate preparation of materials are critical to enable individuals to understand the test process and perform accordingly.

• Detailed Guidance: Information about the test format, timing, and evaluation criteria should be available.
• Practice Materials: Sample questions may be provided to familiarize individuals with the structure and content of the test.

For online assessments, individuals should be provided with information on how to access the platform including instructions on resolving basic technical problems.

6.7.3. Monitoring and Supervision

Proper monitoring and supervision should be maintained to ensure the integrity of the test and address unforeseen issues. Observers who are not involved in training may be used to oversee the process to enhance objectivity and transparency.

• Process Monitoring: The test process should be closely monitored to prevent technical issues or violations of test regulations.
• Real-Time Support: Supervisors should be available to resolve issues promptly and ensure the smooth progression of the test.

6.7.4. Use of Technology

Technology plays a vital role in modern speaking assessments, particularly in online or computer-based settings.

• Equipment Testing: Microphones, speakers, and recording devices should be tested in advance to confirm they are fully operational.
• Network Stability: Stable internet connectivity is required online assessments.
• Technical Support: Technical support should be available to address any unexpected issues.

Regular review and refining of tests are essential to verify that the objectives of the test and the competencies required remain focused, relevant, and effective by evolving to meet the needs of all parties.

To assess the effectiveness of the competency test, systematically gather data on test results (e.g. how well individuals are meeting the competencies) and overall performance to how the test is capturing the intended skills. The analysis may identify trends that indicate strengths or weaknesses in the test.

It is important to incorporate feedback and insights to improve the test's design, content, and administration processes into upcoming competency tests. This may include conducting frequent sessions to evaluate feedback from previous tests, analysing results for areas that need enhancement, and implementing changes to maintain high standards.

After the completion of the test, systematic and sufficient administrative procedures should be implemented to ensure the transparency and reliability of the assessment process while providing clear guidelines for result handling and record management. The following key steps should be included in the post-test administrative procedures.

After the test, the responses provided, such as audio recordings or other related data, should be collected and securely stored to prevent any loss. While the procedures for recording and storing information are similar for both manual and computer-supported methods, extra care is required to ensure accuracy and to avoid technical issues, such as incomplete or corrupted files, as these can affect the reliability of automated processing. This verification step ensures the integrity of the evaluation process, regardless of the method or tools used. Once verified, the collected data should be organized and prepared to enable smooth and efficient processing in either manual or automated workflows.

For consistency and alignment throughout the evaluation and review process a standardized framework for assessing each competency area should be used. A detailed rating scale is provided in Annex A which may be used as a reference level to assess competency against the four evaluation criteria elements. Further, Annex D provides practical examples of areas that may be used to assess the competency of VTS operators in the use of standard messaging structure and phrases.

The performance summary for each individual should include quantitative scores reflecting their proficiency level based on the evaluation criteria outlined in Annex A as well as qualitative feedback offering constructive comments on their strengths and areas for improvement directly linked to the competencies assessed. Providing clear and actionable feedback is essential for helping them to understand their performance, identify strengths and areas for improvement, and outline a path for future development.

The recording and retention of results should be a systematic and secure process to safeguard data integrity, confidentiality, and reliability of the data. All assessment results, including quantitative scores, qualitative feedback, and supporting materials such as audio recordings, should be documented as evidence to demonstrate the competencies attained.

Processes should be established to address the following key aspects:

• Defining information to be retained such as test scores and response records.
• Implementation of secure storage systems, whether digital or physical, to prevent unauthorized access or data breaches.
• Limitation of access to authorized personnel only, ensuring confidentiality.
• Establishment of retention periods that balance regulatory requirements and operational needs, with appropriate archiving or disposal of records after the retention period.

By ensuring these measures are in place, the assessment process not only maintains its transparency and reliability but also complies with data privacy standards. This approach ensures that information is managed responsibly and ethically throughout the entire process.

After the assessment, it is essential to communicate test results clearly and transparently. Effective delivery of results and feedback reinforces the transparency, and reliability of the process, while empowering individuals to understand their performance and identify opportunities for improvement.

• Notification of Results: Results should be delivered using secure methods to ensure confidentiality and prevent unauthorized access. Individuals should be informed of their results within a predefined timeframe to maintain transparency and trust in the process.
• Feedback Delivery: Feedback should summarize the individual's performance, highlighting key strengths and areas for improvement. It should include actionable, personalised recommendations to support clear development goals. Feedback should be constructive and focus on helping individuals build on their strengths while addressing any identified gaps.

Procedures should be in place to address situations where individuals raise concerns about their results or encounter technical issues during the assessment process. Timelines for submitting and resolving appeals should be defined to ensure a timely and objective resolution. Additionally, steps for re-evaluation or remedial measures should be outlined to provide individuals with a transparent and supportive process for addressing their concerns.

Regular analysis and evaluation of assessment data are essential to maintaining a consistent and reliable assessment process while ensuring alignment with evolving standards and best practices. Specific areas for improvement can be identified and addressed by examining scores, feedback, and performance records. Key aspects of this process include:

• Tracking and analysing an individuals' performance over time to monitor trends and evaluate the effectiveness of the process.
• Gathering feedback from both individuals and assessors to refine and update evaluation guidelines.
• Utilizing data analysis to enhance the transparency and reliability of the assessment process, with findings incorporated into the design of future assessments.

This approach ensures that the assessment process remains robust, transparent, and adaptable to changing needs.

The qualifications and experience needed to design and assess VTS communication competency tests should reflect the testing methods adopted, as outlined in Section 6.2 (e.g. traditional tests involving human interactions, computer-based testing, AI-driven evaluations, and simulation-based approaches), each of which may require specific areas of expertise.

Depending on the selected testing method, test development and assessment should be carried out by a qualified and experienced team that reflects the technical, linguistic, educational, and evaluative aspects of VTS communications. This complementary and multidisciplinary approach enhances the overall quality of the test. To support this objective, it is recommended that the team includes professionals with expertise in the following areas, whose collaboration can contribute to a more balanced and effective testing process:

• VTS operational expertise: Professionals with substantial experience as qualified VTS operators, VTS supervisors, or personnel in related VTS roles should be included to provide operational insights that enhance the relevance and effectiveness of the tests.
• Language and Linguistic Expertise: Professionals with academic training or practical experience in theoretical and applied linguistics, language teaching and assessment should be included to ensure the development of reliable, valid, and pedagogically sound language tests. It would also be beneficial for them to understand internationally recognized VHF communication protocols, such as IMO SMCP and IALA Recommendations.
• Educational Expertise: Professionals with experience in education, instructional design, or curriculum development should be involved to ensure that educational principles are effectively embedded throughout the test design and implementation process. These experts help to integrate learning outcomes by ensure that the assessment not only measures relevant skills but also supports test improvement by interpreting results, structuring feedback, and contribute to ongoing improvements. If the test is conducted online, they should have expertise on virtual educational aids, and the design of the test platform should be taken into account.
• Data Analysis and AI Expertise: Professionals focused on the systematic collection, management, and analysis of data generated through computer or AI-based systems should be included to ensure consistency and objectivity. Proficiency in these technologies allows them to evaluate performance data, detect patterns and biases, and support data-informed decision-making. Their work contributes directly to refining testing practices and upholding high standards of evaluation.

Annex A is a table setting out, for each element of the four evaluation criteria (Message Compilation, Message Delivery, Message Interpretation, Standard Phraseology), a descriptor for each of the six levels: Level 1 (Basic), Level 2 (Developing), Level 3 (Eligible), Level 4 (Operational), Level 5 (Proficient) and Level 6 (Expert).

These are some communication and interaction activities that VTS operators regularly complete:

This annex provides some example test questions to illustrate how VTS communication competencies may be assessed through practical, scenario-based tasks. The questions are structured around the evaluation framework (see Section 5) and are designed to reflect realistic VTS communication and interaction activities, which are aligned with the Common Communication Tasks of VTS Operators (see Annex B).

Depending on the specific competencies being assessed, different question type formats may be used, such as text, audio, images, and video. For more information on question types, refer to Section 6.5 – Question Types.

Task Specifications

• Communication Task: Responding to developing unsafe situations
• Question Type: Text-based (Message Delivery)
• Scenario Title: Broadcasting a navigational warning regarding a dangerous wreck

Time allocation: The student may have 45 seconds to review the test instructions and 45 seconds to deliver the response.

Test Instructions: You are required to deliver the Securite message to vessels in the VTS area. Review the information provided and deliver the message using standard message delivery practices.

Written Prompt on Screen:
Securite. Securite. Securite.
All stations. All stations. All stations.
This is Incheon VTS. Incheon VTS. Incheon VTS.
WARNING Dangerous wreck in position 37° 25' 52" North, 126° 34' 02" East.
All vessels in the vicinity are advised to navigate with caution.
Out.

Task Specifications

• Communication Task: Vessels entering VTS area
• Question Type: Text and Audio-based (Analysing and Responding)
• Scenario Title: Provide anchoring instructions to a vessel entering Busan VTS area

Time allocation: The student may have 30 seconds to review the test instructions and 30 seconds to deliver the response.

Test Instructions: You are a VTS operator in Busan. A vessel is entering the VTS area and you will need to provide them with anchorage information. You will hear a message from the vessel and be provided with reference information on key messages to be delivered. Construct your message using appropriate techniques, standard message structure, message marker(s), and phrases. Verify the vessel correctly interpreted the message.

Audio Message (Incoming Vessel's Message):
Busan VTS, this is container ship Halla Star.
I am entering Busan VTS area from the southeast.
What is my anchoring position?
Over.

Reference Information on Screen (Messages to be delivered by VTSO):

• Go to anchorage area B3
• Stay at least 5 cables away from other ships
• Listen on VHF Channel 14 for more information

Task Specifications

• Communication Task: Responding to developing unsafe situations
• Question Type: Text and Image-based (Warning message composition)
• Scenario Title: Instructing a vessel to navigate in the correct traffic lane

Time allocation: The student may have 30 seconds to review the test instructions and 30 seconds to deliver the response.

Test Instructions: You are a VTS operator in Busan. You will use the same image to answer two questions but deliver messages to different vessels — Question 3 to Container Ship Alpha and Question 4 to Ro-Ro Ship Bravo. Review the image and reference information and deliver your response to Container Ship Alpha and Ro-Ro Ship Bravo. Construct your message using appropriate techniques, standard message structure, message marker(s), and phrases. Verify the vessel correctly interpreted the message.

Reference Information on Screen for Question 3 (Messages to be delivered by VTSO):

• Ship name: Alpha
• Ship type: Container ship
• Activity: MV Alpha is currently navigating in an incorrect traffic lane (outbound).
• Required action: MV Alpha needs to change course to starboard and follow the inbound lane.

Reference Image on Screen for Question 3 and 4: [image]

Task Specifications for Question 4

• Communication Task: Responding to developing unsafe situations
• Question Type: Text and Image-based (Situation transmission and warning message composition)
• Scenario Title: Issue a collision warning to prevent a potential collision

Reference Information on Screen for Question 4 (Messages to be delivered by VTSO):

• Ship name: Bravo
• Ship type: Ro-Ro ship
• Activity: Head-on situation developing with MV Alpha, creating a potential risk of collision
• Distance from Ro-Ro Ship Bravo: 2.5 nautical miles
• Required action: MV Bravo should remain alert and take appropriate caution

Task Specifications

• Communication Task: Responding to developing unsafe situations
• Question Type: Text and Image-based (Warning message composition)
• Scenario Title: Delivering a warning message to a vessel regarding nearby towing operations

Time allocation: The student may have 45 seconds to review the test instructions and 30 seconds to deliver the response

Test Instructions: You are a VTS operator in Yeosu. Review the image and reference information and deliver your response to MV Pioneer. Construct your message using appropriate techniques, standard message structure, message marker(s), and phrases. Verify the vessel correctly interpreted the message.

Reference Information on Screen (Messages to be delivered by VTSO):

• Ship type: A tug-barge
• Ship name: Blue Horizon
• Activity: MV Blue Horizion is engaged in a towing operation
• Distance from MV Pioneer: 1 nautical mile
• Relative position: On MV Pioneer's starboard bow
• Required action: MV Pioneer should avoid getting too close to the towing operation. Use Channel 12 to reach the tug.

Reference Image on Screen: [image]

Task Specifications

• Communication Task: Emergency response
• Question Type: Audio-visual based questions (Situation summarization and transmission)
• Scenario Title: Broadcasting a safety message following a shipping incident

Time allocation: The student may have 60 seconds to watch the video, and 30 seconds for preparation, then 30 seconds to deliver their response

Test Instructions: You are a VTS operator in Port Selvana. You will first watch a short video showing a recent maritime accident that occurred inside the port. Based on the information presented in the video, you will be required to compose a message to be broadcast to nearby vessels in the VTS area.

Reference Information on Screen: [Video Script] A maritime accident occurred off the coast of Port Selvana earlier today when a container vessel collided with another ship underway, resulting in nine large containers falling into the sea. The drifting containers are obstructing the main fairway near the port breakwater. The incident happened during heavy fog, as the region enters its monsoon season. Poor visibility is believed to have played a major role in the collision. Due to the ongoing recovery operations, port authorities have temporarily suspended all vessel movements. Port Selvana authorities are currently working to clear the fairway and restore normal traffic flow.

The evaluation criteria for the competency testing of VTS communications is based on the framework outlined in IALA G1132, these include:

• Message Compilation
• Message Delivery
• Message Interpretation
• Standard Phraseology

For each evaluation criteria, the following examples may be used to assess the competency of VTS operators in the use of standard messaging structure and phrases. References to the relevant sections of IALA G1132 and IMO SMCP have been provided.

Further, these examples may also provide guidance to VTS operators on areas that may be assessed against during competency testing.