The Conscience of Acquisition
By SQNLDR D J Hunt (OT&E Staff Officer)
As the Royal New Zealand Air Force prepares to receive upgrades or replacements to our operational fleet, testing and evaluation (T&E) has become a hot topic.
FLTLT Barnes and SGT Jefferies
The various levels of testing and evaluation—from how the contractor meets the contract requirements, to how we fly and support the aircraft—are critical to ensuring that the RNZAF has the capability required to deliver our outputs. Operational Testing and Evaluation (OT&E) is how we in the RNZAF assess the systems suitability and effectiveness to deliver the required military capability.
Testing and Evaluation
Testing and evaluation (T&E) phases are integrated into all of the major projects currently underway. Each testing phase has a distinct focus:
Phase One
—Development Testing and Evaluation (DT&E)
DT&E is the first level of T&E. It’s where the contractor develops the system to meet the user’s requirements, as laid down in the contract between the Ministry of Defence (MoD) and the contractor. This development focuses on the aircraft but can also involve publications, technical training packages, and support facilities such as flight deck trainers and maintenance test rigs.
Phase Two
—Acceptance Test and Evaluation (AT&E)
The aim of AT&E is for the contractor to prove to MoD that they have developed a system that meets the contract requirements. The RNZAF is often involved at this stage, providing personnel as operators, supervisors and advisers. The final step within this stage is for the system to be offered to the RNZAF for acceptance.
Phase Three
—Operational Testing and Evaluation (OT&E)
Engine being delivered
OT&E is the final aspect of T&E. It’s where the RNZAF determines whether the whole system is suitable and effective for operational employment. Up to this point the T&E has focussed on contractual compliance and specifications—OT&E focuses on placing the system into an operationally representative environment, and assessing the system as it completes an operational mission with representative aircrew and maintenance personnel.
Independent of the MoD project team, the contractor, and the Introduction Into Service team, OT&E is where we assess whether the money spent has delivered the required capability. This independence is key—providing integrity to the process and allowing unbiased assessment and reporting.
A-109 cockpit
OT&E within the RNZAF is conducted by the Directorate of Systems Evaluation (DSE). Located within No 485 Wing, DSE provides advice and documentation to both the Operating and Technical Airworthiness Authorities. This assessment focuses on the ability of the whole system to achieve directed outputs, through testing and evaluating integrated operating and logistic requirements.
The RNZAF is building its knowledge and experience around OT&E. The principles that we currently use are the same as those used by our allies—Australia, Canada, and Singapore—and US military and civil organisations (such as US Customs and Border Patrol), ensuring that the RNZAF is aligned with current international OT&E standards. The RNZAF Manual of Operational Testing & Evaluation (NZAP 102) has also been recently published.
The B757 flight deck
Training for OT&E practitioners is completed through a number of private institutions. A 3-5 day basic course provided by NOVA, an Australian based company, introduces participants to the basics of OT&E and focuses on the theory of OT&E. The US National Test Pilot School (NTPS) is a three-week course providing in-depth training on how to produce OT&E test plans, supported by practical exercises both on the ground and in the air.
Currently the RNZAF has five personnel who have completed the NTPS OT&E course. DSE is in the process of developing a course for those additional personnel such as Test Supervisors, who will be involved with OT&E.
In addition, a module for the Engineering Officers Technology Course is being developed and will be taught at RNZAF Base Woodbourne.
Aircraft check
OT&E is the final piece in the introduction of a new capability and is the final test to confirm a capability has been delivered. It provides an assurance to both RNZAF Command and the New Zealand Government that they have got value for money in the upgraded or newly procured military capability.
This is achieved by ensuring the system is ready and able to begin delivering outputs as well as identifying any enhancing features or deficiencies.
Ultimately, OT&E evaluates the Airworthiness of a system, enabling RNZAF Command to make informed decisions.