Increasing Verification Coverage for DO-254 Compliance through QA Testing

There are three aspects of verification that one needs to look at, to achieve DO-254 compliance: Analysis, Test and Review. For achieving compliance with minimum effort, one needs to formulate a requirement-based strategy using all the three aspects.

Analysis here implies computer-based simulation of hardware. Through simulation, a relevant virtual test set-up is created and the test is conducted with various wave forms as input to simulate a realistic environment. For DO-254 compliance, actual hardware tests are more important than simulation since simulation is just for representation and does not signify an actual test.

The Challenge: For DO-254 compliance, the requirements need to be traced from board level to the final pin of the FPGA along with proper verification of the results.

Testing the FPGA device at the board level provides very low FPGA input control making it difficult for analysts to inject certain signals for normal range and assess the overall robustness. An alternative way to test and verify the same is through simulation. But simulation tests are not ironclad enough to anomalies and errors, which may impact safety or reliability of the device under test. Verification of requirements through hardware test during final board testing is challenging and not at all feasible in most cases.

The solution: Increasing verification coverage by test

The following proposed process is instrumental in devising a requirement-based methodology which will leverage the same test cases and inputs for simulation and device testing to minimize on efforts and increase their effectiveness.

Requirement Gathering: While capturing requirements, ensure that they are broken down from the product level to card level and further to FPGA pin level with due verification. This will ensure that the inputs are controllable and the outputs are observable against each requirement right at the FPGA pin level.
Test Case Development: Test cases should be created based on requirements rather than the usual practice of developing test cases based on the design. Test cases usually define the way the requirements need to be verified. The initial set up condition; input condition; expected output along with fail and pass criterion need to be taken into consideration. To increase the effectiveness of the test cases, abnormal input conditions should be added and the results should be monitored for inputs beyond the normal range.
Test Bench Development: Hardware Descriptive Language (HDL) test benches need to be created to implement test cases for simulation. The inputs from the test cases can then be applied to HDL design and the outputs can be collected via waveforms. Self-assessing test benches can be very helpful to automate the overall process. Only care that needs to be taken is to have a qualified tool in place especially if an automated tool is being used.
Functional Simulation Code Coverage: Functional simulation and code coverage analysis can be achieved using test bench. The test vectors can be used for functional simulation, post-layout timing simulation and for actual device testing.
Timing Simulation: The test vectors can be used for defining MIN/MAX and Typical range for post layout timing simulation.
Device testing for DO-254: The target device needs to be tested in isolation with test vectors as test inputs. It is very important to create a hardware test setup which provides 100% input controllability and output observability. This is necessary to ensure that the same test cases used for simulation can also be used for hardware tests. The result of actual hardware test can be captured and verified against functional simulation results.
Final Board Testing: Once the target device is tested in isolation, the final board testing is done. The best way to go for board testing is to use the flight configuration and verify board level functionalities, component interfaces and environmental requirements.