An Overview

Aerospace Software Verification is a crucial phase of airborne software. The amount of rigorous testing involved following DO-178 guidelines makes it a tedious and cumbersome process. Level E, for example, is the least stringent level in DO-178C certification and requires minimal documentation and process rigor compared to higher levels.

Aerospace industries are investing time, effort and resources into making sure the final verification artifacts are up to the mark and meet the desired certification criteria. However, there are certain measures that can be taken to improve the quality of the final artifacts, which can also help reduce the vast amounts of unnecessary rework. Planning for software aspects is essential to ensure compliance, and updating standards and practices is necessary to address issues in safety-critical software. Additionally, adopting new technologies can enhance verification processes and improve overall quality.

Firstly, let us understand the main intent of verification, especially in the aerospace industry. Like any other software industry, the main goal of verification is to find software bugs and get them fixed so that the final software being deployed is error free. This becomes more crucial while dealing with software in airborne systems specifically in the safety critical systems like cockpit display, flight management, or control systems. Failure conditions resulting from software failures can have significant effects on the aircraft, crew, and passengers, making rigorous verification essential. One example of a safety-critical system is aircraft controls, where software reliability is paramount.

As a verification engineer (or even the managers!), one must understand that it is not about how many test artifacts you generate or how many CRs you complete. The import is how many development issues are identified and avoiding such faulty software being a part of a flying aircraft. The count is not to be chased, but rather the quality that is to be strived for! In a real project, these activities must be demonstrated to meet certification objectives, with actual activities and concrete activities clearly defined, documented, and performed to satisfy DO-178C standards. The verification process must also ensure that software requirements are thoroughly verified for completeness and correctness. The importance of actionable test data and metrics cannot be overstated, as they support compliance and informed decision-making throughout the verification lifecycle.

Understanding Safety-Critical Software

In the aerospace industry, safety-critical software forms the backbone of modern airborne systems, from flight control systems to cockpit displays and navigation equipment. This type of software is responsible for ensuring that aircraft operate safely and reliably under all conditions, directly impacting the safety of passengers, crew, and the aircraft itself. Any failure or malfunction in safety-critical software can lead to serious or fatal injuries, passenger discomfort, or even catastrophic accidents, making its quality and reliability non-negotiable.

The development and verification of safety-critical software are governed by stringent regulatory frameworks, such as FAA regulations and standards set by certification authorities like the Federal Aviation Administration and Transport Canada. These regulations require that every aspect of the software development process—from initial requirements to final integration testing—meets the highest standards of quality, traceability, and compliance. Equipment certification is not just a formality; it is a rigorous process that demands comprehensive documentation, supporting information, and evidence that the software performs its intended function under all foreseeable conditions.

Unique software considerations come into play when developing safety-critical systems. Requirements-based testing, code coverage analysis, and the use of formal methods supplement traditional testing approaches to ensure that every requirement is fully validated and verified. Model-based development and object-oriented technology are increasingly used to manage complexity and improve the reliability of software design, while predictive analytics and artificial intelligence are being explored to enhance testing and validation processes. Security testing is also a growing concern, as the integrity and reliability of airborne systems must be protected against both accidental failures and intentional threats.

Achieving the necessary safety margin in aerospace software is a joint effort that involves software developers, quality teams, certification authorities, and the entire supply chain. The process is supported by a core standard of practices, including thorough documentation, exit criteria for each development phase, and continuous quality checks.

Achieving the necessary safety margin in aerospace software is a joint effort that involves software developers, quality teams, certification authorities, and the entire supply chain. The process is supported by a core standard of practices, including thorough documentation, exit criteria for each development phase, and continuous quality checks. By addressing issues early and maintaining a focus on safety-related considerations throughout the software lifecycle, the aerospace industry can reduce rework, improve performance, and ensure that every system operates at the same level of reliability required to protect lives and maintain public trust.

The Right Fit for the Right Job

“If we fail to plan, we plan to fail!”

Choosing the right team members for performing the verification activity is a particularly important part of the planning process. In today’s competitive environment, companies often deploy unskilled engineers to perform aerospace verification, irrespective of their experience, skill sets, or areas of interest. This leads to tremendous overheads later and generates a huge amount of unnecessary rework.

Especially when it is about verifying higher safety critical software like DAL-A or DAL-B, having the right people in a job can help reduce a significant amount of rework. The criteria for choosing the suitable candidates should be:

• Experience working in the same or similar domain.
• If they have no prior experience, then despite the amount of experience in other domains, candidates must be evaluated for their analytical skills, academic record, and enthusiasm to work on the project.
• Candidates need to undergo the required on-the-job training, the duration of which could be set based on the project timeline. Team managers and leaders need to make sure that the ramp-up period is made an integral part of the project timeline.
• SMEs should be assigned to the candidates or the team being ramped up so that their queries are resolved in time.

Clarity of Requirements

“Bring 6 eggs, and if there is bread then bring 8!”

This might not directly be part of the verification process, but it is an essential objective of the DO178 verification guidelines that the requirements being verified should be precise, clear, and verifiable.

Vague and unclear requirements create confusion between the development and verification teams. Development teams need to take the necessary measures to make sure the requirements are easy to understand and result in one and only one meaning. There should not be multiple interpretations possible of a single requirement. These steps might include:

• Rigorous brainstorming and reviewing requirements before they freeze for verification.
• Engineers who have experience in verification could be involved in the requirement writing process so that they can provide their feedback from a verification perspective.
• AI-based automated tools could be used to interpret the requirements to make sure the inputs and outputs can be easily extracted from them.
• Common templates should be followed at the feature level to achieve consistency. During the planning phase, the requirement templates can be designed and distributed among the individual feature owners so that the requirements can be framed with utmost consistency.

Well-defined Processes and Streamlined Flow-down

“A gap in communication is a slap on execution!”

This is the most crucial part of the planning process. Best practices, standards, and processes need to be streamlined and passed down to all the applicable stakeholders before the execution starts.

Whenever there is a change in an ongoing process, it must be formally communicated and brainstormed among the team members.

This is the challenging part about working with a larger team and avoiding conflicts; regular refresher meetings can be called to make everyone aware of a process update. For smooth communication, the following steps could be taken:

• Conduct brainstorming sessions between process authorities, SMEs, quality team, and team-leads to make sure the defined processes are correct and clear.
• Formal communication of processes through email or web-based open forums which the team members can refer to during the project execution.
• Regular sync-up meetings between the process authorities and team-leads, especially when there is a process update or clarity required on any process.
• Regular team meetings for individual teams to make sure that all are on the same page.
• As a part of the recreational activities, a process quiz could be arranged.

Due Time for Implementation and Reviews

“Good things take time, and we want to make the world a better place!”

Rushing to finish a job always results in poor quality and unnecessary rework. The schedule needs to be planned in a way that there is sufficient time allocated for various phases of the verification like feature and requirement understanding, unit testing, test cases and procedures implementation, discussing and reporting development issues, coverage analysis, etc.

• Either a self-review or an informal review is always desirable before the verification artifacts are sent for a formal review. An informal review checklist can be used to rectify the quick technical or process related issues beforehand.
• While planning the schedule, apart from considering the number of requirements, the complexity of requirements also needs to be considered. The complexity of requirements is an important driving factor for estimating test implementation and review efforts.
• The review phase is even more important than that of the implementation. The DO178 guidelines have the objective of “independence” for the same reason. The reviewer might have a different perspective, or the implementer might have missed things during the implementation or the self-review. Reviewing the source code is crucial to ensure compliance with industry standards and system safety. The time required for a review is usually considered to be about 10-20% of that required for the implementation. This may vary based on the complexity, the number of artifacts involved, and of course the quality of implementation in the first place!
• 10% of the total review phase could be allocated for quality checks or oversight reviews as required.

System Understanding

“Dive as deep as you can, the treasure is usually at the bottom!”

Having a system understanding plays a vital role in improving the efficiency and quality of final verification artifacts. Of course, not everyone can have the system knowledge; they may have a brief overview of the end-product and its functionality which may help the verification engineers while debating over the development issues.

System knowledge is gained eventually with time, so verification engineers should always strive to understand the module and features first. This will help them have a better understanding of requirements, which will eventually lead to producing better test artifacts. For a better system understanding the following steps could be performed:

• Ensure that the ramp-up or training modules are designed to provide a brief overview of the system or the feature being worked upon.
• Verification engineers should have access to the system requirements, design documents, and all the required training materials for a better understanding of the features and their internal technical details.
• Engineers need to go through all the required documentation before starting to verify the requirements.
• A brainstorm session between the test implementer and the main reviewer can be helpful to ensure clarity and consensus. This discussion should only be focused on understanding the system/feature and not on the test implementation to make sure the independence is maintained between the implementer and reviewers.

Usage of Tools and Scripts

“I may be dumb, but I know smarter ways of getting things done!”

Today, the aggressive use of AI tools and scripts has completely changed the mode of working of every sector of technology. Aerospace is no exception where the rigorous manual steps are involved in the generation of test artifacts; companies are looking for an AI approach to smoothen and accelerate the processes. This not only reduces human errors but also increases efficiency. However, as per the DO-330, to claim the final output of such tools or scripts, these also need to be certified; otherwise, the generated artifacts still need to be reviewed manually.

However, even if the tools are not certified, they could still be used for improving the efficiency of test implementation. Although here one needs to be cautious with the tool generated artifact and make sure that they go through the formal review process like the manually generated ones.

• The verification team should be encouraged to use the available tools for generating test artifacts, and the team also needs to be trained on how to use the tools.
• If the tool is not qualified, the process needs to be defined to make sure the tool generated artifacts are formally reviewed.
• In case of unqualified tools, regular updates in the tool are required based on feedback from the verification team. This helps to generate more precise and concise output.

Quality Checks

“Safety is directly proportional to the quality of product!”

Regular quality checks are as important as the actual implementation and review phases. A team of designated quality auditors, qualified reviewers, and SMEs can be formed to sample the completed verification artifacts and perform the required quality checks. Safety critical requirements must go through this phase to ensure that the verification of requirements is up to the mark, and there is no loophole in the requirements or their implementation.

The following steps should be followed to ensure that the required quality checks are done:

• Regular audit meetings must be called for verification samples.
• The implementers and reviewers can also be invited to the meetings if required.
• Feedback from the quality meetings is to be provided to the verification team.
• If there is a significant process or technical information update, then the information should be percolated to all the team members from the designated quality representative.

Conclusion

Although the points discussed above may sound obvious, the main intent of authoring this paper is to consolidate all these aspects.

Aerospace verification cannot be left to chance. Quality is not just about merely following processes, but it should be a pervasive attitude. When the engineers realize that the software they are processing is extremely critical and human safety is associated with it, the attitude to deliver the quality output nourishes naturally.

From a business perspective, this is also a massive effort and has a very high monetary value associated. So, any potential rework must be avoided to prevent wasting time and money.