Embedded Software Quality How Do Vendors of Software Debugging and Test Tools Test Their Products?

For safety-relevant applications, tool manufacturers have to subject their software tools to rigorous testing. The experience gained in this process can be for the benefit of customers as well.

Anbieter zum Thema

MicroConsult Academy GmbH

ISH Ingenieursozietät GmbH

Apacer Technology B.V.

Chapter 8-11 of the ISO 26262 standard, a version of IEC 61508 specifically adapted to automotive, is dedicated to confidence in the use of software tools. This standard has a specific impact not only on vehicle manufacturers and their suppliers but also on the development and test processes of tool manufacturers and thus the development process of the actual software tool.

This article depicts the organizational and technical measures iSYSTEM has derived from current functional safety standards to support automotive software development departments in the preparatory stage and especially during the so-called tool qualification phase. Experience from earlier tool qualifications in aviation (DO-178B) have been incorporated in this process.

Ergänzendes zum Thema

Interview: „Integration of test tools into the testing process has to be easy“

What challenges to development and test engineers does the tool suite fitIDEA overcome?

As a start, fitIDEA is the internal implementation of iSYSTEM’s software development and test process. Our focus is mainly on agile development and an internalized process of continuous integration. We have therefore approached the issue test automation systematically. fitIDEA is moreover an environment to prove the correctness of the iSYSTEM tool functionality according to certain use cases, mainly customer-specific ones. This is a requirement of several functional safety standards.

What are the characteristics of an efficient test tool?

This is hard to generalize. After all, there is a wide variety of tools that cover many different techniques – from modeling tools to static code analysis, unit/integration test tools and, last but not least, system test tools. Most of them are a combination of software and hardware. For sure, a key aspect is that the tool is easy to use and can be integrated in the process easily. This creates acceptance among development and test engineers as well as managers.

How can a test technique master the complexity of modern software applications and the variety of devices?

The answer is fitIDEA and internalized test automation in an agile software development environment – in other words, automation and sophisticated techniques. And an occasional glimpse at the aviation sector with its extensive experience in model based/requirements based testing.

There are diverse approaches and philosophies regarding software testing. Which one do you think is most interesting?

In a way, they are all interesting. However, the methodology and the company have to match up. At this time, I think agile software development and the related testing philosophy are quite fascinating. Especially because many habitual processes and patterns are breaking up, and any type of software challenge is overcome with both determination and professionalism.

Some experts expect static code analysis tools to become more significant in future. Will this diminish the significance of testing?

Of course. The significance and deployment of such tools has increased substantially over the past decade and is just gaining momentum. Basically, such tools are easy to integrate in existing development and test processes without major changes to the actual process. Static code analysis tools cover part of the test process and are a useful addition to increased test depth.

The measures discussed have a sustainable impact on the development and test process of a software tool manufacturer to the effect that processes are reassessed and optimized, thus further enhancing the outward transparency of these processes. This degree of transparency is accomplished most efficiently based on test automation and is not only for the benefit of the automotive industry but for the benefit of all customers.

The starting point for high-quality software and hardware are internal process definitions that precisely describe how these products have to be developed and tested. Test of Embedded Systems means test of hardware and software at the same time.

Optical test systems and boundary scan tests have been used in the hardware sector for many years, and so-called dynamic tests have been added. These advanced tests verify different types of memory (Flash, RAM, …), memory access time and peripherals in test mode. The required speed is achieved by using the microcontroller on the test system. iSYSTEM also share their know-how with partner companies in this context.

Software testing focuses on the generation of new software, archiving of old and testing of specific software versions as well as an operational version control system. Continuous new and further developed microcontroller architectures as well as extended functionality of an embedded software application cause an increase in software complexity. Consequently, test automation is required. Another motivation for test automation is the demand for intense testing in the scope of functional safety standards. Tool manufacturers thus create a foundation of trust in their products. This article is dedicated to the topic of test automation.

(ID:38231760)