Test Strategy for Middleware and Firmware

We have defined middleware and firmware and understand that

they are different, yet have many characteristics in common

when it comes to testing. The discussion of test strategy

for these types of software will include both middleware

and firmware, and can be extended to test any software

which is not accessed by a user interface.

Early Testing

Early testing will multiply the testing effectiveness of

any software application, regardless of technology.

However, in the world of middleware and firmware early

testing is most critical because finding defects at later

stages carries a higher penalty of rework. This is due to

the extent of integration with hardware and other software.

The problem with early testing in this environment is that

with so many integration dependencies, how does someone

create test harnesses and stubs that allow for an accurate

test? Manually, the job is possible, but can be

overwhelming when there are many interfaces involved. If

you are developing in a language that has tool support for

structural test case design and testing, you may find that

the job can be very easy. Specifically, for C++ and Java,

Parasoft (www.parasoft.com) has a great toolset to design

and perform structural tests, with a feature to

automatically create a test harness and test stubs. Similar

tools are available from International Software Automation

(ISA) www.softwareautomation.com.

Developer Testing

Developer testing is essential to avoid high rework costs.

To the testers, the software is a black box. Only the

developers have the view and access to the code to test all

conditions. In addition, not only are functional cases at

stake, but also the structural tests for memory boundary

violations and memory leaks.

My experience is that developers can test software if the

have a good process to follow, standards to show what is

expected of them in terms of testing, and a way to hold

developers accountable for the quality of their work.

Management must also be making the message loud and clear

that testing is part of the job and that quality is a

shared responsibility between developers, testers, QA, and

management.

An Object-oriented View of Testing

In the object-oriented view of testing, tests are isolated

at a smaller scope, yet can have high complexity due to the

interfaces with other objects. The object-oriented view of

testing must be able to deal with classes, methods, and

attributes and to validate those at a high level of

coverage.

In Shel Siegel's book, "Object-Oriented Software Testing,"

he describes the Hierarchical approach to O-O testing.

"The hierarchical approach is at the heart of the object-

oriented testing system. This test approach uses and builds

upon several well-understood testing techniques, tying them

together into a comprehensive testing system. The

hierarchical approach leverages the fact that "everything

is a system." It defines and applies testing standards for

several levels of software component: objects, classes,

foundation components, and systems. The hierarchical

approach designates as SAFE those components that meet the

testing standards for that kind of component. Once you

designate a component as SAFE, you can integrate it with

other SAFE components to produce the next-level component.

In turn, you test this component to the level of safety

associated with the component level it represents. SAFE is

always a relative state. It depends entirely on the

standards you choose to enforce, your application, your

attitude toward risk, and the specific risks and risk

management practices you adopt in your project. The

hierarchical approach provides guidelines for minimum

safety; you decide what is right for you."