Background and Issues
Safety-critical systems include transportation systems (e.g. planes,
automobiles), nuclear power plants, medical systems, chemical processing
plants, nuclear and conventional weapons, and many military systems,
just to name a few. All these systems have the potential to cause unintended
physical harm. Monitors and controls are necessary on these systems
to ensure energies do not inflict unwanted damage. Before the age of
computers, completely physical means (hardware) was used to provide
monitors and controls. Nowadays, there are instances of computer systems
being used in critical places in all the systems types mentioned above.
Indeed, the use of software has allowed technological advances in many
There are a number of issues that arise when using software in a critical
system as opposed to only hardware. Because of the flexibility of software,
designers tend to create feature-rich systems which makes for complex
systems, generally much more complex than a hardware solution would
be, though also more powerful. This complexity makes it difficult for
software to be written without any errors. For one, the requirements
for the software are complex and may not have been captured correctly
or may be ambiguous. Designing to such specifications will likely cause
errors, and the design process itself may introduce errors or oversights.
The coding phase can introduce more errors. Testing, including random
testing, can find many errors, but due to complexity, not all pieces
of the software will be exercised in all possible manners. Software
can be verified through formal mathematical techniques, but this is
generally very difficult and extremely time consuming (and there could
always be an error made in the verification process). Additional concerns,
such as the human-computer interface, can cause additional problems,
as some of the case studies show. For such reasons as these, creating
trustworthy software is a challenging task.
The commercial and military sectors have somewhat different approaches
to developing reliable systems, arising because of different needs.
A general assumption is that the military sector holds higher standards
for software testing and risk management because of the extreme nature
of many of their systems, such as the nuclear arsenal. On the other
hand, many of their designs must be very aggressive, such as a fighter
jet which must be superior to enemy fighter jets. Indeed, today's fighter
jets are physically unstable and unable to fly without computer control.
So, interesting questions arise. Is the military able to produce more
reliable systems? What are the differences in design approach between
military and commercial designs? In testing and verification? Where
do errors generally arise? What can we learn from these two sectors?
These are the types of question this web site addresses.