Reliable systems are necessary in cases where there is the potential
for serious loss of life or resources. Because of the ethical burden
placed upon designers and implementers of reliable systems, it becomes
necessary to use methods that are drastically different than those used
in traditional software systems. A substantial development effort is
necessary to produce reliable systems, regardless of whether the system
is being used for aerospace, military, or civilian purposes. In addition,
the consequences of failure in critical systems are so dire beyond the
ethical consequences that it forces companies to expend the necessary
effort to deliver such systems. The design of a reliable system is more
deliberate than the average off-the-shelf software title. The planning
stage typically begins months, even years, before the first line of
code is ever written. Once implementation begins, every design decision
and line of code is scrutinized. When the coding is complete, the work
is not even half done. The majority of time is spent debugging, testing,
and verifying the system.
Tools and methodologies used in building reliable systems
In discussing the reliability of critical systems, it is important
to note the distinction between building reliable subsystems and building
a complete system. Software engineering was originally seen as one of
the multitude of tasks in building a complete system. In complex systems,
there are typically many points of failure, where an error might be
catastrophic. Because software engineering is a relatively new field
compared to its engineering brethren, the focus often falls on software
portions of these systems when disaster strikes. A complete system is
only reliable as its weakest component, and the goal of building complex
systems from the software perspective is to ensure that software is
not that weakest link.
Among the best tools systems developers have at their disposal are
solid methodologies and techniques for developing robust, fault-tolerant
systems. These tools include a well-thought out set of design and specifications,
as well as formal verification of the system once it has been written.
In addition, the government maintains a set of stringent standards for
the regulation of the design of critical systems.
These tools are far from perfect, however, and require time and effort
from the systems developers to use effectively. In addition to realize
the benefits of these tools, developers need to understand exactly what
each tool is meant to accomplish. Formal verification is a tool that
is applied to code only after it has been written. It cannot guarantee
anything about the quality of system design. It can only ensure that
the code written meets the specifications for the project.
The work, therefore, falls in creating a high-quality specification.
The challenge of doing good design work lies in the inability of human
designers to completely and accurately predict all possible scenarios
under which the system will have to perform. If the designers of a system
fail to predict certain scenarios, then the formal verification tools
will have no way of detecting such a shortcoming. It is important that
designers and testers understand exactly the strengths and limitations
of such programs. They are meant to be used as tools to help improve
the quality of a program along with careful testing and good implementation,
rather than as the exclusive means by which to verify program correctness.
Military versus Commercial Sector Differences
One of the questions we had starting this project was systems were
developed in the military versus how such systems were developed in
business. In our research, we discovered several things: 1) That the
government tends to heavily regulate itself, and any critical area where
lives or critical resources are at stake. 2) That good software engineering
practices hold irregardless of what environment the development takes
place in. 3) The government tends to stay with a certain technology
that has proven to be stable in the past, even as the technology becomes
out-dated. 4) Competition and the desire for new and better features
in the commercial sector drive a more rapid product cycle.
Initially, we thought that the more rapid product phase used in the
commercial sector would cause reliability problems, given the massive
debugging and testing effort that typically goes into such systems,
but this turned out not to be the case, as military and commercial systems
both exhibited times when they suffered software reliability disasters.
Possibilities to explain this observation include: 1) the tremendous
amount of testing and debugging does pay off, and eliminates a large
number of bugs from these systems. 2) There are always esoteric bugs
that arise from complex system interactions that are impossible to simulate
in a laboratory environment, since simulations are them themselves programs
designed to test certain aspects of a system.
The ethical responsibilities of a team member working on a reliable
software project transcends military/commercial boundaries-it applies
to everybody. Every person has a responsibility to do their job to the
best of their ability (i.e. following good software engineering practices,
using great circumspection, peer review, etc.). When reliable systems
do fail, it may be politically and socially acceptable to blame a person,
or a portion of the team directly responsible for bug leading to the
failure. However, the team as a whole should shoulder the burden for
the failure, since in such systems there are many dependencies and relationships
between the people who write, test, and implement the system.