The System Effectiveness Concept

Reliability is not the only concern when building a system.

Let’s consider a passenger car. Reliability refers to how often the car in the shop. How often we need to perform preventative or corrective maintenance. How often it fails.

Reliability does not include how long it will be unavailable for use. How long it is in the shop waiting for parts, for example. We want high availability, short repair times, too.

Plus, we also want the vehicle to be capable of doing the wide range of things we may want to do with it. Like, carry us safely to the grocery store and back. And, have the capacity to carry the groceries purchases without leaving behind a passenger or two.

There are other considerations, like cost, color, and style. These other factors may play a role in the capability of the system, or not, that would be locally defined for your project to develop the system.

The elements of reliability, availability, and capability capture the essential concepts that we need to create an effective system, like a passenger car.

Early System Effectiveness Discussions

Much of the early development of reliability engineering tools and techniques was for large military systems. In the 1950’s and ’60’s, various departments and organizations published standards, reports, handbooks, etc.

Many of the tools we use today, parts count predictions and physics of failure modeling became widely published.

Handbook Reliability Engineering, 1964

The US Naval first defined system effectiveness, as far as I know, in the Handbook Reliability Engineering, published by the direction of the Chief of the Bureau of Naval Weapons on 1 June 1964 (NAVWEPS 00-65-502). In the document system effectiveness is defined in section 1-3-1. System “Operational” Effectiveness of the document, excerpted here in its entirety.

The “worth” of a particular system or piece of equipment is determined primarily by the effectiveness with within it does its job — its “operational” effectiveness. An acceptable level of effectiveness is required of every system destined for tactical use.

Emphasis on reliability alone does not necessarily produce the required level of effectiveness. Other factors must be considered simultaneously. They are shown in Figure 1-5.

Each of these characteristics can be expressed as a “probability” of successful fulfillment of requirements, defined as follows:

Performance capability is the probability that the system¹ will satisfy mission performance requirements when operating within specified design limits — a measure of “how well” it does its job when working properly.

Operational reliability is the probability that the system will maintain a specified level of performance throughout a given mission — a measure of “how long” it is capable of working without failure.

Tactical availability, or operational readiness, is the probability that at any point in the time the system will be ready to operate at a specified level of performance, on demand — a measure of “how often” the system is ready when needed.

Operational effectiveness is the product of these three characteristics, i.e.:

Effectiveness = Performance × Reliability × Availability

Operation effectiveness of an equipment or system is then the probability that it can successfully meet an operational requirement, for the duration of the need, when the need arises.

1. The words “equipment” and “system can be used interchangeably.

— End except

The entire document focuses on system effectiveness and expands on each of the three characteristics.

Weapon System Effectiveness Industry Advisory Committee (WSEIAC) 1965

In the committee’s final report (AFSC-TR-65-6) dated January 1965 the authors again stressed system effectiveness as being essential and being more than just reliability. Here is an excerpt from pages 8 and 9.

A. Principal Factors of Effectiveness

The WSEIAC has taken the position that system effectiveness is a quantitative measure of the extent to which a system may be expected to achieve a set of specific mission requirements. It is expressed as a function of three major system attributes:

• Availability (A)
• Dependability (D)
• Capability (C)

 

Availability (A) is a measure of the condition of the system at the start of a mission when the mission is called for at an unknown (random) point in time.

Dependability (D) is a measure of the system condition during the performance of the mission: given its condition (availability) at the start of the mission.

Capability (C) is a measure of the results of the mission; given the condition of the system during the mission (dependability).

— End of excerpt

These concepts are not new and maybe have been forgotten. Take a look at your reliability program and how well do you support system effectiveness?