Here’s the Data

Ralph Evans was editor of the IEEE Transactions on Reliability from 1969 until 2004. He was a very good editor for my 1977 article, and he used me as a reviewer, because I was critical of BS and academic exercises. Ralph moved to University Retirement Community, Davis, CA. He died in 2013, https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6587564. I wish I’d known he lived nearby so I could have visited and argued with him.

Ralph’s editorials [1 and 2] pled, “Data, Data, Oh Where Art Thou Data?” He wrote, “Field-data are largely garbage. I believe they deserve all the negative thinking possible.” “True field-data are wonderful-much better than fancy equations. Unfortunately, they are very difficult to get. Thus data from the field are largely garbage because they do not represent what really happened.”

I beg to differ!

This was my letter to Ralph.

Generally accepted accounting principles (GAAP) require ships and returns counts [3]. Ships are sales, installations, etc. Returns can be any indications of failure to provide adequate service according to customers: complaints, failures, renewals, repairs, spares sales, ordinary returns, or no-trouble-found (NTF). Ships and returns counts are statistically sufficient to make nonparametric estimates of reliability and failure rate functions.

Accountants are required to report sales revenue and service costs [3]. Accountants get in trouble for not reporting revenue and costs accurately, so they’d better have accurate records of the ships and returns for reporting revenue and costs.

Nonparametric maximum likelihood and least squares estimates of field reliability and failure rate functions are available from ships and returns, whether failures are dead forever, renewed, or something in between. The Kaplan-Meier nonparametric estimator from grouped life data has less variance than nonparametric estimators from ships and returns counts. (Unless your life data is a sample!)

Not all returns are failures, but returns incur service and spares costs, even if NTF, complaints, or just plain returns. Reporting information estimated from ships and returns counts, however imperfectly identified, stimulates interest in classifying returns by cause.

There are more errors in life data than in ships and returns counts, because there are more opportunities to err. Tracking warranty returns by vehicle identification number (VIN) requires ~1000 times as many life data entries as ships and returns counts and incurs at least that many more errors, unless you sample life data; then you incur sample uncertainty.

Would you like to reduce complaints, warranty expiration anticipation returns, and NTF returns? Field reliability information could help satisfy those needs even though returns are not failures. Field reliability estimates raise questions as well as help solve problems. Questions often lead to better data.

Uncertainty, Errors, and Population Statistics

Statisticians teach us to use independent and identically distributed ages at failures, perhaps censored, to estimate survivor functions or test hypotheses. Such data are difficult to obtain, expensive, and contain errors. Major companies have quit collecting life data. Others collect some data, such as ages at machine failures and part names but not renewal counts. Others buy mainframes and database software to collect installed base, ages at failures, and survivors’ ages by serial number, and then they don’t use the data.

People say about Oracle data base software: “It’s easy to put data in but it’s hard to get information out.” At least three Oracle service databases record parts’ failures by name and product, but they don’t recognize that multiple parts with the same name may be used in the same product. Was the second failure due to the first part location failing twice or was it the same part name failing in another location?

Sequent Computer Systems and the IBM Way

The company where I worked in the 1990s had several mainframe Sequent hard disk drive failures, so I complained to Sequent and offered to help. No reply. Later I visited Jerry Ackaret, Sequent reliability engineer, because IBM bought Sequent, and the IBM Way no longer allowed Sequent collect life data by tracking service parts by serial number. I showed the Jerry how I estimate field reliability from ships and returns counts.

Sequent’s quality manager asked me, “Can you tell me which customer a failed part came from if we don’t track parts by serial number of product they’re in?” I replied, “Sorry, all we can do is estimate reliability and use it.” The quality manager walked out. Jerry later told me, “He wants to know which customer so he can suck up to the customer with the failed part.”

Why not Estimate and Use Field Reliability?

Estimates of reliability and failure rate functions life help justify design,  engineering, process, and service actions. Using population ships and returns counts eliminates sample uncertainty. Using nonparametric statistics eliminates unwarranted assumptions.

Cost and privacy concerns limit collection of ages at failures and survivors’ ages. If life data are not available, use ships and returns counts; they are statistically sufficient and they have no sample uncertainty. Field reliability estimates may justify collecting better data.

Send data to pstlarry@comcast.net or download and enter it into npmle.xlsx, https://sites.google.com/site/fieldreliability/home/files-workbooks-etc, for nonparametric estimates of field reliability and failure rate functions. Do the best you can with what you have.

References

[1] Evans, Ralph, “Oh Data, Data! Wherefore Art Thou Data?” IEEE Trans. on Rel., Vol. 40, No. 5, Dec. 1991, p. 497

[2] Evans, Ralph, “Data, Data, Oh Where Art Thou Data?” IEEE Trans. on Rel., Vol. 51, No. 3, Sept. 1992, p. 259

[3] Plank, Tom, Accounting Deskbook, 10th Edition, New York: Prentice-Hall, 1995.