Approaches to Assess Customer Reliability Needs

Just asking a customer how reliable they want your product often provokes an honest answer. The customer, and you most likely, do not want any failures.

Failures are troublesome or in some cases dangerous.

You and your customers realize that not every unit produced will operate over a long and useful life. There is some chance that something will fail. The definition of ‘some’ is often vague.

How reliable is reliable enough?

If we want to meet or exceed our customer’s expectations, we need to know how reliable is good enough for our customers.

We need to understand the importance of reliability versus other priorities customers have for our product. And we need to set ’good enough’ reliability targets for a product.

So, how do you determine what the customers want? As you may suspect each individual customer may have a different set of priorities and different expectations. Some may want more features, others lower cost, some consider the lowest operating cost paramount. Unless you have only one customer, you are going to experience a range of expectations.

This really is an ongoing process. You initially start with a general idea or hunch for a reliability target, then refine as you learn more about your customer expectations. The early information shaping a reliability target is inaccurate and may span a wide range of values.

As you and your customers learn more about the product and as technical limitations become clear, you will then refine and narrow the reliability goals. The best customer information concerning reliability expectations is as the customers own and use the product.

If the reliability is not good enough you may experience higher than expected returns, and may not experience the anticipated growth of sales.

Even this direct expression of customer expectations is cloudy.

Product returns and lack of sales or the inverse is not solely governed by the reliability performance, it may include events by competitors, market shifts in needs, or change is customer priorities.

Brief Summary of a Range of Approaches

The best data is once the product is in the hands of the customer over the duration the customer expects the product to survive. You do have to listen and gather the data.

Customer complaints or compliments, social media forums, customer interactions via call centers, stores, or technician visits.

These all require some effort to gather and analyze, yet is often the most fruitful and honest insights concerning your customer’s perception of your product.

Repair vs Replace Decision Point

For major purchases which include repairable elements (building, factory equipment, home appliances, etc.) an indirect reflection on customer’s reliability expectation is the decision to repair or replace point in time. When did the customer perceive the cost of maintenance as being less attractive than buying a new unit?

If they buy a competitor’s product at this point, that is also a negative indicator.

The idea concerns the cost of ownership and the perceived value the customer has received.

If that decision point is reached after the customer considers is long enough (they have received sufficient value) then they are likely satisfied with the product’s reliability performance. If not, they are likely not satisfied.

This approach is reactive for the current product and informative for the next one.

Customer satisfaction surveys and purchase decision studies

These are used to gather data on the relative importance of reliability performance versus other purchase and customer satisfaction surveys.

These require some ongoing relationship with the customer in order to gather initial views and views over time. The surveys can take many forms, from the inclusion of the survey in the shipping containers (inside the box literature) or direct mail or via email.

Response rates are generally low and may require some incentive (product registration benefits such as access to unique information, streamlined support, gift cards, etc.). It does take skill to craft a meaningful survey and may only provide general insights.

The length of the survey impacts response and accuracy rates, thus the tendency is to shorter focused surveys, which tend to provide only general results.

Similar product and competitive analysis studies

In a market where competitors already have products available, purchase a unit or two for comparative life testing and teardown analysis.

Coupled with market share information, this approach may highlight elements of the best and worse performing products and may reflect the associated customer placed value on those features.

For example, if a keypad or touch screen interface for the market leading product is noticeably more robust than the competition, that team may have learned of the importance of the interface reliability, or happened to create a robust design that meets with customer approval.

Reverse engineering products provides insights to the solutions others have taken to meet customer expectations.

Market studies

Some organizations have a marketing department others rely on outside agencies, yet working closely with these teams to understand customer exceptions around reliability is one approach.

These studies may include

purchasing decision priorities,
rank ordering of feature importance,
identification of brand perception and ranking,
identification of brand promise,
effectiveness of advertising and
marketing campaigns

As a reliability profession, working with the marketing team may help garner an understanding with enough detail to be useful.

Delphi Study

A Delphi study involves surveying experts with knowledge of your customer’s reliability expectations concerning reliability.

The survey questions require a numerical answer, such as how long should product x operate without failure for customers? The responses are rank ordered and the lowest and highest responses receive a follow-up survey to ask about the information or rationale behind the response (maybe they have unique information the rest of the group does not poses).

Then the low and high rationales are sent to the entire group and everyone is asked to again to provide numerical responses. The process may require 3 to 5 cycles to narrow the range of responses to a group consensus.

This is a proactive process yet dependent on having a pool of willing participants for the multiple rounds of the survey.

Discussions directly with customers

Despite the customer’s common stated desire to not have any failures or perfect reliability, they are not willing to pay for it.

Engaging in a substantive discussion may reveal meaningful information concerning that person’s reliability expectations. The issue is gathering enough information to accurately represent the population of customers.

The interview process requires skill to avoid leading the respondents to desired responses. While a time consuming and expensive approach it may be an accurate proactive method.

Prototyping

The product development process often includes building prototype units. These units serve many functions including the ability to compare reliability performance to project goals.

The prototypes may also serve to gather internal customers (quality, marketing, sales, customer support, technicians — all groups within the organization) of the reliability performance information.

Alpha and Beta Testing

Alpha and Beta testing are done to gather a range of customer feedback on features and to some extent on early life failures.

These programs rarely run long enough to determine the long-term reliability performance impact on customers, yet it is possible to include requests for long term views of the product and customer expectations.

Alpha testing is primarily done with working prototypes with a select group of trusted customers. Best testing may be accomplished using the initial production runs and with a wider group of potential customers. Both require the customers to be willing participants trading early access to a product in exchange for their insights, suggestions, and issues.

One caution using this approach is the willing participants may only include early adopter which may not reflect the bulk of your customer.

Trial Market Studies

When time permits or the risk of not meeting reliability expectation is high, it may make sense to limit the initial release of the product to a subset of the potential market.

Consumer brands have select cities to test market new concepts that is a means to gather information prior to committing to a wider market launch. It is a way to limit the investment risk to launch.

This may or may not work for your product, yet does provide a means to allow customers to experience a product and to exercise your data collection system. It provides an actual customer buying and usage experience which surveys, and beta testing fails to replicate.

Quality Function Deployment

The ‘house of quality’ is a tool to translator customer stated requirements or needs into design requirements.

As with any of the approaches it may include a broad range of customer expectations beyond just reliability performance, yet does like other approaches provide a means to prioritize the customer stated requirements.

QFD is a process to analyze customer needs and engineering requirements in order to ensure all needs will be adequately met plus the ability to link specific specifications to customer needs. It is a useful tool to translate customer needs once those needs are gathered.

Summary

There are multiple approaches and you may need to employ more than one approach to build your understanding of your customer’s reliability expectations.

Talent and Professionalism as a Reliability Engineer (article)

Reliability Management & Risk (article)

Guiding Programs by Product Goals (article)