Skip navigation
All Places > Keysight Blogs > Insights Unlocked > Blog > Author: Brad Doerr

Insights Unlocked

2 Posts authored by: Brad Doerr Employee

If I close my eyes, lean back in my chair and take a breath, I can almost hear the echoes of kids’ voices. Tri-fold cardboard displays and colorful ribbons are splayed across rows of folding tables. I’m at my 4th grade science fair and I’ve just won first place for my solar-powered hot dog cooker.

Solar Energy School ProjectA lot of things have changed since then, but the scientific method used for my science project still stands: make an observation, form a hypothesis, conduct an experiment, capture and analyze the data and draw conclusions. Sounds simple enough, right?


In my previous blog, I talk about not being afraid to take your first step in data analytics (DA) for testing and measurement (T&M) and I outline tips on getting started. But then what?

If you want that coveted blue ribbon, read on.





Our Ambitions May Be Tripping Us Up

From where I sit in data analytics for T&M, I often see my customers – brilliant scientists, engineers, architects, project managers – get hung up towards the end of the scientific method process. After observing, hypothesizing and conducting the test, they struggle with managing and analyzing the data and drawing confident conclusions.


It’s easy to understand why: we’re all doing everything we can to get to market fast – to get to market first – that we lose the rigor we once had when we started engineering.


Some steps are taken to collect and analyze data, but they are misguided. Engineers use complex Excel spreadsheets, pivot tables, custom MS Access databases and other home-grown tools. While these tools appear good on the surface, they ultimately don’t deliver. Not only do they come up short but the tools can only be used once, and usually poorly.


Despite the best intentions, most don’t have the resources, time or expertise to deal with data or data analysis in an effective way. And it’s not just about dealing with data from one step in the product lifecycle. DA for T&M happens throughout the lifecycle:

Design -> Validation and characterization -> Release to manufacturing -> Full-scale production

So how do you find the right tools to help with your messy data?


More Data, More Confidence

It’s a basic concept: by making accurate measurements (and lots of them), the more confident you’ll be in your conclusion.


I wrote in my previous blog about how you must have your Design of Experiments (DOE) planned before you do anything. What questions are you trying to answer?


If you build a car, you’re not just going to ask, “Does it turn on?” You’ll want to know, what is its maximum speed? How will we measure speed? What kind of gas mileage does it get? How will we measure gas mileage? Will people survive in a fender bender? How many measurements do we need to ensure confidence in our eventual conclusions?


Your DOE can’t be set in stone. It needs to be fluid and you need a tool that allows you to change your DOE on the fly and not spend days – or even weeks – going back and forth with your own personal (and expensive!) IT database architecture team to get things just right. And you certainly don’t want your Team Lead spending weeks on end buried in her cubicle designing a system that only she understands or that blows up with one little change in your DOE plan.


Figure out what questions you have (and how you’ll get answers), and understand you’ll probably have more down the road.


Get Your Hands Off the Data

Imagine: you walk into a bakery and it smells of fresh cinnamon rolls. The baker rushes out to help you. Mouth watering, you point through the glass case at the squishy, doughy roll. The baker takes your $2 with his flour-dusted hands and returns a grimy quarter. He hands you your roll with his bare hands and rushes back to continue his work.

Is something wrong with this picture? Yes! The baker should hire a clerk – let’s call her a retail expert – to focus on helping customers. And the baker should focus on what he does best: bake.


It’s not much different in our world. If your designer is responsible for building cars, she shouldn’t be graphing data from her T&M. She needs to focus on what she does best and not waste valuable time formatting and processing data.


Keep your experts doing what they do expertly.


Make a Decision, but Make It with Confidence!

Analyzing big dataEventually, you need to make a decision about your product.


Is it time to build a set of prototypes? 


Can you launch to market? 


What will be your production yield? 


You can’t make a decision with confidence unless you have good data and insight from this data.


There are hundreds of information sources to help you determine how to achieve 99% confidence based on measured data. Online articles, data entry tools and tables help aggregate data and determine how much you need to get to that desired level of confidence. But first...

  1. Figure out the questions you want answered (remember your DOE!) before you begin collecting and analyzing data, and...
  2. Don’t overlook your testing environment. Would my solar-powered hot dog cooker from 4th grade perform the same on a cloudy and sunny day? Of course not. Your results won’t align if you’re not considering what’s going on in your testing environment.

Once you have your questions figured out and your testing environment set, it’s time to find your data analytics tool. 


A good tool will easily adapt with you. 

A poor tool will cause headaches, delays in your design and, even worse, cost a lot of money.


Product Lifecycles are Global and Your Data Should Be, Too

You have a globally distributed team, so your data analytics tool can’t be located on Some Guy’s laptop. And your data can’t be understood/analyzed/charted by just Some Guy who might be spending half of his time on LinkedIn looking for that next big engineering gig, when he should be designing driverless cars. It’s even worse if Some Guy is your top designer and he’s spending 50% of his time wrangling data (possibly poorly).


Data and decisions need to be communicated across your distributed organization throughout the product’s lifecycle. You need a team collaboration tool that is understood by multiple people, across your organization, and that extends throughout the entire lifecycle.


Here’s a real-life scenario that might make you sweat. Susan, a VP of validation engineering, wants to fabricate a prototype of an important new design. Tom, her top engineer, has been managing the vast amount of test data and analysis with his own system. She asks him if they’re ready to fabricate the design as he’s the only person on this highly skilled team who has the ability to analyze the data, and he says he sees no concerns. Long story short? Tom is an expert engineer, but not an expert at all facets of the design – and the others on the team were not involved in the analysis. This mistake cost the company $1 million and three months as they built something that didn’t fully work.


Imagine if all your company’s presentations had to be created using Adobe Photoshop. The hardship this would cause in terms of time, productivity, and expertise! Your data analysis tool needs to be as intuitive and easy to use as PowerPoint is when putting together a presentation.


Don’t Reinvent the Wheel

If you have butterflies in your gut when talking to your manager about a conclusion you’ve come to, it’s time to look into your data analytics tool. Explore what’s out there and test it with a small project. Report back and let me know how it goes.


In my next blog, we’ll dive a little deeper into decision-making with data analytics and figure out how to move from validation to manufacturing.

It’s safe to say the application of data analytics to test & measurement (T&M) is lagging behind what’s happening in the financial, insurance and retail industries. However, as an optimist, I view this situation as a greenfield opportunity for you and for us. The ultimate benefits are making better decisions in less time, delivering the right product, and getting to market faster.


This is also a good time to pause and define what you and your organization aspire to in the use of data analytics (DA). Your goal may be as bold and transformational as overhauling your product lifecycle (albeit gradually). Or it may be as basic and personal as achieving tighter control over your projects—and, in all honesty, this is what I aspire to.


Addressing the fear

Another confession: While I have a deep appreciation for the benefits of DA, I dislike the term. The reason: “data analytics” can feel so overwhelming that it produces pangs of fear among many of my colleagues and some of our customers.


To narrow the scope, we can reframe it as “data analytics for test and measurement” (DA for T&M). And we can think small: you can use DA for T&M as a way to validate your design intent. Here, the goal is to gather the right data in the right amount to say, with confidence, “Yes, this product delivers on our design goals and we can build it reliably.”


Determining your team’s maturity level

Before diving in, it’s useful to step back and assess your team’s maturity level with DA. In this context, I see four levels: reflect, predict, prepare, and influence.


  • Reflect is a look backward at what has happened in the past, aiming to describe and understand the root causes of recurring problems and, if you’ve been collecting data, investigate any troublesome trends.
  • Predict builds on reflect by creating a working theory and then outlining scenarios for what is likely to happen internally and externally.
  • Prepare is the first step toward an active attitude: given a prediction of what is likely to happen, the organization takes steps to be ready for those situations.
  • Influence is a deeper engagement with the active attitude: if predict and prepare are pointing toward unfavorable scenarios, the organization takes steps to nudge the variables toward a more beneficial outcome. This is when we start aspiring to greater levels of control.


Taking your first steps

From my conversations with customers, most are like us: somewhere between “Where do we start?” and “How do we move beyond reflect?”


It isn’t about moving mountains, at least not at first. Start with the end in mind by defining the question you’re trying to answer. Then, carefully design the “experiments” you want to conduct and make a plan based on the type and amount of data you need to measure.


Next, be sure you have the tools to make the necessary measurements (actual values). Ideally, you will also be able to capture design simulations (predicted values) prior to building anything. Even if you can address only the validation phase—between design and manufacturing—this is still a worthwhile activity.


As a practical tactic, apply some up-front organization: Where should everyone store their data? What format should they use? Who will manage it? What common tool will they use for analysis?


Then get started by making measurements and collecting data from a batch of prototypes or pilot-run units, tested across a sufficiently broad range of properties and operating conditions (yes, this could yield several thousand data points). As you (and your team) are collecting data, sift through the initial raw data, apply “sanity checks” and, if everything looks good, start analyzing. This works best if your team has a common, shared tool that gives everyone access, enabling them to view data through their individual “lenses” and flag anything that seems out of kilter. By enabling your team to monitor progress together, you can adapt to insights and optimize your efforts and your results.


For project managers, I suggest using the early results to make a few less-than-critical decisions. Then, take stock: How much is it helping? Are we spending less time while making better decisions? What can we tweak to improve our DA process and results?


That’s how you start climbing the maturity curve.


Pushing through the fear

For me, brief moments of anxiety still occur when my manager asks, “What’s the basis of your decision?” No one has complete control of their project—but, thanks to DA, my angst is subsiding because I can confidently respond with a credible answer.


When you push through the fear, there are meaningful rewards on the other side. These include making better decisions in less time, delivering the right product, getting to market faster... and sleeping better (most of the time).


My next post will drill down to the next layer of the story, offering tips that will help you extract insights from your “data mess.” In the meantime, let’s discuss: Have you already started down this path? If so, what is or isn’t working? If you haven’t, what’s holding you back?