Screening for the vital few variables quickly and efficiently, a case study.
We were faced with a project that had 13 independent variables that COULD have an impact on the final product. Remember when you were first taught the scientific method? Generally we are taught to change only one variable at a time. With 13 variables, that means we would have to run at least 213 = 8,192 experiments. And in our situation, we had a couple of variables that had the potential for more than 2 conditions, so we could have a lot more runs. Obviously, this is a completely impractical design. We utilized an 8 run experimental design that got to an optimal design in about 40 experimental runs.
Following the general assumptions of the Pareto Principle (or if you prefer, the 80/20 rule), we assumed that there were likely only a few of the 13 variables that mattered. But to find the variables that really mattered, we needed a screening protocol.
Fortunately, there is the field of statistical experimental design. I’m not an expert in the area, but I had an article in my files from George Box at the University of Wisconsin, called “What can you find out from Eight experimental runs?”. In the article, Box lays out various ways to design an experiment of only 8 experimental runs for the maximum power. One of the things that surprised me the most in the article was that Box shows that “mere eyeballing of the data” can show you the most important factors. No fancy data analysis needed.
We decided to follow his methods.
In our case, the need was to develop a method to thicken soda pop quickly and effectively, however it had to retain as much of the carbonation as possible and to not generate a huge foamy mess. Thickening a carbonated drink is difficult due 4 technical factors:
Thickening stabilizes any foam generated in the process. Especially xanthan gum. One industrial application of xanthan gum is to stabilize fight fighting foams.
Mixing physically releases carbonation.
If using a powder, the uneven surfaces of the powder particles are perfect sites to release the dissolved carbonation. It can be quite dramatic and reminiscent of a classic childhood science experiment; people refer to this effect as “volcano-ing”.
The final product needs to retain enough carbonation to provide the familiar mouthfeel.
A few months after the release of SimplyThick® EasyMix™, we became aware of a variety of methods being espoused by our sales team. Each was a personally developed method. So we asked them to send us their method(s) for thickening soda so we could determine the best method. We were surprised by the variety of methods and factors that were identified as important to be successful!
As mentioned previously, we grouped them into 13 distinct factors. And some of them – like speed of mixing or length of time mixed – had many different possible values all by themselves. We decided to hold the type of soda constant and test the other 12 factors in groups of 4 per the design of Box. Each variable was assigned two conditions. In statistical shorthand, each condition is assigned as a “+” or a “-“ condition. (Since we were screening for magnitude of impact at this point, we picked some “good guess” values for the non-binary variables.) We set up the experimental runs just like this example:
We chose to measure the total volume of the beverage after thickening. This seemed to capture the amount of carbonation released and stabilized. In our case, more volume is a negative and less volume is a positive outcome.
My natural inclination would be to look at run 3 (305 mL) and conclude that room temperature soda, pre-stirred to release some carbonation, thickened all at once in a glass cup is the best way. While it is true that this is the best run out of these 8 runs, the design allows us to look more carefully at the impact of each factor. Notice in the design that each factor is used in each condition (+ and -) 4 times during the runs.
When we add up the volume of each factor in each condition and subtract the differences, you can see the relative magnitude of each factor in the final results. For example,
Room Temperature soda was used in experimental runs 1, 3, 5, and 7, so 500 + 305 + 460 + 400 = 1665 mL total.
Refrigerated soda was used in experimental runs 2, 4, 6, and 8, so 475 + 350 + 500 + 410 = 1735 mL total.
The difference 1735 – 1665 = 70 mL.
There were 4 runs, so 70/4 = 17.5 mL.
The conclusion is that using refrigerated soda increases the resulting volume of any run by 17.5 mL. Or, if you prefer, you can say that using room temperature soda decreases the volume of any run by 17.5 mL.
When we do this analysis for all 4 variables, that is where the magic of the statistical design comes out:
It is readily apparent that the most important factor (or these 4 variables) impacting the final volume was whether the sample was pre-stirred before thickening. If you pre-stirred the soda for a few seconds, the final volume went down by more than 117 mL. The magnitude of this impact is 3 – 6 times the impact of the other 3 factors.
After running through the 3 sets of 8 experimental runs, we found the 4 highest impact factors were:
Pre-stirring for 5 seconds
Order of addition (adding the soda or the thickener to the cup first)
Notice that the first three are all variables with more than 2 choices. After just 24 experimental runs, we could turn our attention to fine tuning these non-binary variables. We varied mixing time and speed and amount of pre-stirring in the next 2 sets of experimental runs.
After just 40 experimental runs, we had a method that we felt was very superior to any that had been sent to us, which may be near the complete optimum, and met all of the design criteria. The final method involves these steps:
Pre-stir soda for 5 seconds in a first cup.
Add the thickener to a second cup.
Add the pre-stirred soda to the second cup.
Mix slowly for 45 seconds.
The result is a carbonated, great looking thickened soda. We then trialed the method across a wide variety of sodas with similar excellent results. The method has been used with great success in the real world and many patients are enjoying excellent thickened sodas.
We have published some poster presentations highlighting the power of these designs. The most prominent one so far has been at the 2019 convention of the American Speech Language Hearing Association entitled, “Using Fractional Replicates To Optimize Thickened Soda”.