Hypothesis Testing Decision Guide

The purpose of this article is not to teach hypothesis testing, alpha risk, etc. It's a simple hack to help practitioners wrestle through the mechanics of hypothesis testing and to develop some muscle memory through a simple graphical routine.

The mechanics of hypothesis testing can be tough to remember or follow.  For this article the focus will be on how to decide whether the hypothesis test or P-value is more consistent with the null or the alternate hypothesis.

Let's use the graphic in Figure 1 to construct the hypothesis test decision guide or template.  This template can be created in a PowerPoint or on a Flipchart in the classroom. If in the classroom I like to have each table team construct Figure 1 on a flip-chart and to have a table representative present the results to the rest of their classmates. If in a coaching environment, it's a great graphic to communicate hypothesis testing results.

However, before we proceed to Step 1, let's review key terms in Figure 1 below.

Ho: Null Hypothesis - This is a statement that describes the status quo or statistical random error as if sampled from the original population. Other references include "no change" between the tested populations, "devils advocate", "champion view", etc.

Ha: Alternate Hypothesis - This is a statement that describes the hypothesized difference or change resulting from an improvement action or a categorical difference. Other references include "target population", the "belt's view", etc.

Alpha Risk - The alpha risk is a decision point that represents a reasonable balance between two types of errors - Type I and Type II. Type I error, also known as producer's risk, is the risk of saying a population change or difference exists when in fact it does not. In the example below, alpha risk is set at 0.05 or 5%. Given this is a decision point, a practitioner may make the call that a population characteristic has changed if the P-value of the test is below 0.05. However it should be known that the risk of being wrong is still the alpha risk minus the P-value. If there is a $1MM invested in the target change, the practitioner may want to consider the alpha risk very carefully.

P- Value - The p-value is a probability expression that is consistent with the Null Hypothesis or that the null hypothesis is true. Notice the range in Figure 1 is from 0 to 0.05 (alpha risk) to 1.00.

Figure 1

Now that the preliminary discussion is out of the way, let's use Figure 1 to illustrate the mechanics of this useful template. Assume that you have a hypothesis test to consider. How would you use this template to assist in obtain the correct results - i.e. "reject" the null hypothesis or "fail to reject" the null hypothesis". Let's get started!

Step #1: Write the practical null (Ho:) hypothesis statement in this section of the slide or flip-chart as referenced by the number 1 in the graphic. What is a null (Ho:) hypothesis statement? Again, it's a statement that suggests there is no difference between the populations being compared. 

Step #2: Now write the null (Ho:) hypothesis statement in statistical terms as referenced by the number 2 in the graphic. Yes, you're now permitted to geek out and indulge your statistical prowess. More on this in the example section below.

Step #3: Write the practical alternate (Ha:) hypothesis statement in this section of the slide or flip-chart.  This is the statement that the project leader or belt is attempting to prove. What is a alternate (Ha:) hypothesis statement?  Again, it's a statement that suggests there is a difference between the populations being compared. Keep it practical and explain this relationship in plain English.  

Step #4: As in Step 2, geek out and write the alternate hypothesis statement in statistical terms.  Dig out your notes if necessary to obtain the correct alternate hypothesis statement.   

Ok, great work. You've now setup the template for your test results and are ready to put put this to use. Let's continue.

Step #5: After you run your statistical analysis, copy the P-Value from your analysis and paste or place it on the template similar to the box shown next to the number 5 in the Figure 1.  Now the fun part. If the P-value is above the alpha risk you assigned, position the p-value above the red dotted line (the alpha risk line) under the "Accept Ho".  If the p-value is below the alpha risk, position or place the P-value below the red dotted line above the "Reject Ho" phase.

Step #6: Now move or place a green check next to the decision "Accept Ho:" or "Reject Ho:".  For example if the P-value turns out to be below 0.05 in the case above, place the green check next to or just above the Reject Ho. Easy as pie.  

Figure 2

Now that we've gone through the setup one time, let's use Figure 2 and the write up below to illustrate how this works for an example situation and 2-Sample T test.

Situation: Suppose the American Association of University Professors (AAUP) conducts salary studies of college professors.  Suppose we want to know if the salaries for faculty in public institutions were the same as those for private institutions. We'll assume that we're testing average salaries between the two types of institutions.

The data collected was compared and the results are shown in the table below. Although the private institution has a higher mean salary of $66.4 versus the public institution at $57.5, the probability that this is theoretical sampling error is 0.128 or 12.8%. This is higher than the decision point or alpha risk of 0.05 or 5%.

Since the p-value of 0.128 is greater than 0.05, position the p-value above the 0.05 decision point as shown in Figure 2 which is consistent with Ho or the null hypothesis. In other words, we will not claim there is a difference between the average salaries between the private and public institutions because statistically there is a 12.8% chance the results are from sampling error. Next step is to position or place the green check next to the result.

My students like this approach as it provides a method or routine to arrive at the correct conclusion. Additionally, this template is very handy if you're asking for homework results.

As always, this article makes no claims or declarations as "the way" or "best way", et al. It's only a shared idea for others to use if relevant.

If you enjoyed this article, click the like button and share with your network. If you would more information about this topic or related initiatives contact us at succeed@cikata.com. Don’t forget to click on our social links for the latest.