Hypothesis testing

4. Hypothesis testing#

Hypothesis testing is a statistical method used by engineers to determine if there is enough evidence to support a specific claim. This process is crucial in engineering applications, such as quality control and product testing, where it helps verify designs and improve processes by statistically assessing the likelihood that observed outcomes are due to chance rather than actual effects.

Summary of commands#

In this exercise, we will demonstrate the following:

  • SciPy, specifically the statistics package.

  • We will also work with \(z\)-tests for the difference between means of normally distributed data. Shockingly, while this function exists in MATLAB (ztest()), it does not exist in common Python libraries (statsmodels isn’t quite accurate), so we’ll simply write our own! 😎

    • In writing this function, we will use the sf() survival function which is 1 - cdf().

Demo 1#

The time to failure, \(T\), of a part is known to be approximately normally distributed with a mean of \(1000\) hours and a standard deviation \(20\) hours. A new manufacturing process is proposed to increase the mean failure time by well over \(300\) hours. Data collected on \(10\) sample parts from the new manufacturing process yields the following times in hours. The new process will be implemented at the factory only if it can be shown to result in an increase in \(T\) of at least \(300\) hours. Use a hypothesis test to determine if the new process should be adopted. Report the \(p\) value.

1350

1320

1290

1280

1340

1210

1290

1350

1300

1310

The proper hypothesis test is

\[ H_0 : \mu \le 1300 \qquad \qquad H_1 : \mu > 1300 \]

which calls for a one-sided \(z\)-test; in particular, the right-tailed \(z\)-test.

# import packages
import numpy as np
from scipy.stats import norm

# writing our own z-test function!
def my_ztest(arr, mu, std, side):
    """
    Args: 
        arr - Array of sample values
        mu - Population mean
        std - Population standard deviation
        side - type of test/Ha (two-sided, greater, less)
    Returns:
        z_stat - The z-statistic
        p_value - The p-value for the test
    """
    # compute sample properties
    sample_mean = np.mean(arr)
    n = len(arr)

    # compute z-statistic
    z_stat = (sample_mean - mu) / (std / np.sqrt(n))

    # compute the p-value
    if tail == 'two-sided':
        p_value = 2 * norm.sf(np.abs(z_stat))
    elif tail == 'greater':
        p_value = norm.sf(z_stat)
    elif tail == 'less':
        p_value = norm.cdf(z_stat)

    return z_stat, p_value


# data
x = np.array([1350, 1320, 1290, 1280, 1340, 1210, 1290, 1350, 1300, 1310])

# constants
mu = 1300
sigma = 20
alpha = 0.05
tail = 'greater'

# run the test
z_stat, p_value = my_ztest(x, mu, sigma, tail)
print(f"The p-value is {p_value:.4f}")

The p-value is 0.2635

Since the \(p\)-value is \(> \alpha=0.05\), we cannot reject the null hypothesis and the new process is not adopted.

Important

Indeed, the language is funny, but it reflects the fact that we’re evaluating the strength of the data against \(H_0\), rather than the strength in proving it true. In fact, \(H_0\) could still be false (in the absolute sense), and we just simply didn’t collect the right data.

Demo 2#

It is desired to compare the performance of two types of gasoline engines by comparing their fuel consumption for a certain number of miles traveled. No historical data is available from which to determine the variance of the fuel consumption. However, is it known that the two engines have similar characteristics and that any variability in the fuel usage will be the same for both types of engines. It is also assumed that the fuel consumption is approximately normally distributed. The fuel consumption data in gallons for 3 engines of the first type and 4 engines of the second type are shown in the table below. Can it be concluded that the two types of engines demonstrate different performance at the \(10\%\) significance level?

Type 1 usage (gal)

Type 2 usage (gal)

540

575

520

540

535

560

—

545

The proper hypothesis test is

\[ H_0 : \mu_1 = \mu_2 \qquad \qquad H_1 : \mu_1 \neq \mu_2 \]

Since the variance is unknown, we shall use a two-sided \(t\)-test here.

# import packages
import numpy as np
from scipy.stats import ttest_ind

# data
x = np.array([540, 520, 535])
y = np.array([575, 540, 560, 545])

# constants
tail = 'two-sided'

# run the test
res = ttest_ind(x, y, alternative=tail)
print(f"The t-statistic is {res.statistic:.4f}")
print(f"The p-value is {res.pvalue:.4f}")

The t-statistic is -2.1972
The p-value is 0.0794

Since the \(p\)-value \(< 0.1\), the null hypothesis is rejected and it is concluded that the fuel consumptions are not the same at the \(10\%\) significance level.

Contents