Let us assume two samples, A and B, where A are the results obtained with some standard method, and B are the results obtained with a new method, which is not necessarily more accurate, but has additional advantages (eg: lower cost).

So I'm interested in testing if B is "as good as" A, using non-inferiority hypothesis testing, i.e. that

Ho : $A - B >= \delta$

or, equivalently (?)

Ho : $A - \delta >= B$

where $\delta$ is the maximum clinically-acceptable margin of error.

(Please bear with me if my formulation is not perfect. I'm not a statistician; I'm just a researcher trying to statistically get my way out of a paper bag with minimal damage. Suggestions for improvements are welcome.)

I've seen this hypothesis tested with confidence intervals, which I'm weary of using since my sample size is small. Would it be correct to use Student's t-test to test this hypothesis? If no, why not?

Response to sheldon-cooper: I'm not sure I fully understand what you mean by "the null hypothesis encompasses many values for the mean of one population"

Assuming that testing for Ho: $A >= B$ can validly be tested using Student's two-sample t-test, how does subtracting $\delta$ from $A$ change the validity of the test? In my perhaps naive understanding, you still get two samples from populations with unknown means that we want to test for equivalence, except that one has been artificially penalized.

  • $\begingroup$ I don't know of any t-test that has A >= B as null hypothesis. If such a test were available, then, sure, a test for A - d >= B would be trivially available as well. But all t-tests I know of have A = B as null hypothesis (thus "only one mean"). $\endgroup$ – user3035 Feb 26 '10 at 2:14

I think the answer is no. T tests, basically, work as follows. You manipulate your input data to obtain something that (under the null hypothesis) is normally distributed with mean zero. For example, if the null hypothesis is that the data is normally distributed with mean 28, then the manipulation is simply subtracting 28 from the data. For more complex cases (e.g. if there are two populations with the same, but unknown, mean), the manipulation is a bit more involved, but still some manipulation is needed that gives you a normal distribution with zero mean. Once you obtain that, you use the fact that $\frac{N}{s}$ (where $s$ is sample standard deviation) is distributed according to Student t (that's where the t in t-test comes from) to determine confidence.

In your case, the null hypothesis encompasses many values for the mean of one population. I don't see an obvious way to manipulate it into something that has mean zero. For example, if you have paired data and subtract the two populations, you get $X_i - Y_i \sim N(m, \sigma)$ where $m \geq \delta$. You can further subtract $\delta$ to get mean $m \geq 0$. But that still encompasses many possible values of the mean. I don't see how to manipulate it further to get something that has just one possible value of the mean.

  • $\begingroup$ Hmmm, it does make sense. But what if we reformulate as A - \delta >= B (which should be equivalent, right?), and use a two-sample t-test? Wouldn't A and B both have fixed means then? $\endgroup$ – Bossykena Feb 24 '10 at 2:27
  • $\begingroup$ Thanks for answering this. I've added a follow-up question, as I'm not sure I fully understand how testing for non-inferiority, in this case, differs from testing for equality of two populations A and B. Alternatively, can you suggest references that would help me understand this issue? $\endgroup$ – Bossykena Feb 25 '10 at 20:47

Perhaps, a slight re-formulation may help to deal with the issue pointed our by Sheldon response. State your null as follows:

$H_0: A - B = \delta$

$H_a: A - B >= \delta$

Then you can do a standard one-tailed t-test as the issue raised by Sheldon is no longer applicable. If you have a small sample size then perhaps bootstrap testing will be useful. See http://en.wikipedia.org/wiki/Bootstrapping_(statistics) for details of this approach.

Hope that helps.

  • $\begingroup$ I don't think this formulation works, because then rejecting the null hypothesis proves that A is superior by at least delta, whereas I'm interested in showing that A and B are near equivalent. I'm investigating a couple of papers related to the topic, which seem to suggest that the T-Test could be applicable ("A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability". Schuirmann DJ and "The Power of the Anderson-Hauck Test and the Double T-Test", J. Müller-Cohrs) I'll let you know if/once I dig out a relevant answer. $\endgroup$ – Bossykena Feb 25 '10 at 22:19

Follow-up on my question, after some more research on the question.

The method of using two one-sided two-sample t-tests for clinical equivalence testing is shown in Schuirmann 1987, and appears relatively standard in bioclinical statistics (at least it is very widely cited). Some preconditions must be respected for the test to be valid (including same sample size for both groups and normal distribution of the data); I will refer you to the article for the complete list.

Under the normality assumption that has been made, the two sets of one-sided hypotheses will be tested with ordinary one-sided t-tests.

As the non-inferiority test is only one half of the equivalence testing (In my case, I don't care whether B is superior or simply similar to A), I will assume that the method still holds.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.