How to compare two experimental samples

Question

I am conducting an experiment on a mechanical device. The theoy is that there is a function that maps an input force ($F_I$) to the otput ($F_O$), i.e. $F_O = f(F_I)$ and vice versa (function $f$ is linear). My goal is to test this theory in real-life conditions with an influence of numerous factors. Due to some limitations I can only measure some critical input force $F_{Icr}$ which corresponds to the device failure. Measuring it several times I obtain my first sample which I can convert (using the theory) to $F_{Ocr}$. Next I apply external force until failure and obtain my second sample of $F_{Ocr}$, but now it is measured at the output. Now I can somehow compare these two samples of the output force which are, in theory, should be the same.

The question is how to do it properly. I want to present my results and conclusion with some statistical background behind them. Sample size is around 10-20, since there is a lot of work to do applying forces frome different directions etc. Should I just use a simple t-test (my first samples of size 10 appear to be Gaussian if tested by Anderson-Darling test)? I am not even sure is these samples should be treated as dependent or not and need some advice on what tests should I use.

On the next stage I want to introduce some factors (stiffness in particular) and study how they affect my results. What would be the more reasonable option to analyse possible correlation for these small samples?

Answer 1

$\newcommand{\si}{\sigma}$ Let $X_1,X_2,\dots$ be iid random variables (r.v.'s) representing the measurements of the input force. Let $Y_1=a+bX_1,Y_2=a+bX_2,\dots$ be iid r.v.'s representing the measurements of the output, for some real $a$ and $b$. We assume that $X_i\sim N(\mu,\si^2)$.

Only the observed values $x_1,\dots,x_m$ of $X_1,\dots,X_m$ and $y_{m+1},\dots,y_{m+n}$ of $Y_{m+1},\dots,Y_{m+n}$ are known to us, for some sample sizes $m$ and $n$. The logarithm of the likehood function (that is, of the joint pdf) of $X_1,\dots,X_m,Y_{m+1},\dots,Y_{m+n}$ is given by the formula $$\ln L=c-\frac{m+n}2\,\ln(\si^2)-\frac n2\,\ln(b^2) -\frac1{2\si^2}\,\sum_1^m(x_i-\mu)^2 -\frac1{2b^2\si^2}\,\sum_{m+1}^{m+n}(y_i-a-b\mu)^2, $$ where $c:=\ln((2\pi)^{-m/2-n/2})$. We assume that the parameters $\mu,\si,a,b$ are all unknown, and we want to estimate them for this model, based on the data $x_1,\dots,x_m,y_{m+1},\dots,y_{m+n}$.

Note that $L$ will not change if $b$ and $a$ are replaced by $-b$ and $a+2\mu$, respectively. Therefore, the sign of $b$ is not identifiable in this model and, at best, we can only estimate $b^2$.

Let us now find the maximum likelihood estimators (MLEs) of $\mu,\si^2,a,b^2$. Taking the partial derivative of $\ln L$ in $a$, we see that the maximum of $L$ in $a$ occurs at \begin{equation*} a=\bar y-b\mu, \tag{1} \end{equation*} where \begin{equation*} \bar y:=\frac1n\,\sum_{m+1}^{m+n}y_i,\quad \bar x:=\frac1m\,\sum_1^m x_i, \end{equation*} so that we can write $$\ln L=c-\frac{m+n}2\,\ln(\si^2)-\frac n2\,\ln(b^2) -\frac1{2\si^2}\,\sum_1^m(x_i-\mu)^2 -\frac{ns_y^2}{2b^2\si^2}, $$ where \begin{equation*} s_y:=\sqrt{\frac1n\,\sum_{m+1}^{m+n}(y_i-\bar y)^2},\quad s_x:=\sqrt{\frac1m\,\sum_1^m(x_i-\bar x)^2}. \end{equation*} Now we see that the maximum of $L$ in $\mu$ occurs at \begin{equation*} \mu=\bar x, \tag{2} \end{equation*} so that we can write $$\ln L=c-\frac{m+n}2\,\ln(\si^2)-\frac n2\,\ln(b^2) -\frac{ms_x^2}{2\si^2} -\frac{ns_y^2}{2b^2\si^2}. $$ Now we see that the maximum of $L$ in $b^2$ occurs at \begin{equation*} b^2=s_y^2/\si^2, \tag{3} \end{equation*} so that we can write \begin{align*} \ln L&=c-\frac{m+n}2\,\ln(\si^2)-\frac n2\,\ln(s_y^2/\si^2) -\frac{ms_x^2}{2\si^2}-n/2 \\ &=c-\frac m2\,\ln(\si^2)-\frac n2\,\ln(s_y^2) -\frac{ms_x^2}{2\si^2}-n/2. \end{align*} Now we see that the maximum of $L$ in $\si^2$ occurs at \begin{equation*} \si^2=s_x^2. \tag{4} \end{equation*}

Collecting (4)--(1), we conclude that the values of the MLEs of $\si^2,b,\mu,a$ are, respectively,
\begin{equation*} \hat\si^2:=s_x^2,\quad \hat b:=\pm s_y/s_x, \end{equation*} \begin{equation*} \hat\mu:=\bar x,\quad \hat a:=\bar y\mp\frac{s_y}{s_x}\hat x. \end{equation*}