Let $a^*$ and $b^*$ be the two linear combinations of $a$ and $b$ that satisfy $$ a\cdot a^* = b\cdot b^* = 1\qquad\text{while}\qquad a\cdot b^* = b\cdot a^* = 0, $$ and write $x = v\,a^* + w\,b^* + c$ where $c:I^2\to\mathbb{R}^4$ satisfies $a\cdot c = b\cdot c = 0$. Then the above equations become $v_t = w_u = v_u-w_t=0$, which implies that there are constants $v_0$, $w_0$, and $r_0$ such that $$ x = (v_0 + r_0\, u)\,a^* + (w_0 + r_0\, t)\,b^* + c.\tag 3 $$ Moreover, $c$ is arbitrary subject to the equations $a\cdot c = b\cdot c = 0$. This is the general solution of the system in the case that $a$ and $b$ are (linearly independent) constants. Imposing the given 'auxilliary conditions' forces $v_0=r_0=w_0=0$ and $c(t,0)=c_u(t,0)=0$, so the solution reduces to $x(t,u) = u^2w(t,u)$ where $w:I^2\to\mathbb{R}^4$ is any mapping satisfying the two linear constraints $a\cdot w = b\cdot w = 0$. Thus, the general solution subject to the auxilliary conditions still depends on two arbitrary functions of two variables.

Methods for writing down the solutions of an equation such as (7) are found in the 19th century PDE literature. I can say more about this if you want. The most elementary thing to do is to simply take one of the pair $(v,w)$ to be an arbitrary function on $I^2$ and then use the integrating factor method to solve the resulting linear PDE for the other. There are, of course, more sophisticated things one can do to reduce the problem further to a classical normal form.

Let $a^*$ and $b^*$ be the two linear combinations of $a$ and $b$ that satisfy $$ a\cdot a^* = b\cdot b^* = 1\qquad\text{while}\qquad a\cdot b^* = b\cdot a^* = 0, $$ and write $x = v\,a^* + w\,b^* + c$ where $c:I^2\to\mathbb{R}^4$ satisfies $a\cdot c = b\cdot c = 0$. Then the above equations become $v_t = w_u = v_u-w_t=0$, which implies that there are constants $v_0$, $w_0$, and $r_0$ such that $$ x = (v_0 + r_0\, u)\,a^* + (w_0 + r_0\, t)\,b^* + c.\tag 3 $$ Moreover, $c$ is arbitrary subject to the equations $a\cdot c = b\cdot c = 0$. This is the general solution of the system in the case that $a$ and $b$ are (linearly independent) constants. Imposing the given 'auxilliary conditions' forces $v_0=r_0=w_0=0$ and $c(t,0)=c_u(t,0)=0$, so the solution reduces to $x(t,u) = u^2w(t,u)$ where $w:I^2\to\mathbb{R}^4$ is any mapping satisfying the two linear constraints $a\cdot w = b\cdot w = 0$. Thus, the general solution subject to the auxilliary conditions still depends on two arbitrary functions of two variables. (A similar story holds whenever $a$ and $b$ take values in a fixed $2$-dimensional subspace $S\subset\mathbb{R}^4$.)

Methods for writing down the solutions of an equation such as (7) can be found in the 19th century PDE literature. I can say more about this if you are interested. The most elementary thing to do is to simply take one of the pair $(v,w)$ to be an arbitrary function on $I^2$ and then use the integrating factor method to solve the resulting linear PDE for the other.

There are, of course, more sophisticated things one can do to reduce the problem further to an 'explicit solution' without any integration involved. In particular, it can be shown that, for 'generic' $a$ and $b$ (i.e., for $(a,b)$ in a dense, open set in the $C^\infty$-topology on pairs of smooth maps $I^2\to\mathbb{R}^4$), there exist second-order linear differential operators $A$ and $B$ on $C^\infty(I^2)$ such that $(v,w) = \bigl(A(z),B(z)\bigr)$ solves (7) for every smooth function $z$ on $I^2$ and, moreover, every smooth solution $(v,w)$ of (7) is of the form $(v,w) = \bigl(A(z),B(z)\bigr)$ for some smooth function $z$ on $I^2$. The auxilliary conditions then translate into a pair of linear conditions on $z$ and its partials up to fourth order along the edge $u=0$.

Is there anything else that you are not telling us about $a$ and $b$? The particulars of these two vector-valued functions have a great influence on what the general solution of the system $$ a\cdot x_t = b\cdot x_u = a\cdot x_u - b\cdot x_t = 0\tag 1 $$ looks like.

For example, take the very special case in which $a$ and $b$ are constant. Then the above equations become $$ (a\cdot x)_t = (b\cdot x)_u = (a\cdot x)_u - (b\cdot x)_t = 0\tag 2 $$

Let $a^*$ and $b^*$ be the two linear combinations of $a$ and $b$ that satisfy $$ a\cdot a^* = b\cdot b^* = 1\qquad\text{while}\qquad a\cdot b^* = b\cdot a^* = 0, $$ and write $x = v\,a^* + w\,b^* + c$ where $c:I^2\to\mathbb{R}^4$ satisfies $a\cdot c = b\cdot c = 0$. Then the above equations become $v_t = w_u = v_u-w_t=0$, which implies that there are constants $v_0$, $w_0$, and $r_0$ such that $$ x = (v_0 + r_0\, u)\,a^* + (w_0 + r_0\, t)\,b^* + c.\tag 3 $$ Moreover, $c$ is arbitrary subject to the equations $a\cdot c = b\cdot c = 0$. This is the general solution of the system in the case that $a$ and $b$ are (linearly independent) constants.

At the other extreme, consider the case in which the $5$ vector-valued functions $a$, $b$, $a_t$, $b_u$, and $a_u{-}b_t$ span $\mathbb{R}^4$ at every $(t,u)\in I^2$. Let us write $$ p\,a_t + q\,b_u + r\,(a_u{-}b_t) + f\,a + g\,b = 0\tag 4 $$ for some functions $p$, $q$, $r$, $f$ and $g$, where $p$, $q$, and $r$ do not simultaneously vanish. Up to a common nonzero multiple, these $5$ functions are determined by $a$ and $b$, so they can be regarded as known.

Set $$x\cdot a = v\qquad \text{and} \qquad x\cdot b = w.\tag 5 $$
Then, with the given equations, we have the three identities $$ x\cdot a_t = v_t\qquad x\cdot b_u = w_u\qquad x\cdot(a_u-b_t) = v_u-w_t\,.\tag 6 $$ Given the above relation, we then get a single linear first order PDE for the pair $(v,w)$: $$ p\,v_t + q\,w_u + r\,(v_u-w_t) + f v + g\,w = 0.\tag 7 $$ Given a solution $(v,w)$ to (7), the five linear equations (5)$+$(6) are consistent and uniquely determine a solution $x$ to the original system of $3$ equations. Thus, the solutions of the original overdetermined system of three equations for four unknowns have now been expressed in terms of the solutions to a single linear PDE for two unknowns.

Methods for writing down the solutions of an equation such as (7) are found in the 19th century PDE literature. I can say more about this if you want. The most elementary thing to do is to simply take one of the pair $(v,w)$ to be an arbitrary function on $I^2$ and then use the integrating factor method to solve the resulting linear PDE for the other. There are, of course, more sophisticated things one can do to reduce the problem further to a classical normal form.

In between these two extreme cases of hypotheses on $a$ and $b$, there are some exceptional cases that require separate handling. Also, I can say more about this, if you want.

Is there anything else that you are not telling us about $a$ and $b$? The particulars of these two vector-valued functions have a great influence on what the general solution of the system $$ a\cdot x_t = b\cdot x_u = a\cdot x_u - b\cdot x_t = 0\tag 1 $$ looks like.

For example, take the very special case in which $a$ and $b$ are constant. Then the above equations become $$ (a\cdot x)_t = (b\cdot x)_u = (a\cdot x)_u - (b\cdot x)_t = 0\tag 2 $$

Let $a^*$ and $b^*$ be the two linear combinations of $a$ and $b$ that satisfy $$ a\cdot a^* = b\cdot b^* = 1\qquad\text{while}\qquad a\cdot b^* = b\cdot a^* = 0, $$ and write $x = v\,a^* + w\,b^* + c$ where $c:I^2\to\mathbb{R}^4$ satisfies $a\cdot c = b\cdot c = 0$. Then the above equations become $v_t = w_u = v_u-w_t=0$, which implies that there are constants $v_0$, $w_0$, and $r_0$ such that $$ x = (v_0 + r_0\, u)\,a^* + (w_0 + r_0\, t)\,b^* + c.\tag 3 $$ Moreover, $c$ is arbitrary subject to the equations $a\cdot c = b\cdot c = 0$. This is the general solution of the system in the case that $a$ and $b$ are (linearly independent) constants. Imposing the given 'auxilliary conditions' forces $v_0=r_0=w_0=0$ and $c(t,0)=c_u(t,0)=0$, so the solution reduces to $x(t,u) = u^2w(t,u)$ where $w:I^2\to\mathbb{R}^4$ is any mapping satisfying the two linear constraints $a\cdot w = b\cdot w = 0$. Thus, the general solution subject to the auxilliary conditions still depends on two arbitrary functions of two variables.

At the other extreme, consider the case in which the $5$ vector-valued functions $a$, $b$, $a_t$, $b_u$, and $a_u{-}b_t$ span $\mathbb{R}^4$ at every $(t,u)\in I^2$. Let us write $$ p\,a_t + q\,b_u + r\,(a_u{-}b_t) + f\,a + g\,b = 0\tag 4 $$ for some functions $p$, $q$, $r$, $f$ and $g$, where $p$, $q$, and $r$ do not simultaneously vanish. Up to a common nonzero multiple, these $5$ functions are determined by $a$ and $b$, so they can be regarded as known.

Set $$x\cdot a = v\qquad \text{and} \qquad x\cdot b = w.\tag 5 $$
Then, with the given equations, we have the three identities $$ x\cdot a_t = v_t\qquad x\cdot b_u = w_u\qquad x\cdot(a_u-b_t) = v_u-w_t\,.\tag 6 $$ Given the relation (4), we then get a single linear first order PDE for the pair $(v,w)$: $$ p\,v_t + q\,w_u + r\,(v_u-w_t) + f v + g\,w = 0.\tag 7 $$ Given a solution $(v,w)$ to (7), the five inhomogeneous linear equations (5)$+$(6) for $x$ are consistent and uniquely determine a solution $x$ to the original system of $3$ equations. Thus, the solutions of the original overdetermined system of three equations for four unknowns have now been expressed in terms of the solutions to a single linear PDE for two unknowns. Imposing the 'auxilliary conditions' is equivalent to requiring that the five functions $v$, $w$, $v_t$, $w_u$, and $v_u{-}w_t$ and their first derivatives with respect to $u$ should all vanish along the edge $u=0$ in $I^2$.

Methods for writing down the solutions of an equation such as (7) are found in the 19th century PDE literature. I can say more about this if you want. The most elementary thing to do is to simply take one of the pair $(v,w)$ to be an arbitrary function on $I^2$ and then use the integrating factor method to solve the resulting linear PDE for the other. There are, of course, more sophisticated things one can do to reduce the problem further to a classical normal form.

In between these two extreme cases of hypotheses on $a$ and $b$, there are some exceptional cases that require separate handling. Also, I can say more about this, if you want.