I am wondering what is the derivative of the following function with respect to $x(t)$ in sense of distributions. $$ I\left(\int_0^t x(\tau)d\tau \leq c\right) $$ where $I$ is the indicator function and $c$ is a constant.

$\begingroup$ What is the indicator function? $\endgroup$– Liviu NicolaescuCommented Jan 10, 2013 at 11:01

$\begingroup$ Distributions on a space of functions? Even measures there are notoriously difficult (cylindrical measures of L. Schwartz). $\endgroup$– Peter MichorCommented Jan 10, 2013 at 11:36

$\begingroup$ Could you add a bit more details? For example, it is important what you mean by indicator function  the standard definition I know is extendedrealvalued, and hence has no derivative in the sense of classical analysis. There are other derivative concepts that are applicable here, but it would help to know the context of this question. $\endgroup$– Christian ClasonCommented Jan 10, 2013 at 13:01
2 Answers
Let me try.
Your mapping is:
$1  H(xc)$ , where $H$ is the Heaviside function, derivative is $\delta(xc)$.
composed with
$\int_0^t : C^\infty(\mathbb R) \to \mathbb R$, which is bounded and linear, surjective.
No way with the chain rule, in fact, there is no chain rule in this situation.
Simpler approach, but it might not answer your question:
Suppose $s\mapsto x(\tau,s)$ is a smooth variation of $x(\tau)$, i.e.,
$x\in C^\infty(\mathbb R^2)$.
Then $$s\mapsto 1H\Big(\int_0^t x(\tau,s)d\tau c\Big)$$
can be viewed as a distribution in $s$. Take a test function $f(s)$ and consider
$$
\int \Big(1H\Big(\int_0^t x(\tau,s)d\tau c\Big)\Big)(f'(s))ds
$$
Now try formal partial integration with respect to $s$. Since the left hand integrand is 1 or 0, by the fundamental theorem it will be the difference of the values of $f$ at those $s$ where this jumps. If you fix $x(\tau,s)$ you can work this out.
You are asking for the derivative of a nonlinear function on an infinite dimensional space. (You do not specify the latter but the space locally integrable functions seems a natural candidate). The derivative can only exist in a very weak sense so it is natural to go for the directional derivative which has two inputsthe point $ x $ where the derivative is computed and the direction $ y $ in which the rate of change takes place. A back of the envelope calculation suggests that the derivative should then be $$\sum \frac {Y(a)}{x(a)}\delta_a$$ where we use capitals to indicate the primitives which occur in the formulation and the sum is taken over the $a$ in the preimage of $c$ under $X$.
Some general remarks: the question was posed in such a vague manner that it is not really possible to give a precise, rigorous answer. Presumably you have some concrete application in view and I suggest that you check the above formula there to see if it leads to the expected result (ones I looked at were the functions $t$with primitive $t^2$and $\sin t$ for the function $x$).
As is evident from the above formula, the derivative is in a much weaker sense than the usual concepts of functional analysisone requires special conditions on $x$ (beyond just smoothness) for the above expression to make sense and the limit of the difference quotients used to define the derivative does not take place in the underlying function space but in a larger space (of distributionshence the Dirac functions in the formula. I presume that this is the reason for the reference to distributions in your question).

1$\begingroup$ am having TeX problems in editing this answer. the denominator in the formula should be encased in absolute value symbols. $\endgroup$– jbcCommented Feb 10, 2013 at 6:30