# Boyd & Chua 1985: Is the proof of Lemma 2 correct?

$$\newcommand\norm[1]{\lVert#1\rVert}\newcommand\abs[1]{\lvert#1\rvert}$$I'm reading this article by Boyd and Chua [1], in which they prove the approximability of arbitrary time-invariant (TI) operators (or filters) with polynomials of some basis filters with separation and fading memory (FM) properties. This shows that those operators, indeed have fintie Volterra expansion approximates.

Consider the following:

• $$C(\Bbb R)$$ as the space of bounded continuous functions $$u: \Bbb R \to \Bbb R$$,
• $$\norm u = \sup_{t \in \Bbb R} |u(t)|$$ being their norm,
• $$\Bbb R_-$$ as $$\{t : t \le 0 \}$$, with analogous definitions for $$C(\Bbb R_-)$$ and $$\norm\cdot$$ as above,
• and time-invariant functionals $$F: C(\Bbb R_-) \to \Bbb R$$ and time-invariant operators $$N: C(\Bbb R) \to C(\Bbb R)$$.
• $$K=\bigl\{u\in C(\Bbb R): \abs{u(t)}\le M_1\land\abs{u(s)-u(t)}\le M_2(s-t)\;\forall t, s\in\Bbb R, t: explicitly, $$K$$ is the space of bounded uniformly Lipschitz continuous functions. Functions belonging to $$K$$ are called signals.
• $$K_-=\{u\in K : u(t)=0\text{ if }t>0\}$$: Boyd & Chua prefer to define $$K_-$$ by using a "projection" operator $$P$$ such that $$\DeclareMathOperator{\dmu}{d\!} Pu(t) = \begin{cases} u(t) & t\le 0\\ 0 & t>0 \end{cases}$$ and then noting that $$K_- =P K$$. Be it noted that $$K_-$$ is compact in $$C(\Bbb R)$$ but only with respect to the weighted $$\sup$$ norm defined as $$\norm u_w=\sup_{t\le 1} \abs{u(t)w(-t)}$$ where $$w:\Bbb R_+\to(0,1]$$ is a weight function such that $$\lim_{t\to\infty} w(t)=0$$ (see below).
• For every TI operator $$N$$, an associated $$F$$ is defined by $$Fu=Nu_e(0)$$, in which $$u_e$$ is some extension of $$u$$ from $$C(\Bbb R_-)$$ to $$C(\Bbb R)$$.
• Furthermore, the fading memory (FM) property is defined for TI operators on $$N \in K \subset C(\Bbb R)$$ as the existence of some weight function $$w: \Bbb R_+ \to (0,1]$$ with $$\lim_{t \to \infty} w(t) = 0$$ that makes $$K_-$$ closed w.r.t. the weighted norm $$w$$. i.e. for any $$v,u \in K$$, one can find a $$\delta$$ for any $$\epsilon$$ such that

$$\norm{u-v}_w:= \sup_{t\le 0} \abs{u(t)-v(t)}w(-t) < \delta \implies \abs{Nu(0) - Nv(0)} < \epsilon$$

Lemma 2 states that there are some functionals on $$K_-$$ that separate points. To prove this, a class of functionals $$G \in \mathbf G \subset K_-$$ is defined by $$\mathbf{G}=\left\{G(u)=\int\limits_0^{+\infty}g(\tau)u(-\tau)\dmu\tau :u\in K_-\land\int\limits_0^{+\infty}\abs{g(\tau)}w(\tau)^{-1}\dmu\tau\right\}$$ which are (shown to be) continuous w.r.t. the weighted norm $$w$$ (thanks to the condition introduced above). The authors further construct functions $$g_0$$ defined by:

$$g_0(t):= [u(-t)-v(-t)] w(t)\exp (-t)$$

and their associated $$G_0$$ assuming that they belong to $$\mathbf G$$. Then, they show that $$g_0$$ indeed separates points on $$K_-$$. However, I don't understand why $$G_0 \in \mathbf G$$ in the first place. $$g_0$$, by construction, dependends on $$u$$ and $$v$$. Thus, it does not look to me that $$G_0$$ is even time-invariant (simply shifting $$u$$ and $$v$$ in time will yield a different $$g_0$$). So my question is why is the proof of this lemma correct?

Reference

[1] Stephen Boyd, Leon O. Chua, "Fading memory and the problem of approximating nonlinear operators with Volterra series" (English), IEEE Transactions on Circuits and Systems 32, 1150-1161 (1985), MR0809696, Zbl 0587.93028, doi:10.1109/TCS.1985.1085649.

• You should explain what you mean by (i) "a functional $G_0 \in \mathbf{G}$, by defined an associated function $g_0$" ("defined" in what way exactly? "associated" in what way exactly? "by defined" or "defined by"? (ii) "endowed with a decaying weight $w(t)$ to ensure its compactness" ("endowed" in what way exactly? "ensure" how?) (iii) "w.r.t. $w(t)$ on $K_-$". Also, (i) undefined, non-mathematical terms such as "inputs" and "past times" should be avoided or, at least, defined mathematically; (ii) the functions are $u$ and $v$, not $u(t)$ and $v(t)$, please. Jan 5, 2023 at 15:19
• @IosifPinelis Thanks for your comment. Please note that I used their own wording precisely. Yet, I'm sure they would agree with you on the word choice: they addressed this issue in section IX, starting with "It is possible to generalize the results of this paper to a clean and simple mathematical form, at the cost of some engineering intuition". Indeed, they avoid confusion by setting up a context for every single term they used. I skipped explaining it, assuming that it is clear enough. But if not, I'll update my question. Jan 5, 2023 at 16:35
• @DanieleTampieri, re, TeX note: although I think they are usually typeset identically, if it is to be written symbolically, then it is semantically better to use \land for conjunction rather than \wedge. While fixing some typos, I edited accordingly. Jan 8, 2023 at 0:17
• Thanks @LSpice, your suggestions pertaining TeX are always very useful to me. Jan 8, 2023 at 6:09
• @DanieleTampieri Thank you very much. Truly appreciate the time and effort you put into this. Jan 8, 2023 at 11:56

Don't let the bad wording of the paper fool you. The statement of lemma 2 in [1] above does not mean that the functional $$G$$ constructed by the kernel $$g_0$$ separates all the points $$u, v\in K_-$$ such that $$u\neq v$$: it means only that, for any such two functions, you can construct a kernel such that the associated linear functional constructed as above is such that $$Gu\neq Gv$$. At first I was fooled by the same thought but then I read Remark 4 in the same page ([1], §IV p. 1153), which contains the statement quoted below
Suppose $$E$$ is a compact metric space and $$\mathbf G$$ a set of continuous functionals on $$E$$ which separate points, that is, for any distinct $$u,v\in E$$ there is a $$G\in\mathbf G$$ such that $$Gu \neq Gv$$.
Then I simply realized that the authors want only to show that the functional $$\DeclareMathOperator{\dmu}{d\!}G_0(q)=\int\limits_0^{+\infty}g_0(\tau)q(-\tau)\dmu\tau \quad \forall q\in K_-$$ with the kernel $$g_0$$ constructed as described above is such that $$G_0 u\neq G_0v$$ for any fixed couple of different functions chosen for its construction. If $$u_1, v_1\in K_-$$ are a different couple such that $$u_1\neq v_1$$, you do not know if $$G_0u_1\neq G_0 v_1$$ (an in general this is not true): however you can construct another functional, say $$G_1$$, whose kernel is defined as exactly as $$g_0$$ i.e. $$g_1(t):= [u_1(-t)-v_1(-t)] w(t)\exp (-t)$$ and separates the new points but obviously possibly not the points $$u$$ and $$v$$.
• I'm convinced by this answer. Yet, if that's what this Lemma wanted to say, it seems to me that the main premise of the paper (existence of $u$-independent Volterra expansionis for TI operators) not met. I may be wrong. I'll think about it and post an update or another question. Nonetheless, thanks to all who engaged. Jan 8, 2023 at 12:03