3
$\begingroup$

I encountered this problem in a book "Pattern Recognition and Machine Learning" by Christopher M. Bishop. I excerpt it below:

screenshot of the book

In the excerpt, the big-O notation $O(\xi^3)$ in equation (5.130.1) and (5.130.2) fits my understanding because $\xi$ is the variable of the function being expanded as a Taylor series. But I don't understand the big-O notation $O(\xi^3)$ in equation (5.130.3), because the thing being expanded here, $\widetilde E$, is not a function of $\xi$. Effectively, the term represented by $O(\xi^3)$ in equation (5.130.3) is integral $$\int O(\xi^3)p(\xi){\rm d}\xi.\label{1}\tag{1}$$

Next the author discards the $O(\xi^3)$ term in the following analysis. I can see intuitively that the value of integral \eqref{1} is really small so that it can be safely discarded, but I can find no rigorous math argument to support it. Concretely, what does $O(\xi^3)$ in equation (5.130.3) mean? In typical definition of Big-O (e.g., here), it should be a function, but, as an integral, it is a constant here with regard to $\xi$. Next, how to prove that the integral $(1)$ is $O(\xi^3)$? As a side note, the author later in his book also writes $\mathbb E[\xi^2]=O(\xi^2)$. I really don't know what the author means by using these big-O notations. I would appreciate it very much if you can help me understand. Thanks a lot.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ I presume that the author means ${\cal O}(\sigma^3)$ rather than ${\cal O}(\xi^3)$, where $\sigma^2$ is the variance of $\xi$ (assumed to be small) $\endgroup$
    – Carlo Beenakker
    Commented Dec 21, 2022 at 9:01
  • $\begingroup$ @Carlo Beenakker: Since the distribution of $\xi$ has a zero mean and small variance, $\xi$ is also small most of the time, hence $\xi^3$, $O(\xi^3)$ and the integral of $(1)$. I just cannot formulate this idea rigorously in math. $\endgroup$
    – zzzhhh
    Commented Dec 21, 2022 at 10:38
  • 2
    $\begingroup$ the point I want to make is that it makes no sense to write $\mathbb E[\xi^2]=O(\xi^2)$ ; it should be $\mathbb E[\xi^2]=O(\sigma^2)$, I presume this is just a sloppy notation of the author. $\endgroup$
    – Carlo Beenakker
    Commented Dec 21, 2022 at 15:03
  • $\begingroup$ @Carlo Beenakker: Thank you. As you commented, I think $O(\xi^3)$ here has nothing to do with regular big-O notation. It is only "a sloppy notation" meaning that it is a small number (a third or second moment of $\xi$ multiplied by a finite integral which is still small by assumption), so it can be discarded safely. $\endgroup$
    – zzzhhh
    Commented Dec 22, 2022 at 5:19

0

Reset to default

You must log in to answer this question.