Skip to main content
Name of paper; slight proofreading
Source Link
LSpice
  • 12.9k
  • 4
  • 45
  • 69

What is a good way of summarizing when "generating function" approach is useful to an audience of practitioners?

I'm giving a talk on training neural networks (see paperVelikanov, Kuznedelev, and Yarotsky - A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta), and want to motivate the idea of generating functions to audience not exposed to this concept.

One example I was considering was this: consider following matrix sequences

$$ \begin{array} \ A^0,A^1,A^2,\ldots\\ B^0,B^1,B^2\ldots\\ \end{array} $$\begin{gather*} A^0,A^1,A^2,\dotsc\\ B^0,B^1,B^2\dotsc.\\ \end{gather*}

We can consider a new sequence $(A+B)^n$ $$\ (A+B)^0,(A+B)^1,(A+B)^2,\ldots$$$$(A+B)^0,(A+B)^1,(A+B)^2,\dotsc.$$

If the original sequences had generating functions $G_A, G_B$$G_A$, $G_B$, the combined sequencesequence's generating function is this:

$$G_{A+B}=(I-x G_A B)^{-1} G_A$$$$G_{A+B}=(I-x G_A B)^{-1} G_A.$$

In this case, the "generating function" approach replaces matrix powering with inversion, so it helps when inversion is easy (iei.e., $A+B$ is diagonal + rank1rank 1), and doesn't help if matrix powering is easy.

What is a good way of summarizing when "generating function" approach is useful to an audience of practitioners?

I'm giving a talk on training neural networks (paper), and want to motivate the idea of generating functions to audience not exposed to this concept.

One example I was considering was this: consider following matrix sequences

$$ \begin{array} \ A^0,A^1,A^2,\ldots\\ B^0,B^1,B^2\ldots\\ \end{array} $$

We can consider a new sequence $(A+B)^n$ $$\ (A+B)^0,(A+B)^1,(A+B)^2,\ldots$$

If original sequences had generating functions $G_A, G_B$, combined sequence generating function is this:

$$G_{A+B}=(I-x G_A B)^{-1} G_A$$

In this case, "generating function" approach replaces matrix powering with inversion, so it helps when inversion is easy (ie, $A+B$ is diagonal + rank1), and doesn't help if matrix powering is easy.

What is a good way of summarizing when "generating function" approach is useful to an audience of practitioners?

I'm giving a talk on training neural networks (see Velikanov, Kuznedelev, and Yarotsky - A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta), and want to motivate the idea of generating functions to audience not exposed to this concept.

One example I was considering was this: consider following matrix sequences

\begin{gather*} A^0,A^1,A^2,\dotsc\\ B^0,B^1,B^2\dotsc.\\ \end{gather*}

We can consider a new sequence $(A+B)^n$ $$(A+B)^0,(A+B)^1,(A+B)^2,\dotsc.$$

If the original sequences had generating functions $G_A$, $G_B$, the combined sequence's generating function is:

$$G_{A+B}=(I-x G_A B)^{-1} G_A.$$

In this case, the "generating function" approach replaces matrix powering with inversion, so it helps when inversion is easy (i.e., $A+B$ is diagonal + rank 1), and doesn't help if matrix powering is easy.

Source Link

Introducing generating functions to engineer audience?

What is a good way of summarizing when "generating function" approach is useful to an audience of practitioners?

I'm giving a talk on training neural networks (paper), and want to motivate the idea of generating functions to audience not exposed to this concept.

One example I was considering was this: consider following matrix sequences

$$ \begin{array} \ A^0,A^1,A^2,\ldots\\ B^0,B^1,B^2\ldots\\ \end{array} $$

We can consider a new sequence $(A+B)^n$ $$\ (A+B)^0,(A+B)^1,(A+B)^2,\ldots$$

If original sequences had generating functions $G_A, G_B$, combined sequence generating function is this:

$$G_{A+B}=(I-x G_A B)^{-1} G_A$$

In this case, "generating function" approach replaces matrix powering with inversion, so it helps when inversion is easy (ie, $A+B$ is diagonal + rank1), and doesn't help if matrix powering is easy.