In the [paper][1] "Transformations of infinite series" Bryden Cais gives the following transformations of infinite products[![enter image description here][2]][2][![enter image description here][3]][3]

With some modification of Cais's method using contour integration one can obtain the following generalizations of these infinite product transformations
$$
\prod_{n=1}^\infty\left(\frac{1-e^{-\pi\alpha\sqrt{n^2+\beta^2}}}{1+e^{-\pi\alpha\sqrt{n^2+\beta^2}}}\right)^{(-1)^n}=\sqrt{\frac{\tanh\frac{\pi\beta}{2}}{\tanh\frac{\pi\alpha\beta}{2}}}\prod_{n=1}^\infty\left(\frac{1-e^{-\pi\sqrt{n^2/\alpha^2+\beta^2}}}{1+e^{-\pi\sqrt{n^2/\alpha^2+\beta^2}}}\right)^{(-1)^n},\tag{1}
$$
$$
\prod_{n=1}^\infty\left(1-\tfrac{2\sqrt{5}}{1+\sqrt{5}+4\cosh{\frac{2\pi\alpha\sqrt{n^2+\beta^2}}{5}}}\right)^{\left(\frac{n}{5}\right)}=\prod_{n=1}^\infty\left(1-\tfrac{2\sqrt{5}}{1+\sqrt{5}+4\cosh{\frac{2\pi\sqrt{n^2/\alpha^2+\beta^2}}{5}}}\right)^{\left(\frac{n}{5}\right)}.\tag{2}
$$
It is clear that $(1)$ and $(2)$ reduce to theorem 4 and proposition 26 respectively, when $\beta=0$.

Note that infinite products in theorem 4 and proposition 26 are modular forms, however the infinite products in $(1)$ and $(2)$ are not.

>Q1: What are the most general modular forms that admit generalized transformation formulas like $(1)$ and $(2)$?

In chapter 5 of his paper Cais gives a general methodology to construct modular forms which then can be generalized as above, thus giving an infinite family of formulas like $(1)$ and $(2)$. Then one can take linear combination of arbitrary number of these functions. Will this be the most general function of this kind or there are others?

This question is related to the previous question. The formula 
$$
\prod_{n=0}^\infty\frac{1+e^{-\pi\alpha\sqrt{(2n+1)^2+\beta^2}}}{1+e^{-\pi\sqrt{(2n+1)^2/\alpha^2+\beta^2}}}=\exp\left\{\frac{1}{2}\int_0^\infty\ln\frac{1+e^{-\pi\alpha\sqrt{x^2+\beta^2}}}{1+e^{-\pi\sqrt{x^2/\alpha^2+\beta^2}}}\ dx\right\}.\tag{1}
$$
is a limiting case ($m,n\to\infty$, with $m/n$ fixed) of the following proposition

If $\cos\frac{\pi (j-\frac{1}{2})}{n}+\cosh\alpha_j= \cos\frac{\pi (k-\frac{1}{2})}{m}+\cosh\beta_k=x$ for all integers $1\le j\le n,\ 1\le k\le m$ then

$$
\prod_{j=1}^n2\cosh m\alpha_j=\prod_{k=1}^m2\cosh n\beta_k.\tag{1a}
$$

The formulas defining $\alpha_j$ and $\beta_k$ arise during solution of Helmholtz equation on a finite rectangular lattice with suitable boundary conditions (see e.g. Phillips, B.; Wiener, N. (1923). [Nets and the Dirichlet problem][4]. Journal of Math. and Physics, Massachusetts Institute, 105–124).

>Q2: Is there a finite analog of $(1)$ similar to $(1a)$?


Any references regarding these infinite products are welcomed. If the question is not clear please ask in the comments and I will clarify it.

**Note.** Q2 has been answered, however Q1 is still open.


  [1]: http://math.arizona.edu/~cais/Papers/Unpublished/SeriesTransform.pdf
  [2]: https://i.sstatic.net/ZFs6a.png
  [3]: https://i.sstatic.net/3LbJv.png
  [4]: http://onlinelibrary.wiley.com/doi/10.1002/sapm192321105/abstract