This is worked out in Higher Algebra, example 3.2.4.4.
Concretely, $\mathrm{Alg}_{\mathcal{O}}(\mathcal{C})^\otimes$ is defined as follows: it is the simplicial set over $\mathrm{Fin}_\ast$ such that for any simplicial set $K\to \mathrm{Fin}_\ast$, the set of maps $K\to \mathrm{Alg}_{\mathcal{O}}(\mathcal{C})^\otimes$ over $\mathrm{Fin}_\ast$ is equivalent to the set of diagrams $$ \require{AMScd} \begin{CD} K\times\mathcal{O}^\otimes @>>> \mathcal{C}^\otimes\\ @VVV @VVV \\ \mathrm{Fin}_\ast\times\mathrm{Fin}_\ast @>{\wedge}>> \mathrm{Fin}_\ast \end{CD} $$ such that for every $k\in K$ restriction of the top arrow to $\{k\}\times \mathcal{O}^\otimes$ sends inert morphisms to inert morphisms. Here the bottom arrow is the smash product of pointed sets, sending $(I_+,J_+)$ to $(I\times J)_+$.
If you consider the fiber over $1_+$, you see that it is exactly $\mathrm{Alg}_\mathcal{O}(\mathcal{C})$, moreover for every $o\in\mathcal{O}$ there is a canonical symmetric monoidal functor $\mathrm{Alg}_\mathcal{O}(\mathcal{C})^\otimes\to \mathcal{C}^\otimes$, given by taking the fiber over $o$ of the previous diagram, thus showing that this deserves the name "pointwise tensor product".
This might seem an abstract way of defining it, and it is, but if you reflect a bit on what it is doing you'll see that it makes sense: the smash product of pointed sets is precisely encoding the multiplication you want on the algebra $A\otimes B$. For example, for an associative algebra, the product on $A\otimes B$ is given by $$(A\otimes B)\otimes (A\otimes B)\cong (A\otimes A)\otimes (B\otimes B)\to A\otimes B$$ as desired.