In Huang & Lepowsky's series of papers A theory of tensor products for module categories for a vertex operator algebra, they defined for a rational vertex algebra $V$ the $P(z)$ tensor product of two modules $W_1$ and $W_2$ to be $$ W_1\boxtimes_{P(z)} W_2=\coprod_k{(\mathcal{M}[P(z)]^{M_k}_{W_1~W_2})}^*\otimes M_k $$ where $\mathcal{M}[P(z)]^{M_k}_{W_1~W_2}$ is the (finite dimensional) vector space of conformal blocks for modules $M'_k$, $W_1$ and $W_2$ at points $\infty$, $z$ and $0$ on the compact Riemann surface $\mathbb{P}^1$. Here $M'_k$ is the module contragedient to $M_k$. They showed that under certain conditions this tensor product gives a vertex tensor category. Now if we define the tensor product by using conformal blocks on an arbitary compact Riemann surface, can we get a vertex tensor category (and hence a braided tensor category) under certain reasonable conditions? If this is true, can we show furthermore the rigidity and the modularity for this category?