The answer is yes. Let $C$ be a weakly contractible quasicategory. We wish to show that a category with pushouts and filtered colimits has $C$ colimits. Let $\Delta^0 \to C$ be the inclusion of your favorite object. By the small object argument, we can factor this as $\Delta^0 \to X \to C$ where the first map is cellularly anodyne and then the second is a trivial fibration. So any $C$-indexed diagram pulls back to an $X$-indexed one whose colimit is the same and we may assume that $C=X$ so the inclusion $\Delta^0 \to C$ is cellularly anodyne.
Since the colimit of a diagram indexed by a colimit of categories is the colimit of the colimits, and because cellularly anodyne maps are built by transfinite composition (a filtered colimit) and cobase change (a pushout) from horn inclusions $\Lambda_k^n \to \Delta^n$, it will suffice to construct colimits indexed by $\Lambda_k^n$ and $ \Delta^n$ using pushouts. The latter is trivial (evaluation at the terminal object) and so is the former in the inner anodyne case. Colimits indexed by $\Lambda_2^2$ are trivial and those indexed by $\Lambda_0^2$ are literally pushouts. Higher horns are built from lower ones using pushouts and cones as $\Lambda^{n+1}_k = \Delta^0 * \Lambda^n_k \cup_{\Lambda^n_k} \Delta^n$, so it now suffices to observe that $A \to \Delta^0 * A$ is final when $A$ is weakly contractible. So inductively we do get all these colimits.