This answer only goes half way, but I think it is worth pointing out that the inner workings of the six functor formalism have been studied a lot in the motivic literature.

*Motivic six functor formalisms:* The definite reference would be the thesis of Ayoub which you can find on his homepage. The formalism of cross functors (based on unpublished works of Voevodsky and Deligne) gives a general framework how to set up the whole six functor formalism. Another relevant reference is the work of Cisinski-Déglise on triangulated categories of mixed motives. They develop the framework further, with the notions of motivic categories fibered over the category of schemes.

My point is that the input to these frameworks can be things more general than triangulated categories, these frameworks work with stable model categories, $\infty$-categories or dg-categories (as long as you feed in the right data). In the motivic setting, the framework is usually applied to yield six functors for categories of motives. But if you check the validity of the axioms for étale sheaves or $\ell$-adic sheaves, the framework would give you dg-versions of the corresponding derived categories. You should also look at the mixed Weil cohomologies paper of Cisinski-Déglise: plugging in $\ell$-adic cohomology in their constructions gives you dg-versions of $\ell$-adic derived categories.

*A grain of salt:* The frameworks above do not (explicitly) produce dg-enhancements of the six functors. Four of the functors are easy to deal with because they are derived functors of functors on the level of abelian categories - so they have dg-enhancements. Note, however, the exceptional functors are only constructed on the triangulated level in the abovementioned references. But I think that these can be upgraded to dg-versions: e.g. the construction of $f_!$ via a colimit over the category of compactifications of $f$ and then using the adjoint $i_!$ for an open immersion and $p_\ast$ for a proper map should work in the corresponding dg-setting. Similarly, for the definition of $f^!$ one could apply a version of the $\infty$-categorial adjoint functor theorem, together with suitable compact generation properties. If it's possible, it's only going to be a matter of time before an $\infty$-version of these formalisms will be available in the motivic world.

*Constructibility conditions:* The frameworks above usually work with fairly big categories. However, in the paper of Cisinski-Déglise, you can find the description of compact objects in these big categories - they agree with the classically defined constructible objects. Moreover, under rather weak assumptions, the six functors also preserve compact objects - the framework above then gives you dg-versions of $\ell$-adic constructible sheaves.

*Another grain of salt:* Ok, this is all for the algebraic setting, working over schemes and such. If you are more interested in the locally compact topological setting, the literature mentioned above probably does not apply directly. However, all the techniques are there, and I am fairly sure that the framework can be adapted to this setting as well.