So there is a paper A Hybrid GPU Rendering Pipeline for Alias-Free Hard Shadows

That claims to calculate the distance to a triangle $d(\omega,T)$ efficiently, they

resort to some tricks based on the concepts of barycentric coordinates

they describe the squared distance between a point $w$ and some vertex $v_i$ of the triangle $T$ as

$d(\omega,v_i)^2=\left\Vert \omega-v_{i}\right\Vert=\lambda_{i-1}^{2}\left\Vert e_{i-1}\right\Vert ^{2}+\lambda_{i+1}^{2}\left\Vert e_{i+1}\right\Vert ^{2}-2\lambda_{i-1}\lambda_{i+1}\left(e_{i-1}\cdot e_{i}\right)$

There is a derivation in the paper.

So there is a paper A Hybrid GPU Rendering Pipeline for Alias-Free Hard Shadows

That claims to calculate the distance to a triangle $d(\omega,T)$ efficiently, they

resort to some tricks based on the concepts of barycentric coordinates

they describe the squared distance between a point $w$ and some vertex $v_i$ of the triangle $T$ as

$d(\omega,v_i)^2=\left\Vert \omega-v_{i}\right\Vert=\lambda_{i-1}^{2}\left\Vert e_{i-1}\right\Vert ^{2}+\lambda_{i+1}^{2}\left\Vert e_{i+1}\right\Vert ^{2}-2\lambda_{i-1}\lambda_{i+1}\left(e_{i-1}\cdot e_{i}\right)$

There is a derivation in the paper.

I think using the notation you described it'd look something like this

$d(p,A)^2=\left\Vert p-A\right\Vert=w^2b^2+v^2a^2-2wv(CA\cdot AB)$

but I'd check the math just to be sure

So there is a paper A Hybrid GPU Rendering Pipeline for Alias-Free Hard Shadows

That claims to calculate the distance to a triangle $d(\omega,T)$ efficiently, they

resort to some tricks based on the concepts of barycentric coordinates

they describe the squared distance between a point $w$ and some vertex $v_i$ of the triangle $T$ as $d(\omega,v_i)^2=\left\Vert \omega-v_{i}\right\Vert=\lambda_{i-1}^{2}\left\Vert e_{i-1}\right\Vert ^{2}+\lambda_{i+1}^{2}\left\Vert e_{i+1}\right\Vert ^{2}-\lambda_{i-1}\lambda_{i+1}\left(e_{i-1}\cdot e_{i}\right)$

There is a derivation in the paper.

Note: this is meant for efficient computation in the context of a graphics pipeline. I'm having some trouble getting this to work as expected myself, which led me to this original question. So, I figured I'd just throw this out there, in case it makes more sense to anyone here.

So there is a paper A Hybrid GPU Rendering Pipeline for Alias-Free Hard Shadows

That claims to calculate the distance to a triangle $d(\omega,T)$ efficiently, they

resort to some tricks based on the concepts of barycentric coordinates

they describe the squared distance between a point $w$ and some vertex $v_i$ of the triangle $T$ as

$d(\omega,v_i)^2=\left\Vert \omega-v_{i}\right\Vert=\lambda_{i-1}^{2}\left\Vert e_{i-1}\right\Vert ^{2}+\lambda_{i+1}^{2}\left\Vert e_{i+1}\right\Vert ^{2}-2\lambda_{i-1}\lambda_{i+1}\left(e_{i-1}\cdot e_{i}\right)$

There is a derivation in the paper.