Here is an idea, though I have to make some assumptions.

Suppose you have two sets of points $p_1,...,p_n\in\Bbb R^d$ and $q_1,...,q_n\in\Bbb R^d$ (for example, the vertices of your polyhedra, but with a fixed order).

Assume that they are translated to be centered at the origin, i.e. $p_1+\cdots +p_n=0$, and respectively for the $q_i$, so that we can ignore translations.

In a first step you could compute the covariance matrices of both point clouds and compare them. That is

$$C_p:=\sum_{i=1}^n p_ip_i^\top,\quad C_q:=\sum_{i=1}^n q_i q_i^\top.$$

These are positive semi-definite matrices, and you can compare their lists of eigenvalues, say $\lambda_i^p$ and $\lambda_i^q$ for all $i\in\{1,...,n\}$, sorted in descending order. They tell you about how unevenly these points clouds are distributed direction-wise.

The next step is to remove this unevenness from the point clouds. If we assume that the point clouds are full-dimensional (i.e. $\mathrm{span}(p_1,...,p_n)=\Bbb R^d$), then we can define

$$p_i':=C_p^{-1/2} p_i,\qquad q_i':=C_q^{-1/2} q_i.$$

Both point sets can now no longer be distinguished by translations or directional unevenness. The last step is to define, what I call, their arrangement matrices

$$ A_p := \begin{pmatrix} - & p_1' & - \\ & \vdots & \\ - & p_n' & - \end{pmatrix}, \qquad A_q := \begin{pmatrix} - & q_1' & - \\ & \vdots & \\ - & q_n' & - \end{pmatrix} $$

Compute $\delta:=\det(A_p^\top A_q)$. Geometrically, this value is related to the angle between the linear subspace $\mathrm{span}(A_p)$ and $\mathrm{span}(A_q)$. We can use it as follows:

• if $\delta=\pm1$, then the point clouds are just reorientations of each other, that is, there exists an orthogonal matrix $X\in\mathrm{O}(\Bbb R^d)$ with $\det(X)=\delta$ and $p_i=X q_i$ for all $i\in\{1,...,n\}$.
• the further $|\delta|$ is from one, the more different these point sets are.

In the end you have to somehow use the numbes $\delta,\lambda_i^p,\lambda_i^q$ for $i\in\{1,...,n\}$ to quantify the difference between the point sets. I do not have a recipe for this. All I can tell you is, that if $\lambda_i^p=\lambda_i^q$ for all $i\in\{1,...,n\}$ and if $\delta=\pm 1$, then these point sets are the same up to some (possible orientation reversing) orthogonal transformation.

This of course assumes that your point sets have a predefined order (which might be given by the isomorphism between your polyhedra).

Here is an idea, though I have to make some assumptions.

Suppose you have two sets of points $p_1,...,p_n\in\Bbb R^d$ and $q_1,...,q_n\in\Bbb R^d$ (for example, the vertices of your polyhedra, but with a fixed order).

Assume that they are translated to be centered at the origin, i.e. $p_1+\cdots +p_n=0$, and respectively for the $q_i$, so that we can ignore translations.

In a first step you could compute the covariance matrices of both point clouds and compare them. That is

$$C_p:=\sum_{i=1}^n p_ip_i^\top,\quad C_q:=\sum_{i=1}^n q_i q_i^\top.$$

These are positive semi-definite matrices, and you can compare their lists of eigenvalues, say $\lambda_i^p$ and $\lambda_i^q$ for all $i\in\{1,...,n\}$, sorted in descending order. They tell you about how unevenly these points clouds are distributed direction-wise.

The next step is to remove this unevenness from the point clouds. If we assume that the point clouds are full-dimensional (i.e. $\mathrm{span}(p_1,...,p_n)=\Bbb R^d$), then we can define

$$p_i':=C_p^{-1/2} p_i,\qquad q_i':=C_q^{-1/2} q_i.$$

Both point sets can now no longer be distinguished by translations or directional unevenness. The last step is to consider the correlation matrix

$$C_{pq}:=\sum_{i=1}^n p_i'q_i^{\prime \top}.$$

You could e.g. compute $\delta:=\det(C_{pq})$. This values lies between $-1$ and $1$. We can use it as follows:

• if $\delta=\pm1$, then the point clouds are just reorientations of each other, that is, there exists an orthogonal matrix $X\in\mathrm{O}(\Bbb R^d)$ with $\det(X)=\delta$ and $p_i=X q_i$ for all $i\in\{1,...,n\}$.
• if $\delta=0$, then these point sets are as distinct as possible.
• in general, the smaller the value of $|\delta|$, the more different these point sets are.

In the end you have to somehow use the numbes $\delta,\lambda_i^p,\lambda_i^q$ for $i\in\{1,...,n\}$ to quantify the difference between the point sets. I do not have a recipe for this. All I can tell you is, that if $\lambda_i^p=\lambda_i^q$ for all $i\in\{1,...,n\}$ and if $\delta=\pm 1$, then these point sets are the same up to some (possibly orientation-reversing) orthogonal transformation.

This of course assumes that your point sets have a predefined order (which might be given by the isomorphism between your polyhedra).