i found quite a satisfying answer to this question which gives enough trails for further enquiry. the answer involves exterior algebra and gradation (check also hodge duality).

> if $\mathrm A\colon V \to V$ is a linear map, then it naturally induces a graded operator $\mathrm A^*$ in the exterior algebra of $V$. when $V$ has finite dimension, there are two pieces of the exterior algebra which are isomorphic to $V$, namely $\Lambda^1(V)$, the dual space of $V$, and $\Lambda^{n-1}(V)$, being the space of $(n-1)$-multivectors of $V$.

> the restriction of $\mathrm A^*$ to $\Lambda^1(V)$ is the *dual map* (also called *adjoint map*). generally the dual space is not canonically isomorphic to $V$ if it has only linear structure; this changes when some sort of duality is established thanks to an extra structure, e.g. inner product. this is the adjoint linear map acting in $V$ with matrix $\mathbf A^\mathrm T$ depending on the choice of basis.
> on the other hand $\Lambda^{n-1}(V)$ is naturally isomorphic to $V$ (the isomorphism is given by the determinant), and so the restriction of $\mathrm A^*$ to $\Lambda^{n-1}(V)$ naturally induces an linear map on $V$ denoted $\operatorname{adj} \mathrm A$ with matrix $\mathbf A^\mathrm D$. since this is natural, an matrix adjugate can be defined in terms of this.

> both notions of adjointness (classical and conjugate transpose) are quite related. the simple one is just duality and makes more sense for operators in general vector spaces (or better in inner product space). the seemingly not-so-simple one is natural, makes sense for operators or matrices, is usually expressed in terms of matrices, and appears often in relation to quadratic forms.

<sub>source: [google groups on sage developement](https://groups.google.com/d/msg/sage-devel/YjImMWVVwo4/5ok9dcxrZFMJ)</sub>