I apologize to Andrew Stacey in advance since what I was going to say is basically what he's saying.

Consider a finite-dimensional space. There's a notion of determinant of an operator (operator is defined as map $V \otimes V^* \to \mathbb R$). This is a simple notion, just a product of eigenvalues, or a trace on a one-dimensional space of $V^{top} \to V^{top}$.

Now you can find an isomorphism $V^* \to V$ for any non-degenerate scalar form $(\dot, \dot)$ — so there's a way to say that for pairing and scalar form there's a determinant that is a number. But still let's not think of it as a determinant of a pairing — it's really a determinant of an operator.

The situation is basically the same with infinite-dimensional spaces. For an operator $X$ you can define $\det X$ as — suitably regularized (for example, zeta-regularized) — product of its eigenvalues. For a pairing $A$ you can do that provided there's a scalar form fixed, that is whenever you have a Hilbert space.

I apologize to Andrew Stacey in advance since what I was going to say is basically what he's saying.

Consider a finite-dimensional space. There's a notion of determinant of an operator (operator is defined as map $V \otimes V^* \to \mathbb R$). This is a simple notion, just a product of eigenvalues, or a trace on a one-dimensional space of $V^{top} \to V^{top}$.

Now you can find an isomorphism $V^* \to V$ for any non-degenerate scalar form $(\dot, \dot)$ — so there's a way to say that for pairing and scalar form there's a determinant that is a number. But still let's not think of it as a determinant of a pairing — it's really a determinant of an operator.

The situation is basically the same with infinite-dimensional spaces. Select your favorite regularization scheme — a way to compute products of infinite series of numbers — (e.g. zeta-regularization) and define for an operator $X$ determinant $\det X$ as product of its eigenvalues, if it converges, or leave it undefined it still doesn't. For a pairing $A$ you can do that provided there's a scalar form fixed, that is whenever you have a Hilbert space.

You can think about wedge-top form here, but then you have to define wedge-top for an inifite-dimensional space. Sure, it can be done formally and you'll be able to define the action of operator on it by referring to the above paragraph for the meaning of det, but I don't know of any sufficiently different way to start from wedge-top and get det from there.

I'll look up the references. Note that the philosophy of physics is slightly different. They don't need to compute dets of arbitrary operators, rather they have some specific operators for which they know the answer must be finite and the fact that our initial computation gives infinity is because it just an approximation to the right calculation and a stupid approximation at that.