In my language, a differential two-form on $\mathbb{R}^4$ (viewed as a differentiable manifold with coordinates $t,x,y,z$) is a differentiable choice at each point of an alternating bilinear function from the tangent space at that point to $\mathbb{R}$, or equivalently something of the form $a_{xy} dx \wedge dy + a_{yz} dy \wedge dz + a_{zx} dz \wedge dx + a_{tx} dt \wedge dx + a_{ty} dt \wedge dy + a_{tz} dt \wedge dy$. A one-form just gives a cotangent vector. The most natural operation on a two-form is to take two tangent vectors at a given point and to apply the function to them to get a number. Another natural operation is to integrate this over an oriented surface in $\mathbb{R}^4$. As well, divergence is a natural operation on two-forms, whereas curl is a natural operation on one-forms, even though physicists often take the curl or the divergence of the same type of quantity, which they call a "vector field" (which at least seems bad from a mathematician's perspective).
We also have a metric on $\mathbb{R}^4$ (known by the name Lorentz), meaning a symmetric bilinear function on the tangent space at each point (that varies differentiably). Since $\mathbb{R}^4$ is a linear and homogenous space, we can actually think of that form as a form on space (manifold) as a whole. This means that there is a natural identification between tangent and cotangent vectors at each point, implying an identification between two-forms on the $T_p(\mathbb{R}^4)$ and elements of $T_p(\mathbb{R}^4) \otimes_{\mathbb{R}} T_p(\mathbb{R}^4)$ at each point $p$, and this identification extends differentiably to differential two-forms. Well, okay, it isn't really "natural" in the purely categorical sense, but because we have specified a metric, it kind of is.
This means that Lorentz transformations, ones which preserve the metric, commute with the identification between two-forms and elements of $T_p(\mathbb{R}^4) \otimes_{\mathbb{R}} T_p(\mathbb{R}^4)$ (equivalently, bilinear functions on the cotangent space). That is, if we have a Lorentz transformation sending $p$ to $q$, then we have a commutative diagram where the vertical arrows are the isomorphisms between the tangent and cotangent spaces (or more specifically their second tensor powers) at $p$ and $q$ respectively, and the horizontal arrows are the morphism and comorphism induced on the cotangent spaces of the Lorentz transformation (which is a diffeomorphism).
I think it also means something along the lines of the idea that a Lorentz transformation preserves the "form" of the equation, meaning that if you apply the Lorentz transformation as a change of coordinates, the coordinate-independent object (say the bilinear function) has another set of coordinates, but because we are using a Lorentz transformation, the coordinates are the same.
Diffeomorphisms which are not Lorentz transformations still send tangent and cotangent vectors at points to the same things at other points in a functorial way (specifically, a functor from the category of pointed differentiable manifolds to the category of vector spaces), but they don't necessarily satisfy this nice commutativity.
So could someone please explain electromagnetism in a simple way in this language? Also, how does it relate to the classical picture? E.g. does it look something like $E_x dy \wedge dz + \cdots + B_y dt \wedge dy + B_z dt \wedge dz$ or whatever? (This is wrong, but it's the kind of answer I'm looking for.) Once we have, say, a two-tensor, what natural operations (in the mathematical sense) can we do to it to get the basic physical quantities? E.g., if we have a two-form, what two vectors (or vector fields) do we plug into it? (I mean, that's what a two-form is made for - it's a bilinear function that you can plug two tangent vectors into! That and you can also integrate it and take its divergence.) How do Maxwell's equations work in this context?