Here are some remarks on the answers of Charles Matthews, Kevin Buzzard, and Victor Protsak. For justification of Kevin Buzzard's claim, see G. Prasad, "An elementary proof of a theorem of Bruhat-Tits-Rousseau and of a theorem of Tits" from Bull. Soc. Math. France 110 (1982), pp. 197--202, for an incredibly elegant and short proof that over any henselian valued field $F$, a connected reductive $F$-group $G$ is $F$-anisotropic if and only if $G(F)$ is "bounded" (a property defined in terms of a choice of closed immersion of $G$ into an affine space over $F$, the choice of which doesn't matter; this is meaningful for any affine $F$-scheme of finite type and equivalent to compactness when $F$ is locally compact). Platanov-Rapinchuk has a universal assumption that all fields of characteristic 0 (except when they're finite), so unfortunately that reference is insufficient for uniform arguments over all non-archimedean local fields. I suppose (near?-)circularity (suggested by Victor Protsak) is a more serious issue. :)
There remains the matter of determining, for locally compact non-archimedean $F$ and connected reductive $F$-groups, precisely when the $F$-anisotropic case can actually occur. As Victor Protsak mentioned, via Bruhat-Tits theory one sees that for connected semisimple $F$-groups which are absolutely simple and simply connected over a non-archimedean local field $F$ (i.e., $G$ a simply connected $F$-form of a Chevalley group), such forms never exist away from type A, and in type A the $F$-anisotropic examples are precisely the $F$-groups of norm-1 units of central simple algebras over $F$. (Note the contrast with the case $F = \mathbb{R}$, for which there's always a "compact form" of any Chevalley type.)
Let me now briefly explain why this handles the general connected reductive case, by a standard kind of argument with central isogenies and separable Weil restriction. (This is explained also in the article [2] of Tits referenced in Charles Matthews' answer.) If $f:G' \rightarrow G$ is a (possibly inseparable) central $F$-isogeny between connected reductive $F$-groups then the preimage of an $F$-torus of $G$ is an $F$-torus of $G'$ (since maximal tori in $G'$ are their own functorial centralizers, so $\ker f$ is of multiplicative type, nothing funny happens when $f$ is not separable). Since $F$-anisotropcity of an $F$-torus is invariant under $F$-isogenies (as we see using the $F$-rational character group, or more direct arguments), it follows that $G$ is $F$-anisotropic if and only if $G'$ is. (This argument has the advantage of working over any field $F$, in contrast with a direct attack on the topology of rational points by using finiteness theorems for Galois cohomology of connected reductive groups.) Thus, by considering an arbitrary connected reductive $F$-group $G$ and letting $G'$ denote the product of its maximal central $F$-torus and the simply connected central cover of the derived group $D(G)$, we see that the problem comes down to the simply connected case.
But in the simply connected semisimple case, the general structure of connected semisimple groups over fields (in terms of central isogenous quotient of direct product of commuting simple "factors") implies that $G$ is uniquely a direct product of commuting $F$-simple connected semisimple $F$-groups, each of which is simply connected, so we may assume $G$ is $F$-simple. Then by an elementary result of Borel and Tits (6.21 in "Groupes reductif", IHES), $G = {\rm{Res}}_ {F'/F}(G')$ for a finite separable extension $F'/F$ and a connected semisimple $F'$-group $G'$ that is absolutely simple and simply connected. By the good behavior of Weil restriction with respect to the formation of the topological group of rational points, it follows that the equality ${\rm{Res}}_ {F'/F}(G')(F) = G'(F')$ of abstract groups is a homeomorphism, so we can replace $(G,F)$ with $(G',F')$ to reduce to the case when $G$ is also absolutely simple, the case addressed by Victor Protsak above.
(A more algebraic argument with Galois descent relating maximal $F'$-tori in $G'$ and maximal $F$-tori in its Weil restriction to $F$ shows the equivalence of anisotropicity for $G'$ and its Weil restriction through the finite separable $F'/F$, where $F$ can be taken to be any field at all.)
Conclusion: for non-archimedean local $F$, the $F$-anisotropic connected reductive $F$-groups are precisely the central quotients of products of an $F$-anisotropic torus and groups of norm-1 units of central division algebras over finite separable extensions of $F$.