I want to know if there is complete theory on real representation of finite groups.
Say, given a finite group G, can we know all the injections from G to GL(n,R) for a particular n?
Recall that if $G$ is a group, $k$ a field, and $V_k$ an irreducible representation of $G$ over $k$ then by Schur's lemma $\mathrm{End}_G(V_k)$ is a division ring $D$ over $k$. For example, if $V$ is a real representation then the endomorphism ring is $\mathbb{R}$, $\mathbb{C}$, or $\mathbb{H}$ (as these are the only division rings over $\mathbb{R}$. Such representations are typically called real, complex, or quaternionic.
Now Frobenius-Schur indicator theory explains how to determine which representations over $\mathbb{C}$ "come from" which kind of real representations. Explicitly, if V is not selfdual then it plus its dual is the complexification of a representation over $\mathbb{R}$ with $D=\mathbb{C}$; if V orthogonal (i.e. has an invariant bilinear form) then it is the complexification of a representation over $\mathbb{R}$ with $D=\mathbb{R}$; finally, if V is symplectic then it plus itself is the complexification of a representation over $\mathbb{R}$ with $D=\mathbb{H}$.
Finally the Frobenius-Schur indicator is given in terms of the character by a simple formula, namely the expression $1/|G| \sum_g \chi(g^2)$ is -1 if $\chi$ has an invariant symplectic form, 0 if $\chi$ is not self-dual, and 1 if $\chi$ has an invariant orthogonal form.
In principle, we can compute the character table of a finite group algorithmically. For example, [McKay, J. K. S. A method for computing the character table of a finite group. 1968 Computers in Mathematical Research pp. 140--148 North-Holland, Amsterdam MR0236278 (38 #4575)], [McKay, J. K. S. Algorithm 307. Comm. ACM 10, 7, (July 1967) 450-451.], Dixon, John D. High speed computation of group characters. Numer. Math. 10 1967 446--450. MR0224726 (37 #325)] and others; the ideas go back to Burnside, at least, it seems. McKay's program was used to compute the characters of $J_1$ and $J_3$ in all of 84 minutes at the time, with a whole 16K store (which is 12 times smaller than the size of the background image I am using as background to my desktop computer!)
Using a bit of character theory, as in Noah's answer, we can then tell which characters are afforded by real representations, and what are the real faithful representations.
The answer to your question is thus: yes.
All this is described very nicely in Serre's book on representations of finite groups.
Some of the other answers omit several details and I had already written this on another forum. At any rate, perhaps the example would be useful:
More or less yes, though for larger groups it is not very practical.
Calculate the character table of G. Use the Frobenius-Schur indicator to fix the complex and quaternionic irreducible characters to be characters of irreducible real representations. Take all non-negative integer combinations of these characters of real reps whose total degree is n and whose kernel is trivial (the degree and the kernel of a character are very easy to calculate). There is a 1-1 correspondence between injections of G into GL(n,R) up to GL(n,R) conjugacy and characters of real reps whose total degree is n and whose kernel is trivial.
Except for cyclic groups G and n=1, there will be uncountable many injections into GL(n,R), so you'll want to only consider them up to GL(n,R) conjugacy in order to get a finite number.
Also notice that if f1:G→GL(m1,R) is an injection and f2:G→GL(m2,R) is any group homomorphism, then another injection is "f1⊕f2":G→GL(m1+m2,R):g→[f1(g),0;0,f2(g)], the function that takes the two matrices from f1 and f2 and makes them the blocks of a block diagonal matrix. For this reason the number of injections, even up to conjugacy, can get unwieldy.
Example: The non-cyclic group of order four:
For G the Klein four-group the irreducible characters are all real, and are [1,1,1,1], [1,-1,-1,1], [1,-1,1,-1,], [1,1,-1,-1]. The degree of each of these is 1, the first number. The kernel of each of these are the positions where the value and the degree are equal, so all of them have a kernel. If n was 10, then I'd look at all the ways of choosing non-negative integers a,b,c,d such that a+b+c+d=10 (so that the degree was 10) and such that at least 2 of b,c,d were non-zero (so that the kernel was trivial). There are 282 such combinations, one of which is a=5, b=0, c=2, d=3. This corresponds to the character 5*[1,1,1,1]+0*[1,-1,-1,1]+2*[1,-1,1,-1]+3*[1,1,-1,-1] = [ 10, 6, 4, 0 ]. It has degree 10 (the first and largest positive number) and trivial kernel (all the other positive numbers are strictly smaller). It corresponds to the representation g1 → diag( 1,1,1,1,1, -1,-1, 1,1,1 ) and g2 → diag( 1,1,1,1,1, 1,1, -1,-1,-1 ) and of course g1*g2 → diag( 1,1,1,1,1, -1,-1, -1,-1,-1 ). This is 5 copies of the representation g→1, g2→1, followed by two copies of g1→-1, g2→1, and finally three copies of g1→1, g2→-1. Since b=0 we did not include any copies of g1→-1, g2→-1.
By some silly counting, the number of injections of the Klein four-group into GL(n,R) up to GL(n,R) conjugacy is (n^3+6*n^2+11*n-18)/6, so it grows somewhat quickly but not too crazily.
For non-abelian groups G, the conversion of the characters into the matrices is a little harder (for abelian groups, the matrices are all 1x1 and really are just the character values), but at some point the matrices are too big anyways, and the characters are a more efficient data type.