I agree with Ben that the question is a little confusing.
There are two possible questions:
How do you calculate direct summands of the direct image when we start with an IC (not necessarily the constant sheaf) on the source?
In this case I think the direct image need not be perverse. In which case you are in the general situation of describing the direct summands of a semi-simple complex, for which you need to know the characters of all IC's, or be very clever.
How do you calculate the local systems occurring in the direct image of the constant sheaf?
Ben describes what happens above. Semi-small means that the fibre over each stratum of the base has dimension bounded by half the codimension of the stratum (a number I will call d). The local system is then given by local system of 2d^th cohomology of the fibre.
In the semi-small situation this is beautiful: the 2d^th cohomology of the fibre is zero if the stratum isn't relevant, and has a basis given by fundamental classes of irreducible components if the stratum is relevant.
Note that the fundamental group of the stratum acts on the irreducible components of the fibre via monodromy, and this is precisely the local system that you get. (As an aside, this explains why the local systems are semi-simple, even though the fundamental group might be infinite: the representation always factors through the permutation group on the irreducible components.)
I first learnt this in the beautiful article "The Hard Lefschetz Theorem and the topology of semismall maps" by de Cataldo and Migliorini.