If $M$ has boundary, then the fixed point set might contain arcs. One would have to account for the number of such arcs, and whether a fixed point arc connected two different boundary components or had both end points in the same boundary component. I don't think the corners are a problem. They would actually be useful for keeping track of where to re-attach the neighborhoods of the fixed point arcs.

I added some discussion about attaching the annuli. Hope that's a little clearer. I also added a note about the meaning of the mod 2 homology class represented by the fixed point set.