Nice question! First a comment about terminology. The object you are considering is a "balanced frequency square". Frequency squares generalise Latin squares in the sense that each row and column is a permutation of the same multiset of symbols. The term "balanced" means that each symbol occurs the same number of times.
The question as asked has a negative answer. For example, take b=2 and a to be odd. Let X be the addition table for integers modulo a, and let Y be obtained by adding a to each entry of X. Form F by, initially juxtaposing these blocks:
XY
YX
Now switch the principal entries of these four blocks to make F. For example, when a=3 then F is
312045
120453
201534
045312
453120
534201
Consider an axa submatrix S of this frequency square. Since a is odd, S must include more than half the rows and more than half of the columns of one of the blocks. If S misses the principal entry of the block, that means it will include all a of the symbols that are native to the block, and that leads easily to the conclusion that S cannot be a Latin square.
If S hits the principal entry it is a little more fiddly, but not too hard to show that S won't be a Latin square.
-Ian