Let me answer the questions in order.

**a**) It was invented by Grothendieck, see EGA I, Springer edition, especially chapter 0, discussion of representable functors.

**b**) Quite possibly is a shortcut for $Hom$. Sometimes the letter $y$ is used (for Yoneda). The trouble is when you are considering the representable functor defined over several categories, e.g. a category and a subcategory.

Further evidence on **a**) The notation is already on SGA 3 and 4. There are several exposés by Grothendieck in Henri Cartan's seminar from 1960/61 in which he explains his point of view of Teichmüller's space through representable functors in the analytical category and he uses the notation $h_X$.

I an not aware of anyone else using these ideas at that time. Cartan's seminar is available at numdam:

http://www.numdam.org/article/SHC_1960-1961__13_1_A7_0.pdf

See also Bourbaki seminar, exposé 195 (February 1960)

http://www.numdam.org/article/SB_1958-1960__5__369_0.pdf

**Bonus**: If you, instead of considering contravariant functors $\mathrm{Sch}^{o} \to \mathrm{Set}$, use covariant functors $\mathrm{Aff} \to \mathrm{Set}$ the notation used in EGA is $h_X^{o}$. Perhaps the reason is that Yoneda's map is contravariant in this case.