Stealing an answer from "user940" on Math.SE, the answer is yes, such measures can exist. In the paper
Davies, Roy O., Measures not approximable or not specifiable by means of balls, Mathematika, Lond. 18, 157-160 (1971). ZBL0229.28005.
the author constructs a compact metric space $X$ and two distinct Borel probability measures $\mu_1, \mu_2$ that agree on every closed ball. (They must therefore also agree on open balls, because an open ball is a countable increasing union of closed balls.) Taking $\mu = \mu_1 - \mu_2$ provides your desired signed measure.