Questions about distributions, usually involving the $\delta$ distribution ("function"), are of frequent occurrence on this and related sites.  Since they are often treated in a rather cavalier fashion, I would like to attempt to answer this query in some detail.
The basic problem lies in the interpretation of the functions $\frac 1 x$ and $\log x$
(notabene not $\log |x|  $) as distributions.  As a preliminary remark, it is not surprising that theoretical physicists are guided by their physical intuition and not by mathematical rigour in dealing with such questions and it is perfectly acceptable for mathematicians to proceed in the same way in  formulating such conjectures.  However, since this is a mathematical forum, one does have the right to expect that the final formulation of the solution conforms to the usual standards of mathematical rigour (as is implied in the OP).  In fact it is rather disquieting that this is often not the case, since this task can be achieved by elementary methods which have been on record in the primary and secondary literature for at least 50 years.

For the example in question (and, indeed, for most of the examples in such forums), one need only be cognisant of the following simple facts about distributions.

$1$.  Every continuous function on the reals, better, every locally integrable function, determines in a natural way a distribution.

$2$.  There is a notion of convergence for distributions.  For our purposes, it suffices to know that if a sequence of continuous functions converges uniformly on compacts, then it converges in the sense of distributions. In fact, local $L^1$ convergence suffices.

$3$.  The dream theorem of every freshman calculus student holds---if a sequence of distributions converges, then the sequence obtained by differentiating term by term also converges.

We can now turn to the above query.  Note that the problem lies in the fact that while the function $\log |x|$ determines a distribution, the same is not true a priori for $\frac 1  x$ and $\log x $.  In these two cases, we have to proceed in a more delicate manner.

Firstly, note that the function $\log |x|$ is locally integrable and so is a distribution.  More importantly the same is true for its derivative in the distributional sense.  It is then rather natural to define this derivative to be the distribution $\frac 1 x$.

The case of the distribution $\log x$ is rather more subtle. In this case we resort to the complex logarithm.  
thus we define, for non-zero $\epsilon $, the distribution $\log(x+i\epsilon)$ to be $\log|x|+i \arctan \frac \epsilon x$ (i.e., we are using the principal branch of the complex logarithm).  We now define the distribution $\log x $ to be the limit of this distribution as $\epsilon$ tends to zero.  At this point, we see that we get different values, depending on whether we consider the limit $\epsilon \to 0_+$ or $\epsilon \to 0_-$.  If we now differentiate this equation we obtain the required formula.

To conclude, a few remarks.

The above approach is due to the portuguese mathematician J. Sebastião e Silva who developed it in the context of his axiomatic approach to the theory of distributions.   
Sadly, the definitive monograph on his approach that he was preparing 40 years ago was never completed, due to his premature passing.                                         
However, the elementary part has been presented in the book "An introduction to the theory of distributions" by Campos Ferreira which is based on lectures at the University of Lisbon (ca.  1970) and this contains the material described here.

The above is a special case of the family of distributions $x^\lambda$ for general $\lambda$ (even complex)  which is developed in detail in the above reference.  They
are also described in the standard four volume text of 
Gelfand and 
Silov.

The distribution $\frac 1 x$ is also treated by Laurent Schwartz who used the Hadamard principal value
(the latter was his great uncle by marriage).  The connection with the above approach can easily be established by considering the truncated function $\log_\epsilon$ (which is set equal to zero on the interval $]-\epsilon,\epsilon[$), letting $\epsilon$ tend to zero and differentiating.

The ambiguity in the definition of the distribution $\log x$ is no more disconcerting than that in the definition of the logarithm function for complex arguments.  In a more sophisticated approach this distribution woukd not be defined on the real line but on the 
natural domain of definition of the latter, i.e., on the unversal covering of the punctured plane.