False. As we want a counterexample, we naturally start with the Petersen graph, P. Note that for any vertex v of P and non-adjacent edge uw of P there is a Hamiltonian path from v to w that does not use the uw edge. On the other hand, there is no Hamiltonian cycle in P.
Our graph G will have 10t+15 vertices (where t is some large number) and is obtained from P by subdividing each edge with a vertex and replacing each original vertex by t vertices, connected to each other and to the subdividing vertices which the original vertex was connected to. Let v be a vertex that was not obtained by subdividing an edge and let 2k+1=9t+16 (or +17, whichever is odd). Using the above mentioned properties of P, it is easy to see that G satisfies the requirements if t is big enough.
This graph is even 3-connected and can be generalized to give even stronger counterexamples.
Note that my original construction used t=2, which is not sufficient for the above reasoning, as was noted in the comment by Dani.