Set $f_k(x)= f(x,k)$ so we think of $f(x,k)$ as a $1$-parameter family of functions depending on $x$. Assume $f_k$ is strictly convex for each $k$, i.e.,
$$
f''_k(x)>0,\;\;\forall x.k.
$$
We denote by $f^*_k$ its Legendre transform
$$
f_k^*(\xi) =\sup_x\big(x\xi-f_k(x)\,\big).
$$
Then $v(k)=f^*_k(1)$. Assume that $f'_k(0)<1$. This implies that $x^*(k)\in(0,\infty)$ so $f'_k(x^*)=1$. Note that
$$v(k)=x^*-f_k(x^*).$$
$\newcommand{\pa}{\partial}$
We deduce that
$$
0=\pa_k\big(f'_k(x^*)\big)=f''_k(x^*)\frac{ dx^*}{dk}+ \frac{\pa f'_k}{\pa k}(x^*),
$$
so that
$$
\frac{ dx^*}{dk}=-\frac{\frac{\pa f'_k}{\pa k}(x^*)}{f''_k(x^*)}
$$
Hence the condition $\frac{ dx^*}{dk}<0$ is achieved if $f'_k=\frac{df_k}{dx}$ is an increasing family of functions, i.e.
$$
\frac{\pa f'_k(x)}{\pa k}>0,\;\;\forall x,k\geq 0.
$$
I will assume this is the case. Next observe that
$$
v'(k)=\frac{\pa}{\pa k}\Big(x^*-f_k(x^*)\Big)=\frac{dx^*}{dk}-\Big( f'_k(x^*)\frac{ dx^*}{dk}+ \frac{\pa f_k}{\pa k}(x^*)\Big)
$$
($f'_k(x^*)=1$)
$$
= \frac{dx^*}{dk}-\Big( \frac{ dx^*}{dk}+ \frac{\pa f}{\pa k}(x^*)\Big)= -\frac{\pa f}{\pa k}(x^*).
$$
The last term is negative if
$$
\frac{\pa f_k}{\pa k} >0,\;\;\forall x,k\geq 0.
$$
and I will assume this is the case. Derivating the last equality with respect to $k$ we deduce
$$ v''(k)=-\frac{\pa^2 f}{\pa^2 k}(x^*)-\Big(\frac{\pa}{\pa k}f'_k(x^*) \Big)\frac{dx^*}{dk}=-\frac{\pa^2 f}{\pa^2 k}(x^*)+\frac{\pa}{\pa k}f'_k(x^*)\frac{\frac{\pa f'_k}{\pa k}(x^*)}{f''_k(x^*)}
$$
Thus, if $v''(k)<0$, then $f_k(x)$ must be strictly convex in $k$.
We seek $f_k(x)=a(k)+u(k)e^x$. The condition $f'_k(0)<1$ implies $u(k)<1$. Next, $\pa_kf'_k(x)>0$ means $u'(k)>0$. Finally
$$
-\frac{\pa^2 f}{\pa^2 k}(x^*)+\frac{\pa}{\pa k}f'_k(x^*)\frac{\frac{\pa f'_k}{\pa k}(x^*)}{f''_k(x^*)}=-a''(k)-u''(k)+\frac{(u'(k)^2}{u(k)}
$$
$$
=-a''(k)-\frac{u'(k)^2-u''(k)u(k)}{u(k)}=-a''(k)+\frac{u'(k)^2}{u(k)}\frac{u'(k)^2-u''(k)u(k)}{u'(k)^2}
$$
$$
=-a''(k)+\frac{u'(k)^2}{u(k)}\frac{d}{dk}\frac{u(k)}{u'(k)}
$$
Now try
$$
u(k)=\frac{k+1}{k+2}.
$$
Then
$$u'(k)=\frac{1}{(k+2)^2},\;\;\frac{u(k)}{u'(k)}=(k+1)(k+2)$$
$$
\frac{d}{dk}(k+1)(k+2)=2k+3,
$$
$$
\frac{u'(k)^2}{u(k)}=\frac{1}{(k+1)(k+2)^3}
$$
$$
\frac{u'(k)^2}{u(k)}\frac{d}{dk}\frac{u(k)}{u'(k)}=\frac{2k+3}{(k+1)(k+2)^3}=\frac{2}{(k+2)^3}+\frac{1}{(k+1)(k+2)^3}.
$$
Now choose $a(k)=e^k$. The function
$$
f_k(x)=e^k+\frac{k+1}{k+2}e^x
$$
will do the trick.