$\zeta^{(k)}(s) < 0$ for $s\in (0,1)$ A bit of plotting suggests that $\zeta^{(k)}(s) < 0$ for all $s\in (0,1)$ and all integers $k\geq 0$. (Or, what is the same: $\zeta^{(k)}(s)$ has no zeroes on $(0,1)$.) Is there a brief, clean proof of this apparent fact (and/or a reference for it)?
 A: Here is a proof that $\zeta^{(i)}(s) < 0$ for all $s \in [0,1[$ and $i \in \mathbf N$ (I can't say whether it counts as brief and clean, though). 

We'll use that the Riemann zeta can be expanded as a Laurent series about $s = 1$, so that
$$\tag{1} \zeta(s) = \frac{1}{s-1} + \sum_{n =0}^\infty \frac{(-1)^n \gamma_n}{n!} (s-1)^n,\quad\text{for all }s \ne 1,$$
where $\gamma_n$ is the $n$-th Stieltjes constant. We'll also need an inequality of A.F. Lavrik from 

On the main term of the divisor's problem and the power series of the Riemann's zeta function in a neighbourhood of its pole, Trudy Mat. Inst. Akad. Nauk. SSSR 142 (1976), 165-173 (in Russian),

which yields
$$
\tag{2}|\gamma_n| \le \frac{n!}{2^{n+1}},\quad\text{for all }n \in \mathbf N^+.
$$
In particular, (2) implies that the series on the right-hand side of (1) is absolutely convergent in the interval $[-a,a]$ for every $a \in [0,1[$. 
With this in mind, let $k \in \mathbf N^+$. We have
$$
\zeta^{(k)}(s) = - \frac{k!}{(1-s)^{k+1}} + (-1)^k \sum_{n\ge k} \frac{\gamma_n}{(n-k)!} (1-s)^n,\quad\text{for all }s \in {]-1,1[}\,,
$$
and hence
$$
\tag{3}\zeta^{(k)}(0) = - k! + (-1)^k \sum_{n\ge k} \frac{\gamma_n}{(n-k)!}.
$$
We claim 
$$
\tag{4}\sum_{n \ge k} \frac{\gamma_n}{(n-k)!\,k!} < 1.
$$
Indeed, a classical result from Section 1 of

W.E. Briggs, Some Constants Associated with the Riemann Zeta-Function, Mich. Math. J. 3 (1955), No. 2, 117-121, 

gives that $\gamma_n < 0$ for infinitely many $n$. Therefore, it is sufficient for (4) to hold that
$$
\sum_{n \ge k} \frac{|\gamma_n|}{(n-k)!\,k!} \le 1.
$$
This, in turn, follows from (2) and the fact that
$$
\sum_{n \ge 0} \frac{1}{2^{n+1}} \binom{n}{m} = 1, \quad\text{for all }m \in \mathbf N
$$
(see Noam Elkies's comment below). By (3) and the considerations made by Gerald Edgar in the comments to the OP, we can thus conclude that $\zeta^{(i)}(s) < 0$ for all $s \in [0,1[$ and $i \in \mathbf N$ (recall that $\zeta(0) < 0$).
A: The coefficients computed in the comments appear to imply that the Taylor expansion at $s=0$ of $\zeta(s)+\frac1{1-s}-\frac12$ has very small coefficients, which would imply the result.
Following section 2.1 in Titchmarsh Theory of the Riemann zeta function, 
by integration/summation by parts (or one step of Euler-Maclaurin),
$$\zeta(s)=\frac1{s-1}+\frac12+s\int_1^{\infty}\frac{1/2-\{x\}}{x^{s+1}}dx$$
absolutely convergent for $\Re(s)>0$ where $\{x\}$ denotes the fractional part of $x$.
Integrating by parts again (or two steps of Euler-Maclaurin),
$$\zeta(s)=\frac1{s-1}+\frac12+\frac{s}{12}-(s+1)\int_1^{\infty}(\{x\}^2-\{x\}+\frac16)x^{-s-2}dx$$
Consider the integrand $(\{x\}^2-\{x\}+\frac16)x^{-s-2}dx$ as a function of $s$.  Its Taylor coefficients are alternating in sign and dominated by the coefficients of $\frac16x^{-s-2}$, so the Taylor coefficients of the absolutely convergent integral are dominated by those of
$$\int_1^{\infty}\frac16x^{-s-2}dx = \frac1{6(s+1)}$$
and the result follows.
