$\newcommand{\Diff}{\operatorname{Diff}}$<!--
-->
First let $M$ be closed. Then $\Diff^\infty(M)$ is locally modelled on the space of smooth vector fields in $M$ which is a Frechet space. All Frechet spaces are homeomorphic. So this is reduced to the case you described. The same is true for $\Diff^k(M)$.

Let $M$ be open (non-compact without boundary). Then $\Diff^\infty(M)$ is open in $C^\infty(M,M)$ in the Whitney $C^\infty$ topology. But it is not locally contractible in this topology, and the arc connected component of the identity is contained in the group $\Diff^\infty_c(M)$ of diffeomorphisms which differ from the identity only on a compact set.
The group $\Diff^\infty_c(M)$ is a regular Lie group locally modeled on the space of $\mathfrak X_c(M)$  smooth vector fields with compact support which is a nuclear (LF)-space. 
Similar for $\Diff^k(M)$. I do not remember whether $\mathfrak X_c(M)$ is a CW-complex.

See [[Peter W. Michor: Manifolds of differentiable mappings. Shiva Mathematics Series 3, Shiva Publ., Orpington, (1980), iv+158 pp., MR 83g:58009, ZM 433.58001]][1]   
for lots of details on this, including the case of manifolds with boundary and with corners. 
See also section 41 of [[here]][2].


  [1]: http://www.mat.univie.ac.at/~michor/manifolds_of_differentiable_mappings.pdf
  [2]: http://www.mat.univie.ac.at/~michor/apbookh-ams.pdf