Here is an example that shows that you can expect things to get as bad as it goes (I learned about this algebra from the wonderful article The Non-Commutative Gröbner Freaks by Green, Mora, and Ufnarovski who attribute it to Backelin).
The algebra $A$ has three generators $x,y,z$, and impose the relations $x^2=0, xy+zx=0$. The algebra $B$ is the subalgebra of $A$ generated by $x,y$. To determine the defining relations of $B$, we may compute the Gröbner basis for the weighted graded lexicographic ordering where the weight of $z$ is $1$ and the weights of $x,y$ are $0$. Then the weight $0$ part of the Gröbner basis is clearly going to be the Gröbner basis of the ideal of relations of $B$. Now, an easy calculation shows that the reduced Gröbner basis for this ordering consists of the elements $$ xy+zx, \ xy^kx\ \ (k\ge 0), $$ and so the subalgebra $B$ is not only not quadratic, but its minimal set of relations is infinite.
EDIT: following a comment of BugsBunny, here is an example of a Koszul algebra $A$ and its non-quadratic subalgebra $B$: let $A$ have three generators $x,y,z$, and relations $xy-yx=z^2, yz-zy=x^2, zx-xz=y^2$. This algebra (known as one of the Sklyanin algebras) is Koszul (see Artin, Tate, van den Bergh, "Some Algebras Associated to Automorphisms of Elliptic Curves"), and its component of dimension $n$ has the same dimension as the component of dimension $n$ of the algebra of polynomials in three variables. However, if we consider the subalgebra $B$ generated by $x,y$, we see that it has no quadratic relations, so if it were quadratic, it would be free and hence would have exponential growth of components, which is a contradiction.
EDIT2: here is a commutative version of the previous result for the reader curious about subalgebras of commutative Koszul algebras. Let $A$ be the commutative algebra with three generators $x,y,z$, and relations $xy=z^2, yz=x^2, zx=y^2$. Then the reduced Gröbner basis of relations of $A$ for the ordering $x>y>z$ consists of the four elements $x^2 - yz$, $xy - z^2$, $xz - y^2$, $y^3 - z^3$, which instantly implies that the subalgebra $B$ generated by $y,z$ has just one relation $y^3=z^3$. In fact, the algebra $A$ can be shown to be Koszul (for example, its [noncommutative] Koszul dual algebra has a quadratic Gröbner basis).