Suppose $A \subseteq \{1,\dots,n\}$ does not contain any arithmetic progressions of length $k+1$. What is the largest number of $k$term arithmetic progressions that $A$ can have? (one may also wish to put some lower or upper on the size of $A$) We can work over $\mathbb{Z}_p$ if it makes the answer any easier. The "degenerate" case $k=2$ asks for the largest size of the set without arithmetic progressions and it is known that there exist $A$'s with this property of almost linear size.

Let $B\geq2k$ and let $$A=\left\{\sum_{i=0}^na_iB^i:n=0,1,...;a_i=0,1,...,k1\right\}$$ It's not hard to show that $A$ has no $k+1$long arithmetic progression. Using the density HalesJewett theorem we get that any subset $B\subset A$ with positive relative density has a $k$long arithmetic progression. I don't know the best bounds on the density HalesJewett, but I think there are some from the polymath proof, so in principle this would give an answer to your question. 


I assume that $k$ as fixed. The answer to this problem is closely related to the maximum density $p=p_{k+1}(n)=r_{k+1}(n)/n$ of a subset of $\{1,\ldots,n\}$ without a $(k+1)$term arithmetic progression. Indeed, let $S \subset \{1,\ldots,n\}$ be a set of cardinality $r_{k+1}(n)$ without a $(k+1)$term arithmetic progression. Now construct a random subset $A \subset [4kn]$ as follows. The set $A$ consists of the union of $k$ random translations $S_i$ of $S$, where $S_i=S+d_i$ and $d_i$ for $1 \leq i \leq k$ is picked uniformly at random from the interval $\{4(i1)n,4(i1)n+1,\ldots,4(i1)n+2n1\}$ of $2n$ integers. It is easy to check that $A$ cannot contain a $(k+1)$term arithmetic progression. It is also not difficult to check that the expected number of $k$term arithmetic progressions in $A$ is at least $(cp)^k n^2$ where $c>0$ is an absolute constant. Hence, letting $N=4kn$ and changing $c$ slightly, there is a subset of $[N]$ with no $(k+1)$term arithmetic progression but the number of $k$term arithmetic progressions is at least $(cp)^k N^2$. In the other direction, any subset $B \subset \{1,\ldots,N\}$ with no $(k+1)$term arithmetic progression has density at most $2p$. A $k$term arithmetic progression is determined by its first two terms. The number of possible first two terms in $B$ is at most ${2pN \choose 2} < 2p^2N^2$. Hence, there are at most $2p^2N^2$ total $k$term arithmetic progressions in $B$. In summary, the number of $k$term arithmetic progression in $\{1,\ldots,N\}$ which can be in a set with no $(k+1)$term arithmetic progression is between $(cp)^k N^2$ and $2p^2N^2$. However, there is still a large gap between Rankin's lower bound and Gowers' upper bound on $p=r_{k+1}(n)/n$. Hence, the bound on this problem is closely tied to quantitative bounds for Szemerédi's theorem. 

