Replace $y$ by $y+t\delta y$ and then compute

$$\lim_{t\to 0} \Bigl(\; F(y+t\delta y)-F(y)\;\Bigr). $$

Here are the details. Observe first that

$$\frac{1}{y+ A+ t\delta y}-\frac{1}{y+A} = -\frac{1}{(y+A)^2} t\delta y+ O(t^2),$$

so that

$$F(y+t\delta y)= \left(\int\;\Bigl(\; \frac{1}{y+A} -\frac{1}{(y+A)^2} t\delta y+O(t^2)\;\Bigr) \cos(x) dx\right)^2 $$

$$= F(y) -2t\left(\int\frac{\cos (x)}{y+A}\right)\left(\int\frac{(\delta y)\cos(x)}{\bigl(y+A)^2} dx\right) +O(t^2). $$

From here you can read that

$$\frac{\delta F}{\delta y} =-2 \left(\int\frac{\cos (x)}{y+A}dx\right)\frac{\cos(x)}{\bigl(\;y(x)+A \;\bigr)^2} $$

**Update.** Here are some rules that will help you solve the problem you mentioned in your comment.

Suppose that $I$ is an interval.  If

$$F(y) =\int_I f(y(x)) w(x) dx.$$

Then

$$\frac{\delta F}{\delta y}= f'(y(x)) w(x). \tag{1}  $$

If   

$$ F(y)=\left| \int_I  f(y)  w(x) dx\right|^2, $$

$w$ complex valued, then $\DeclareMathOperator{\re}{\boldsymbol{Re}}$

$$\frac{\delta F}{\delta y}=  2\re\left(\; f'(y(x)) \overline{w(x)} \int_I f(y) w(x) dx \;\right) \tag{2} $$

Let me set 

$$ F_1(y)=\int_I \frac{A}{y(x)+ A} dx,\;\;I=(-\kappa \pi,\kappa\pi), $$

$$ F_2(y) =\left|\int_I \frac{A}{y(x)+ A} e^{- i x} dx\right|^2, $$

$$ E(y) = F_1-\frac{F_2}{F_1} $$

Then

$$ \frac{\delta E}{\delta y} =\frac{\delta F_1}{\delta y} -\frac{ \frac{\delta F_2}{\delta y} F_1- F_2\frac{\delta F_1}{\delta y} }{ F_1(y)^2 }. $$

Now compute the various derivatives using (1) and (2).   

Your problem reduces via Lagrange multipliers to the system

$$\frac{\delta E}{\delta y} =\lambda,\;\;\int_I y dx=1. $$