These are really two questions but I hope that the same method will solve both of them.
For the purpose of this question let us fix a real closed field $R$, a bounded semialgebraic set $X$ over $R$, $A\subseteq X$ an algebraic subset (that is a subset described by the zero locus of some polynomials) and $Y$ a smooth algebraic variety over $R$. In fact I only care about the case $X=\{(t_0,\dots,t_n)\in R^{n+1}\mid \sum_i t_i=1; t_i\ge 0\}$ and $A=\bigcup_i \{t\in X\mid t_i=0\}$.
Q1: Let $R\subseteq R'$ be an inclusion of real closed fields and $f:X_{R'}\to Y(R')$ be a continous semialgebraic map such that the restriction to $A$ is defined over $R$. Is there an homotopy relative to $A$ to a map defined over $R$? That is, does there exists a continous semialgebraic map $$ H:X_{R'}\times [0,1]_{R'}\to Y(R') $$ such that $H|_{A_{R'}\times [0,1]_{R'}}$ factors through $A_{R'}$, $H|_{X_{R'}\times\{0\}}=f$ and $H_{X_{R'}\times\{1\}}$ is defined over $R$?
Q2: Assume that $R=\mathbb{R}$, the field of real numbers and let $f:X\to Y(\mathbb{R})$ be a continous map such that $f|_A$ is semialgebraic. Does there exists an homotopy $H$ relative to $A$ to a semialgebraic map?
I think that by a standard tubular neighbourood argument in both cases it is enough to prove that $f$ can be approximated by semialgebraic maps coinciding on $A$ in some compact-open topology.