The effective dynamics of a colloidal particle immersed in a complex
medium is often described in terms of an overdamped linear Langevin
equation for its velocity with a memory kernel which determines the
effective (time-dependent) friction and the correlations of
fluctuations. Recently, it has been shown in experiments and numerical
simulations that this memory may depend on the possible optical
confinement the particle is subject to, suggesting that this description
does not capture faithfully the actual dynamics of the colloid, even at
equilibrium. Here, we propose a different approach in which we model the
medium as a Gaussian field linearly coupled to the colloid. The
resulting effective evolution equation of the colloidal particle
features a non-linear memory term which extends previous models and
which explains qualitatively the experimental and numerical evidence in
the presence of confinement. This non-linear term is related to the
correlations of the effective noise via a novel fluctuation-dissipation
relation which we derive.