In this note we establish some rigidity and stability results for Caffarelli's log-concave perturbation theorem. As an application we show that if a 1-log-concave measure has almost the same Poincaré constant as the Gaussian measure, then it almost splits off a Gaussian factor.