We perform a detailed study of the relaxation towards equilibrium in the Hamiltonian Mean-Field (HMF) model, a prototype for long-range interactions in $N$-particle dynamics. We point out in particular the role played by the infinity of stationary states of the associated $N o infty$ Vlasov dynamics. In this context, we derive a new general criterion for the stability of any spatially homogeneous distribution, and compare its analytical predictions with numerical simulations of the Hamiltonian, finite $N$, dynamics. The system first shows a rapid convergence towards a stationary state of the Vlasov equation. We numerically characterize this dynamical instability in the finite $N$ system by introducing appropriate indicators. This first step of the evolution towards Boltzmann-Gibbs equilibrium is followed by a slow (quasi-stationary) process, that proceeds through different stable stationary states of the Vlasov equation. When starting from a Vlasov stable homogenous initial state, the finite $N$ system remains trapped in a quasi stationary state for times that increase with the nontrivial power law $N^{1.7}$. Single particle momentum distributions in such a quasi-stationary regime cannot be fitted by the single particle Tsallis distributions used in Latora, Rapisarda, Tsallis, Phys. Rev. E extbf{64}, 056134 (2001).