The interaction-driven transition between quantum spin Hall and Mott insulators in the Bernevig, Hughes, and Zhang model is studied by dynamical cluster approximation, and found to be accompanied by the emergence of Green's function zeros already in the quantum spin Hall regime. The nontrivial interplay between Green's function poles and zeros leads to an exotic quantum spin Hall insulator exhibiting two chiral branches of edge Green's function poles and one of zeros. When symmetry breaking is allowed, a nontopological excitonic insulator is found to intrude between quantum spin Hall and Mott insulators. We find evidence that excitons in the Mott insulator, which become soft at the transition to the excitonic insulator, are actually bound states between valence and conduction bands of Green's function zeros, rather than between lower and upper Hubbard bands.
