Strong correlation effects on topological quantum phase transitions in three dimensions

We investigate the role of short-ranged electron-electron interactions in a paradigmatic model of three-dimensional topological insulators, using dynamical mean-field theory and focusing on nonmagnetically ordered solutions. The noninteracting band structure is controlled by a mass term M, whose value discriminates between three different insulating phases, a trivial band insulator and two distinct topologically nontrivial phases. We characterize the evolution of the transitions between the different phases as a function of the local Coulomb repulsion U and find a remarkable dependence of the U-M phase diagram on the value of the local Hund's exchange coupling J. However, regardless of the value of J, following the evolution of the topological transition line between a trivial band insulator and a topological insulator, we find a critical value of U separating a continuous transition from a first-order one. When the Hund's coupling is significant, a Mott insulator is stabilized at large U. In proximity of the Mott transition we observe the emergence of an anomalous "Mott-like" strong topological insulator state. © 2016 American Physical Society.