Multicritical behavior in frustrated spin systems with non-collinear order

We investigate the phase diagram and, in particular, the nature of the multicritical point in three-dimensional frustrated N-component spin models with non-collinear order in the presence of an external field, for instance easy-axis stacked triangular antiferromagnets in the presence of a magnetic field along the easy axis. For this purpose we study the renormalization-group flow in a Landau-Ginzburg-Wilson phi(4) theory with symmetry O(2) circle times [Z(2) circle plus O(N - 1)] that is expected to describe the multicritical behavior. We compute its (MS) over bar beta functions to five loops. For N greater than or equal to 4, their analysis does not support the hypothesis of an effective enlargement of the symmetry at the multicritical point, from O(2) circle times [Z(2) circle plus O(N - 1)] to O(2) circle times O(N). For the physically interesting case N = 3, the analysis does not allow us to exclude the corresponding symmetry enlargement controlled by the O(2) circle times O(3) fixed point. Moreover, it does not provide evidence for any other stable fixed point. Thus, on the basis of our field-theoretical results, the transition at the multicritical point is expected to be either continuous and controlled by the O(2) circle times O(3) fixed point or to be of first order. (C) 2004 Elsevier B.V. All rights reserved.