Neural-network architectures have been increasingly used to represent quantum many-body wave functions. These networks require a large number of variational parameters and are challenging to optimize using traditional methods, as gradient descent. Stochastic reconfiguration (SR) has been effective with a limited number of parameters, but becomes impractical beyond a few thousand parameters. Here, we leverage a simple linear algebra identity to show that SR can be employed even in the deep learning scenario. We demonstrate the effectiveness of our method by optimizing a Deep Transformer architecture with 3 x 105 parameters, achieving state-of-the-art ground-state energy in the J1-J2 Heisenberg model at J2/J1 = 0.5 on the 10 x 10 square lattice, a challenging benchmark in highly-frustrated magnetism. This work marks a significant step forward in the scalability and efficiency of SR for neural-network quantum states, making them a promising method to investigate unknown quantum phases of matter, where other methods struggle.Stochastic reconfiguration (SR) is a method to enhance the optimization of variational functions in quantum many-body systems but becomes impractical beyond a few thousand parameters. Here, the authors combine an alternative SR formulation with a Deep Transformer architecture with 105 parameters, achieving the best variational energy for the J1-J2 Heisenberg model on the 10 x 10 square lattice
