We present N-body simulations of intermediate-mass (3000-4000 M-circle dot) young star clusters (SCs) with three different metallicities (Z = 0.01, 0.1 and 1 Z(circle dot)), including metal-dependent stellar evolution recipes and binary evolution. Following recent theoretical models of wind mass-loss and core-collapse supernovae, we assume that the mass of the stellar remnants depends on the metallicity of the progenitor stars. In particular, massive metal-poor stars (Z = 0.3 Z(circle dot)) are enabled to form massive stellar black holes (MSBHs, with mass >= 25 M-circle dot) through direct collapse. We find that three-body encounters, and especially dynamical exchanges, dominate the evolution of the MSBHs formed in our simulations. In SCs with Z = 0.01 and 0.1 Z(circle dot), about 75 per cent of simulated MSBHs form from single stars and become members of binaries through dynamical exchanges in the first 100 Myr of the SC life. This is a factor of greater than or similar to 3 more efficient than in the case of low-mass (<25 M-circle dot) stellar black holes. A small but non-negligible fraction of MSBHs power wind-accreting (10-20 per cent) and Roche lobe overflow (RLO, 5-10 per cent) binary systems. The vast majority of MSBH binaries that undergo wind accretion and/or RLO were born from dynamical exchange. This result indicates that MSBHs can power X-ray binaries in low-metallicity young SCs, and is very promising to explain the association of many ultraluminous X-ray sources with low-metallicity and actively star-forming environments.