We deal with the position-based visual servoing control of a robot manipulator equipped with a camera, mounted in an eye-in-hand configuration. The goal is to move the robot in order to reach the desired pose of the camera with respect to an object. We assume that the internal parameters of the camera are known, while the pose of the camera with respect to the robot flange is unknown, except for some approximate bounds on the rotation and translation components; i.e., we get rid of hand–eye calibration (HEC). We exploit the fact that the Jacobian of the camera displacement is included in a suitable polytope. Thus we can adopt a Lyapunov-based technique that guarantees asymptotic convergence to the desired camera-to-object reference pose. Our experimental results, performed on a 6 degrees-of-freedom (DoF) robotic manipulator, show that the proposed approach is effective in both achieving a single target pose and performing target object tracking.