In this work we propose the F-score measure as a novel means to perform online selection of the members of a classifier ensemble. This allows the fast application of a small number of selected classifiers for real-time applications such as target tracking for video surveillance. The proposed selection criterion relies on a performance evaluation to assess the ability of individual classifiers to predict the class membership, that is to discriminate between foreground and background in the context of video tracking. Preliminary experiments have shown encouraging results on real-world sequences.