Childhood pneumonia, the leading cause of children mortality globally, is most commonly diagnosed based on the radiographic data, which requires radiologic interpretation of X-ray images. With recent advancements in the field of deep learning, the convolutional neural networks (CNN) have proven to be able to achieve great performance in medical image segmentation, analysis and classification tasks. However, developing and training methods utilizing CNN is still a complex and time-consuming process with several open issues – generalization, demand for large datasets, and high time complexity. The optimization objective in the training of CNN models has multiple minima, which do not necessarily generalize well and may result in poor performance. Ensemble methods are commonly employed to address the generalization issue, but they require a group of diverse models and are generally even more time-consuming. To address the issues of generalization, dataset size and time complexity, we developed an ensemble method based on stochastic gradient descent with warm restarts (SGDRE) that exploits the generalization capabilities of ensemble methods and SGD with warm restarts mechanism, which is adopted to obtain a diverse group of classifiers, necessary for ensemble, in one single training process, spending the same or less training time than a single CNN classification model. The SGDRE method has been trained on publicly available pediatric chest X-ray images dataset and evaluated using 10-fold cross-validation approach. The experimental results show a significant improvement of SGDRE over the two compared baseline methods. With an achieved test accuracy of 96.26% and AUC of 95.15%, the proposed method proved to be a very competitive classification method.