Abstract— Recently, Convolutional Neural Networks (CNNs) have shown a great significance in the field of computer vision and image recognition. However, they have two major difficulties to be addressed. The first one is the need to have too many samples per class for training. It is not always an easy task to have a huge number of labelled samples per class for every problem; actually, in some cases only one sample per class is all what is available. The second difficulty is the enormous computational power needed to perform the training task. In this paper, an experimental study of how to handle problems where the data available provides only one sample per class is carried out. The implemented technique utilizes proper data augmentation in solving the aforementioned problems. A database of the world countries flags with one flag sample per country is used to start with. One sample per class is the worst case for CNN training, but the proposed approach helps in enhancing the accuracy from being unfeasible at the beginning of training to above 99% of validation accuracy when the technique is applied. Promising results have been achieved without the need for a very deep CNN. Also, the findings reveal that the utilized type of data augmentation technique must be carefully selected for each application to avoid over-fitting while obtaining the best validation accuracy. As a solution, an adequate selection of different augmentation options is tried out to improve the network ability to generalize well to new testing samples.