Analysis of Deep Learning Methods in Adaptation to the Small Data Problem Solving

Abstract

This paper discusses a specific problem in the study of deep neural networks – learning on small data. Such issue happens in situation of transfer learning or applying known solutions on new tasks that involves usage of particular small portions of data. Based on previous research, some specific solutions can be applied to various tasks related to machine learning, computer vision, natural language processing, medical data study and many others. These solutions include various methods of general purpose machine and deep learning, being successfully used for these tasks. In order to do so, the paper carefully studies the problems arise in the preparation of data. For benchmark purposes, we also compared “in wild” the methods of machine learning and identified some issues in their practical application, in particular usage of specific hardware. The paper touches some other aspects of machine learning by comparing the similarities and differences of singular value decomposition and deep constrained auto-encoders. In order to test our hypotheses, we carefully studied various deep and machine learning methods on small data. As a result of the study, our paper proposes a set of solutions, which include the selection of appropriate algorithms, data preparation methods, hardware optimized for machine learning, discussion of their practical effectiveness and further improvement of approaches and methods described in the paper. Also, some problems were discussed, which have to be addressed in the following papers.