The problem that often occurs in agriculture is about diseases in plants. Plant diseases can result in reduced yields from agricultural production. Therefore, the detection and analysis of plant diseases are critical and should be done as early as possible. Unfortunately, diseases in plants often appear on the leaves, and the characteristics of the affected leaves are very diverse and difficult to distinguish. This phenomenon results in difficulty in the identification of plant diseases automatically. One of the technologies that can be used in identifying leaf problems is digital image processing technology. The plant used as a case study in this research is the tomato plant. Alternaria Solani, Septoria leaf spot, Yellow virus are some of the disorders that tomato plants can experience. These disorders should be classified according to their type. This research designs a system to classify three types of disease experienced by the tomato plant leaves. A dataset of 4400 leaf images is collected and learned to the Convolutional Neural Network (CNN) to classify three tomato plant problems using the Augmentation process. An evaluation using 5-fold cross-validation shows that CNN with augmentation data gives an average accuracy of 97.8% and the highest accuracy of 99.5%. This result is better than the previous methods: AlexNet, Faster R-CNN, and CNN + red green blue (RGB).