A sign language recognition system can be used as a communication interpreter by deaf people. Convolutional neural networks (CNNs) based on deep learning have become the most effective architecture for the implementation of vision-based sign language recognition systems because CNNs can extract features automatically and classify images according to the extracted features. Although a large amount of training data is required for training a CNN, the collection of hand images requires much effort because it involves human resources. In this paper, we propose a hand posture classification method based on a state-of-the-art CNN model and a data augmentation method that is specialized in hand depth data. To overcome the difficulty of constructing a large dataset, we focus on generating more data with various appearances by applying the data augmentation method based on 3D rotation and adding virtual hand thickness.