Abstract: Research shows that there is redundancy in the output feature maps of ResNet bottleneck structure. Such redundancy ensures a comprehensive understanding of the input data, but generates the redundant information which consumes additional computational resources. And the proportion of redundant information is very large when processing small category classification tasks. To solve this problem, a new dimension increasing structure is designed to improve the bottleneck structure by residual-like structure and concatenation operation. This structure is called residual concatenation (RC). The RC can not only reduce the amount of calculation and parameters of bottleneck structure, but also enhance the gradient transmission of back-propagation to improve the accuracy. In this work, the RC is combined with multiple residual networks and image classification experiments are performed on multiple datasets. The results show that the RC-based bottleneck structure can reduce the consumption and improve the accuracy of classification tasks while processing small category classification tasks.