Fine-grained Image recognition is a difficult problem for computer vision area, and it is also a challenge for big data and artificial intelligence. In order to solve this problem, an object localization method based on bilinear convolutional network is proposed. The bilinear convolutional network is used to obtain different features of VGG network, and the intersection set is used as the localization framework. In order to verify the feasibility of this paper, the proposed method is simulated on CUB-200-2011 and Stanford Cars-196. The simulation result shows the accuracy of the proposed method is higher than other representative methods. After adding the Softmax classier, this proposed method improved largely compared with the primitive image as the input.