Analyzing the scene of pest and disease recognition based on aerial images, it is found that this scene requires high information extraction of small targets in shallow feature maps in the network. To address this issue, improvements are made to the SSD network by optimizing the object detection layer and proposing a dual backbone network strategy called DB-SSD (Dual Backbone SSD). This strategy involves adding an additional ResNet50 as a secondary backbone network that can better represent shallow details while maintaining the original VGG16 network, in order to enrich the expression of feature maps. Experiments have shown that the proposed detection algorithm has significant advantages in identifying pests and diseases based on aerial images.