Combined with semantic segmentation and visual attention mechanism, a new method for UAV detection in complex background is presented. The semantic segmentation of target image, which is implemented by Mask R-CNN, is used to exclude invalid regions. Then the processes of small target detection based on visual attention mechanism is built. This method is used to detecting the position of UAV in the sky. The effectiveness of algorithm proposed in this paper is proved by a large number of experiments.