High security and high reliability identification methods are desired in unmanned aerial vehicle (UAV) swarms. An identification method for UAVs based on optical spectrum modulation is proposed by using narrow-linewidth CdSe/ZnS quantum dots (QDs). By generating an environmental-information-based key-generation algorithm, UAV1 is capable to promptly produce a key optical spectrum, which conceals its identity codes of UAV1. When the key optical spectrum transmits to the photodetector of UAV2, UAV2 can decode the key optical spectrum and gain the identity of UAV1. Solutions are proposed to improve the success rate of identification for UAV2, as well as to avoid the problem of spectral floating during the flighting process of UAVs. A success rate of 100% is predicted when the identification system supports no more than 27 types of UAV identities in a group under conditions: 1) the digital numbers of all identity codes are within the range of 1–3 and 2) the floating percentage of the optical spectrum is not higher than 14%. The proposed method possesses high simplicity, high success rate, and high confidentiality in identification. It has potential to be applied as a secure communication method in large-scale UAV swarms.