In this paper, we propose an original deep learning framework for the automated counting and geolocation of palm trees from aerial images using convolutional neural networks. For this purpose, we collected aerial images from two different regions in Saudi Arabia, using two DJI drones, and we built a dataset of around 11,000 instances of palm trees. Then, we applied several recent convolutional neural network models (Faster R-CNN, YOLOv3, YOLOv4, and EfficientDet) to detect palms and other trees, and we conducted a complete comparative evaluation in terms of average precision and inference speed. YOLOv4 and EfficientDet-D5 yielded the best trade-off between accuracy and speed (up to 99% mean average precision and 7.4 FPS). Furthermore, using the geotagged metadata of aerial images, we used photogrammetry concepts and distance corrections to automatically detect the geographical location of detected palm trees. This geolocation technique was tested on two different types of drones (DJI Mavic Pro and Phantom 4 pro) and was assessed to provide an average geolocation accuracy that attains 1.6 m. This GPS tagging allows us to uniquely identify palm trees and count their number from a series of drone images, while correctly dealing with the issue of image overlapping. Moreover, this innovative combination between deep learning object detection and geolocalization can be generalized to any other objects in UAV images.