The use of unmanned aerial vehicles (UAVs) for different applications has increased tremendously during the past decade. The small size, high maneuverability, ability to fly at predetermined coordinates, simple construction, and affordable price have made UAVs a popular choice for diverse aerial applications. However, the small size and the ability to fly close to the terrain make the detection and tracking of UAVs challenging. Similarly, unmanned underwater vehicles (UUVs) have revolutionized underwater operations. UUVs can accomplish numerous tasks that were not possible with manned underwater vehicles. In this survey paper, we provide features and capabilities expected from current and future UAVs and UUVs, and review potential challenges and threats due to use of such UAVs/UUVs. We also overview the countermeasures against such threats, including approaches for the detection, tracking, and classification of UAVs and UUVs.