Aquaculture and salt-culture are relevant economic activities in the Brazilian Coastal Zone (BCZ). However, automatic discrimination of such activities from other water-related covers/uses is not an easy task. In this sense, convolutional neural networks (CNN) have the advantage of predicting a given pixel’s class label by providing as input a local region (named patches or chips) around that pixel. Both the convolutional nature and the semantic segmentation capability provide the U-Net classifier with the ability to access the “context domain” instead of solely isolated pixel values. Backed by the context domain, the results obtained show that the BCZ aquaculture/saline ponds occupied ~356 km2 in 1985 and ~544 km2 in 2019, reflecting an area expansion of ~51%, a rise of 1.5× in 34 years. From 1997 to 2015, the aqua-salt-culture area grew by a factor of ~1.7, jumping from 349 km2 to 583 km2, a 67% increase. In 2019, the Northeast sector concentrated 93% of the coastal aquaculture/salt-culture surface, while the Southeast and South sectors contained 6% and 1%, respectively. Interestingly, despite presenting extensive coastal zones and suitable conditions for developing different aqua-salt-culture products, the North coast shows no relevant aqua or salt-culture infrastructure sign.