Context. Recent surveys indicate that planets in binary systems are more abundant than previously thought, which is in agreement with theoretical work on disc dynamics and planet formation in binaries. So far, most observational surveys, however, have focused on short-period planets in binaries, thus little is known about the occurrence rates of planets on longer periods (≥10 au). Aims. In order to measure the abundance and physical characteristics of wide-orbit giant exoplanets in binary systems, we have designed the “VIsual Binary Exoplanet survey with Sphere” (VIBES) to search for planets in visual binaries. It uses the SPHERE instrument at VLT to search for planets in 23 visual binary and four visual triple systems with ages of <145 Myr and distances of <150 pc. Methods. We used the IRDIS dual-band imager on SPHERE to acquire high-contrast images of the sample targets. For each binary, the two components were observed at the same time with a coronagraph masking only the primary star. For the triple star, the tight components were treated as a single star for data reduction. This enabled us to effectively search for companions around 50 individual stars in binaries and four binaries in triples. Results. We derived upper limits of <13.7% for the frequency of sub-stellar companions around primaries in visual binaries, <26.5% for the fraction of sub-stellar companions around secondaries in visual binaries, and an occurrence rate of <9.0% for giant planets and brown dwarfs around either component of visual binaries. We have combined our observations with literature measurements to astrometrically confirm, for the first time, that 20 binaries and two triple systems, which were previously known, are indeed physically bound. Finally, we discovered a third component of the binary HD 121336. Conclusions. The upper limits we derived are compatible with planet formation through the core accretion and the gravitational instability processes in binaries. These limits are also in line with limits found for single star and circumbinary planet search surveys.