Abstract In most practical scenarios, optical susceptibilities can be treated as a local property of a medium. For example, in the context of nonlinear optics we can typically treat the Kerr and Raman response as local, such that optical fields at one location do not produce a nonlinear response at distinct locations in space. This is because the electronic and vibrational disturbances produced within the material are confined to a region that is smaller than an optical wavelength. By comparison, Brillouin interactions, mediated by traveling-wave acoustic phonons, can result in a highly nonlocal nonlinear response as the elastic waves generated in the process can occupy a region in space much larger than an optical wavelength. The unique properties of these interactions can be exploited to engineer new types of processes, where highly delocalized phonon modes serve as an engineerable channel that mediates scattering processes between light waves propagating in distinct optical waveguides. These types of nonlocal optomechanical responses have recently been demonstrated as the basis for information transduction, however the nontrivial dynamics of such systems has yet to be explored. In this work, we show that the third-order nonlinear process resulting from spatially extended Brillouin-active phonon modes involves mixing products from spatially separated, optically decoupled waveguides, yielding a nonlocal susceptibility. Building on these concepts, we illustrate how nontrivial multi-mode acoustic interference can produce a nonlocal susceptibility with a multi-pole frequency response, as the basis for new optical and microwave signal processing schemes within traveling wave systems.