Space division multiplexing (SDM) is currently widely investigated in order to provide enhanced capacity thanks to the utilization of space as a new degree of multiplexing freedom in both optical fiber communication and on-chip interconnects. Basic components allowing the processing of spatial modes are critical for SDM applications. Here we present such building blocks implemented on the silicon-on-insulator (SOI) platform. These include fabrication tolerant wideband (de) multiplexers, ultra-compact mode converters and (de) multiplexers designed by topology optimization, and mode filters using one-dimensional (1D) photonic crystal silicon waveguides. We furthermore use the fabricated devices to demonstrate on-chip point-to-point mode division multiplexing transmission, and all-optical signal processing by mode-selective wavelength conversion. Finally, we report an efficient silicon photonic integrated circuit mode (de) multiplexer for few-mode fibers (FMFs).