This survey paper discusses the three-dimensional image processing challenges posed by present and future immersive telecommunications, especially immersive video conferencing and television. We introduce the concepts of presence, immersion, and co-presence and discuss their relation to virtual collaborative environments in the context of communications. Several examples are used to illustrate the current state of the art. We highlight the crucial need of real-time, highly realistic video with adaptive viewpoint for future immersive communications and identify calibration, multiple-view analysis, tracking, and view synthesis as the fundamental image-processing modules addressing such a need. For each topic, we sketch the basic problem and representative solutions from the image processing literature.