Text

theses

Cameras have become commonplace in phones, laptops and music-players. Similarly, light emitting displays are ubiquitous in the form of electronic billboards, computer monitors, and hand-held devices. The prevalence of cameras and displays in our society creates a novel opportunity to build camera-based optical wireless communication systems and we name them visual MIMO. MIMO is a common term from the field of communications. It represents “multiple-input multiple-output” that is typically used to describe multiple antenna radio frequency communications channel. In our proposed visual MIMO communications paradigm, inputs are pixels displayed from light-emitting devices such as electronic displays and outputs are camera pixels from the captured display images. This thesis will discuss several challenges in creating a visual MIMO system. The goal is to propose solutions based on computational photography methods to enhance the robust information transmission in the physical layer. Visual MIMO is a physical illumination system. We first propose a physics-based photometric modeling of image formation in the visual MIMO channel, considering display emittance function and camera sensitivity property. Based on this model, a novel photographic steganography method is presented for visual MIMO communications through message embedding and recovery. Such design enables dual use of electronic displays so that visual observation for human observers can coexist with a visual MIMO wireless communication channel. One example is a novel advertising application such as smartphone users pointing cell phone cameras at an electronic billboards to receive more specific information about the displayed advertisement. Consumer digital cameras produce visually pleasing images based on camera tone mappings. In this way, however, the captured intensity values might not be proportional to the corresponding spectral radiance of the scene. In the second part, we propose a radiometric calibration method to realize color correction for each frame in realtime. We name this approach optimal online radiometric calibration (OORC). The key innovation is to formulate both radiometric calibration and message recovery in one convex optimization problem. By modeling a camera-display transfer function (CDTF), we derive a physics-based kernel function for the Support Vector Machine (SVM) classifier, which serves as the optimization solver for OORC. Our experiments demonstrate that OORC is efficient and robust for computational messaging in multiple visual MIMO instances. An evaluation of results has been provided for video messaging with twelve different combinations of commercial cameras and displays. The third part of this thesis focuses on display detection and tracking methods. While traditional computer vision concentrates on images of objects, people or scenes; visual MIMO is confronted with the problem of an “image of image”. Specifically, when the electronic display is observed by a camera, it is critical to detect/track the display from the captured images in real time. In this section, we present screen detection for both message visible and invisible cases. Additionally, solutions to screen tracking and message classification are also proposed. To further explore camera-display communications, in the fourth section of this thesis we extend our visual MIMO paradigm with the time-of-flight (ToF) cameras. We design and construct a phase messaging array (PMA), which is the first of its kind, to communicate to a ToF depth camera by manipulating the modulated phase of the depth camera's infrared light signal. We show a complete implementation of a 3×3 array with a PCB-based circuit. Experiments demonstrate that the average bit accuracy is as high as 97.8% and the prototype data rate can achieve up to 1 Kbps.

ETD_6035

Ph.D.

Includes bibliographical references

by Wenjia Yuan

doi:10.7282/T3KK9DHB

ETD doctoral

The author owns the copyright to this work.