Text

theses

Material recognition plays an important role for a machine to understand and interact with the world. For example, an autonomous vehicle can use material recognition to determine whether the terrain is asphalt, grass, gravel, ice or snow in order to optimize the mechanical control and a robot can easily grab an object with proper pressure if the object’s composition is known. This thesis is dedicated to developing fast material recognition techniques towards the goal of building camera seeing materials in real-time. Color and geometry are not a full measure of the richness of visual appearance. Reflectance describes the characteristics of light interaction with a surface, which is uniquely determined by how the surface is made up at a microscopic (e.g., pigments in the surface medium) and mescopic scale (e.g., geometric 3D texture). Naturally, reflectance provides an invaluable clue about the surface, including what it is made of (i.e., material), and how it is shaped (i.e., surface roughness). We study how reflectance can reveal the material categories and physical properties. 1. Reflectance Hashing: Reflectance is challenging to measure and use for recognizing materials due to its high-dimensionality. In this work, we bypass the use of a gonioreflectometer by using a novel one-shot reflectance camera based on a parabolicmirror design. The pixel coordinates of these reflectance disks correspond to the sur- face viewing angles. The reflectance has class-specific structure and angular gradients computed in this reflectance space reveal the material class. These reflectance disks encode discriminative information for efficient and accurate material recognition. We introduce a framework called reflectance hashing that models the reflectance disks with dictionary learning and binary hashing. We demonstrate the effectiveness of reflectance hashing for material recognition with a number of real-world materials. 2. Deep Reflectance Code: We introduce a framework that enables prediction of actual friction values for surfaces using one-shot reflectance measurements. This work is a first of its kind vision-based friction estimation. We develop a novel representation for reflectance disks that capture partial BRDF measurements instantaneously. Our method of deep reflectance codes combines CNN features and fisher vector pooling with optimal binary embedding to create codes that have sufficient discriminatory power and have important properties of illumination and spatial invariance. The experimental results demonstrate that reflectance can play a new role in deciphering the underlying physical properties of real-world scenes. 3. Texture Encoding Network: We propose a Deep Texture Encoding Net- work (DeepTEN) with a novel Encoding Layer integrated on top of convolutional layers, which ports the entire dictionary learning and encoding pipeline into a single model. The features, dictionaries and the encoding representation for the classifier are all learned simultaneously. The representation is orderless and therefore is particularly useful for material and texture recognition. The Encoding Layer generalizes robust residual encoders such as VLAD and Fisher Vectors, and has the property of dis- carding domain specific information which makes the learned convolutional features easier to transfer. The experimental results show superior performance as compared to state-of-the-art methods using gold-standard databases such as MINC-2500, Flickr Material Database, KTH-TIPS-2b, and two recent databases 4D-Light-Field-Material and GTOS. 4. Real-time Texture Synthesis We introduce a Multi-style Generative Net- work (MSG-Net) with a novel Inspiration Layer, which retains the functionality of optimization-based approaches and has the fast speed of feed-forward networks. The proposed Inspiration Layer explicitly matches the feature statistics with the target styles at run time, which dramatically improves versatility of existing generative net- work, so that multiple styles can be realized within one network. The proposed MSG- Net matches image styles at multiple scales and puts the computational burden into the training. The learned generator is a compact feed-forward network that runs in real-time after training. In conclusion, this thesis developed robust visual techniques for material and texture modeling. We introduce the concept of angular gradient that has been proved effective in material recognition. Our proposed texture encoding network has achieved state of the art results on material and texture recognition. We expect the proposed solutions for material material recognition have a tremendous impact on a great number of real- world applications.

ETD_8414

Ph.D.

Includes bibliographical references

by Hang Zhang

doi:10.7282/T3Z89GMT

ETD doctoral

The author owns the copyright to this work.