Staff View
Reflectance and angular luminance for material recognition and segmentation

Descriptive

TitleInfo
Title
Reflectance and angular luminance for material recognition and segmentation
Name (type = personal)
NamePart (type = family)
Xue
NamePart (type = given)
Jia
NamePart (type = date)
1993-
DisplayForm
Jia Xue
Role
RoleTerm (authority = RULIB)
author
Name (type = personal)
NamePart (type = family)
Dana
NamePart (type = given)
Kristin J.
DisplayForm
Kristin J. Dana
Affiliation
Advisory Committee
Role
RoleTerm (authority = RULIB)
chair
Name (type = personal)
NamePart (type = family)
Ortiz
NamePart (type = given)
Jorge
DisplayForm
Jorge Ortiz
Affiliation
Advisory Committee
Role
RoleTerm (authority = RULIB)
internal member
Name (type = personal)
NamePart (type = family)
Rouayheb
NamePart (type = given)
Salim El
DisplayForm
Salim El Rouayheb
Affiliation
Advisory Committee
Role
RoleTerm (authority = RULIB)
internal member
Name (type = personal)
NamePart (type = family)
Rusinkiewicz
NamePart (type = given)
Szymon
DisplayForm
Szymon Rusinkiewicz
Affiliation
Advisory Committee
Role
RoleTerm (authority = RULIB)
outside member
Name (type = corporate)
NamePart
Rutgers University
Role
RoleTerm (authority = RULIB)
degree grantor
Name (type = corporate)
NamePart
School of Graduate Studies
Role
RoleTerm (authority = RULIB)
school
TypeOfResource
Text
Genre (authority = marcgt)
theses
OriginInfo
DateCreated (encoding = w3cdtf); (keyDate = yes); (qualifier = exact)
2020
DateOther (encoding = w3cdtf); (qualifier = exact); (type = degree)
2020-01
CopyrightDate (encoding = w3cdtf); (qualifier = exact)
2020
Language
LanguageTerm (authority = ISO 639-3:2007); (type = text)
English
Abstract (type = abstract)
Real world scenes consist of surfaces made of numerous materials, such as wood, marble, dirt, metal, ceramic and fabric, which contribute to the rich visual variation we find in images. Materials play a fundamental role in numerous applications including asphalt for automated driving, tree-cover in fire risk assessment, path material (grass vs concrete) for robot navigation, and landcover albedo analysis for climate studies. This thesis is dedicated to developing compact and robust material and texture representations for material recognition and segmentation.

Material properties affect the spatial variation of surface appearance and the angular variation of reflectance with respect to both view and illumination. Modeling the apparent or latent characteristic appearance of different materials is essential to robustly recognize them in images. We build representations that capture the intrinsic invariant properties of the surface, which enables fine-grained material recognition and segmentation. In particular, this thesis develop the following methods:

1. Differential Angular Imaging: We present a new measurement method called differential angular imaging where a surface is imaged from a particular viewing angle $v$ and then from an additional viewpoint $v+delta$. The motivation for this differential change in viewpoint is improved computation of the angular gradient of intensity $partial{I_v}/partial{v}$. We develop a framework for differential angular imaging, where small angular variations in image capture provide an enhanced appearance representation and significant recognition improvement.
We build a large-scale material database, Ground Terrain in Outdoor Scenes (GTOS) database, geared towards real use for autonomous agents. The database consists of over 30,000 images covering 40 classes of outdoor ground terrain under varying whether and lighting conditions.

2. Deep Texture Manifold: For ground terrain recognition, many class boundaries are ambiguous. Therefore, it is of interest to find not only the class label but also the closest classes, or equivalently, the position in the manifold.
We present a texture network called Deep Encoding Pooling Network (DEP) for the task of ground terrain recognition. Recognition of ground terrain is an important task in establishing robot or vehicular control parameters, as well as for localization within an outdoor environment. The architecture of DEP integrates orderless texture details and local spatial information. The resultant network shows excellent performance not only for GTOS-mobile, but also for more general databases (MINC and DTD). Based on DEP, we introduce a new texture manifold method, DEP-manifold, to find the relationship between newly captured images and images in dataset.

3. Texture Encoded Angular Network: We develop a novel approach for material recognition called texture-encoded angular network (TEAN) that combines deep encoding pooling of RGB information and differential angular images for angular-gradient features for the task of ground terrain recognition. With this novel network architecture, we extract characteristics of materials encoded in the angular and spatial gradients of their appearance. Our results show that TEAN achieves recognition performance that surpasses single view performance and standard (non-differential/large-angle sampling) multiview performance.

4. Angular Luminance: We utilize per-pixel angular luminance distributions as a key feature in discriminating the material of the surface. The angle-space sampling in a multiview image sequence is an unstructured sampling of the underlying reflectance function of the material. For real-world materials there is significant intra-class variation that can be managed by building a Angular Luminance Network (AngLNet). This network combines new angular reflectance cues from multiple images with more traditional spatial cues as in fully convolutional networks for semantic segmentation. We demonstrate the increased performance of AngLNet over prior state-of-the-art in material segmentation from drone video sequences and satellite imagery.
Subject (authority = RUETD)
Topic
Electrical and Computer Engineering
Subject (authority = local)
Topic
Material recognition
Subject (authority = LCSH)
Topic
Materials -- Imaging
RelatedItem (type = host)
TitleInfo
Title
Rutgers University Electronic Theses and Dissertations
Identifier (type = RULIB)
ETD
Identifier
ETD_10413
PhysicalDescription
Form (authority = gmd)
InternetMediaType
application/pdf
InternetMediaType
text/xml
Extent
1 online resource (xxiii, 113 pages) : illustrations
Note (type = degree)
Ph.D.
Note (type = bibliography)
Includes bibliographical references
RelatedItem (type = host)
TitleInfo
Title
School of Graduate Studies Electronic Theses and Dissertations
Identifier (type = local)
rucore10001600001
Location
PhysicalLocation (authority = marcorg); (displayLabel = Rutgers, The State University of New Jersey)
NjNbRU
Identifier (type = doi)
doi:10.7282/t3-hq8q-qf59
Genre (authority = ExL-Esploro)
ETD doctoral
Back to the top

Rights

RightsDeclaration (ID = rulibRdec0006)
The author owns the copyright to this work.
RightsHolder (type = personal)
Name
FamilyName
Xue
GivenName
Jia
Role
Copyright Holder
RightsEvent
Type
Permission or license
DateTime (encoding = w3cdtf); (qualifier = exact); (point = start)
2019-11-05 14:16:37
AssociatedEntity
Name
Jia Xue
Role
Copyright holder
Affiliation
Rutgers University. School of Graduate Studies
AssociatedObject
Type
License
Name
Author Agreement License
Detail
I hereby grant to the Rutgers University Libraries and to my school the non-exclusive right to archive, reproduce and distribute my thesis or dissertation, in whole or in part, and/or my abstract, in whole or in part, in and from an electronic format, subject to the release date subsequently stipulated in this submittal form and approved by my school. I represent and stipulate that the thesis or dissertation and its abstract are my original work, that they do not infringe or violate any rights of others, and that I make these grants as the sole owner of the rights to my thesis or dissertation and its abstract. I represent that I have obtained written permissions, when necessary, from the owner(s) of each third party copyrighted matter to be included in my thesis or dissertation and will supply copies of such upon request by my school. I acknowledge that RU ETD and my school will not distribute my thesis or dissertation or its abstract if, in their reasonable judgment, they believe all such rights have not been secured. I acknowledge that I retain ownership rights to the copyright of my work. I also retain the right to use all or part of this thesis or dissertation in future works, such as articles or books.
Copyright
Status
Copyright protected
Availability
Status
Open
Reason
Permission or license
Back to the top

Technical

RULTechMD (ID = TECHNICAL1)
ContentModel
ETD
OperatingSystem (VERSION = 5.1)
windows xp
CreatingApplication
Version
1.5
DateCreated (point = end); (encoding = w3cdtf); (qualifier = exact)
2019-11-05T17:30:27
DateCreated (point = end); (encoding = w3cdtf); (qualifier = exact)
2019-11-05T17:30:27
ApplicationName
pdfTeX-1.40.18
Back to the top
Version 8.5.5
Rutgers University Libraries - Copyright ©2024