RUcore: Rutgers University Community Repository
Perception-based generalization in model-based reinforcement learning
Descriptive
TypeOfResource
Text
TitleInfo
(ID = T-1)
Title
Perception-based generalization in model-based reinforcement learning
SubTitle
PartName
PartNumber
NonSort
Identifier
ETD_1481
Identifier
(type = hdl)
http://hdl.rutgers.edu/1782.2/rucore10001600001.ETD.000051041
Language
(objectPart = )
LanguageTerm
(authority = ISO639-2);
(type = code)
eng
Genre
(authority = marcgt)
theses
Subject
(ID = SBJ-1);
(authority = RUETD)
Topic
Computer Science
Subject
(ID = SBJ-2);
(authority = ETD-LCSH)
Topic
Reinforcement learning--Mathematical models
Subject
(ID = SBJ-3);
(authority = ETD-LCSH)
Topic
Machine learning
Abstract
In recent years, the advances in robotics have allowed for robots to venture into places too dangerous for humans. Unfortunately, the terrain in which these robots are being deployed may not be known by humans in advance, making it difficult to create motion programs robust enough to handle all scenarios that the robot may encounter. For this reason, research is being done to add learning capabilities to improve the robot's ability to adapt to its environment. Reinforcement learning is well suited for these robot domains because often the desired outcome is known, but the best way to achieve this outcome is unknown.
In a real world domain, a reinforcement-learning agent has to learn a great deal from experience. Therefore, it must be sample-size efficient. To do so, it must balance the amount of exploration that is needed to properly model the environment with the need to use the information that it has already obtained to complete its original task. In robot domains, the exploration process is especially costly in both time and energy. Therefore, it is important to make the best possible use of the robot's limited opportunities for exploration without degrading the robot's performance.
This dissertation discusses a specialization of the standard Markov Decision Process (MDP) framework that allows for easier transfer of experience between similar states and introduces an algorithm that uses this new framework to perform more efficient exploration in robot-navigation problems. It then develops methods for an agent to determine how to accurately group similar states. One proposed technique clusters states by their observed outcomes. To make it possible to extrapolate observed outcomes to as-yet unvisited states, a second approach uses perceptual information such as the output of an image-processing system to group perceptually similar states with the hope that they will also be related in terms of outcomes. However, there are many different percepts from which a robot could obtain state groupings. To address this issue, a third algorithm is presented that determines how to group states when the agent has multiple, possibly conflicting, inputs from which to choose. Robot experiments of all algorithms proposed are included to demonstrate the improvements that can be obtained by using the approaches presented.
In a real world domain, a reinforcement-learning agent has to learn a great deal from experience. Therefore, it must be sample-size efficient. To do so, it must balance the amount of exploration that is needed to properly model the environment with the need to use the information that it has already obtained to complete its original task. In robot domains, the exploration process is especially costly in both time and energy. Therefore, it is important to make the best possible use of the robot's limited opportunities for exploration without degrading the robot's performance.
This dissertation discusses a specialization of the standard Markov Decision Process (MDP) framework that allows for easier transfer of experience between similar states and introduces an algorithm that uses this new framework to perform more efficient exploration in robot-navigation problems. It then develops methods for an agent to determine how to accurately group similar states. One proposed technique clusters states by their observed outcomes. To make it possible to extrapolate observed outcomes to as-yet unvisited states, a second approach uses perceptual information such as the output of an image-processing system to group perceptually similar states with the hope that they will also be related in terms of outcomes. However, there are many different percepts from which a robot could obtain state groupings. To address this issue, a third algorithm is presented that determines how to group states when the agent has multiple, possibly conflicting, inputs from which to choose. Robot experiments of all algorithms proposed are included to demonstrate the improvements that can be obtained by using the approaches presented.
PhysicalDescription
Form
(authority = gmd)
electronic resource
Extent
xv, 105 p. : ill.
InternetMediaType
application/pdf
InternetMediaType
text/xml
Note
(type = degree)
Ph.D.
Note
(type = bibliography)
Includes bibliographical references (p. 100-104)
Note
(type = statement of responsibility)
by Bethany R. Leffler
Name
(ID = NAME-1);
(type = personal)
NamePart
(type = family)
Leffler
NamePart
(type = given)
Bethany R.
Role
RoleTerm
(authority = RULIB);
(type = )
author
DisplayForm
Bethany R. Leffler
Name
(ID = NAME-2);
(type = personal)
NamePart
(type = family)
Littman
NamePart
(type = given)
Michael
Role
RoleTerm
(authority = RULIB);
(type = )
chair
Affiliation
Advisory Committee
DisplayForm
Michael L. Littman
Name
(ID = NAME-3);
(type = personal)
NamePart
(type = family)
Stone
NamePart
(type = given)
Matthew
Role
RoleTerm
(authority = RULIB);
(type = )
internal member
Affiliation
Advisory Committee
DisplayForm
Matthew Stone
Name
(ID = NAME-4);
(type = personal)
NamePart
(type = family)
Pavlovic
NamePart
(type = given)
Vladimir
Role
RoleTerm
(authority = RULIB);
(type = )
internal member
Affiliation
Advisory Committee
DisplayForm
Vladimir Pavlovic
Name
(ID = NAME-5);
(type = personal)
NamePart
(type = family)
Roy
NamePart
(type = given)
Nicholas
Role
RoleTerm
(authority = RULIB);
(type = )
outside member
Affiliation
Advisory Committee
DisplayForm
Nicholas Roy
Name
(ID = NAME-1);
(type = corporate)
NamePart
Rutgers University
Role
RoleTerm
(authority = RULIB);
(type = )
degree grantor
Name
(ID = NAME-2);
(type = corporate)
NamePart
Graduate School - New Brunswick
Role
RoleTerm
(authority = RULIB);
(type = )
school
OriginInfo
DateCreated
(point = );
(qualifier = exact)
2009
DateOther
(qualifier = exact);
(type = degree)
2009-01
Location
PhysicalLocation
(authority = marcorg)
NjNbRU
RelatedItem
(type = host)
TitleInfo
Title
Rutgers University Electronic Theses and Dissertations
Identifier
(type = RULIB)
ETD
RelatedItem
(type = host)
TitleInfo
Title
Graduate School - New Brunswick Electronic Theses and Dissertations
Identifier
(type = local)
rucore19991600001
Identifier
(type = doi)
doi:10.7282/T3C53M35
Genre
(authority = ExL-Esploro)
ETD doctoral
Back to the top
Rights
RightsDeclaration
(AUTHORITY = GS);
(ID = rulibRdec0006)
The author owns the copyright to this work.
Copyright
Status
Copyright protected
Availability
Status
Open
RightsEvent
(AUTHORITY = rulib);
(ID = 1)
Type
Permission or license
Detail
Non-exclusive ETD license
AssociatedObject
(AUTHORITY = rulib);
(ID = 1)
Type
License
Name
Author Agreement License
Detail
I hereby grant to the Rutgers University Libraries and to my school the non-exclusive right to archive, reproduce and distribute my thesis or dissertation, in whole or in part, and/or my abstract, in whole or in part, in and from an electronic format, subject to the release date subsequently stipulated in this submittal form and approved by my school. I represent and stipulate that the thesis or dissertation and its abstract are my original work, that they do not infringe or violate any rights of others, and that I make these grants as the sole owner of the rights to my thesis or dissertation and its abstract. I represent that I have obtained written permissions, when necessary, from the owner(s) of each third party copyrighted matter to be included in my thesis or dissertation and will supply copies of such upon request by my school. I acknowledge that RU ETD and my school will not distribute my thesis or dissertation or its abstract if, in their reasonable judgment, they believe all such rights have not been secured. I acknowledge that I retain ownership rights to the copyright of my work. I also retain the right to use all or part of this thesis or dissertation in future works, such as articles or books.
Back to the top
Technical
ContentModel
ETD
MimeType
(TYPE = file)
application/pdf
MimeType
(TYPE = container)
application/x-tar
FileSize
(UNIT = bytes)
12984320
Checksum
(METHOD = SHA1)
a63e5a45267bd205664359fbcc71fd6d8257703f
Back to the top
Statistical Profile