Home
Resource
Improved empirical methods in reinforcement-learning evaluation
PDF
PDF format is widely accepted and good for printing.
Plug-in required
PDF-1(1.47 MB)
Citation & Export
View Usage Statistics
Staff View

Citation & Export
Hide

Simple citation

Marivate, Vukosi N.. Improved empirical methods in reinforcement-learning evaluation. Retrieved from https://doi.org/doi:10.7282/T34X59H0

Export

Click here for information about Citation Management Tools at Rutgers.

Statistics
Hide

Description

TitleImproved empirical methods in reinforcement-learning evaluation
NameMarivate, Vukosi N. (author); Littman, Michael L (chair); Eliassi-Rad, Tina (internal member); Marian, Amélie (internal member); Murphy, Susan A (outside member); Rutgers University; Graduate School - New Brunswick
Date Created2015
Other Date2015-01 (degree)
SubjectComputer Science, Reinforcement learning, Machine learning, Algorithms
Extent1 online resource (xvi, 132 p. : ill.)
DescriptionThe central question addressed in this research is ”can we define evaluation methodologies that encourage reinforcement-learning (RL) algorithms to work effectively with real-life data?” First, we address the problem of overfitting. RL algorithms are often tweaked and tuned to specific environments when applied, calling into question whether learning algorithms that work for one environment will work for others. We propose a methodology to evaluate algorithms on distributions of environments, as opposed to a single environment. We also develop a formal framework for characterizing the ”capacity” of a space of parameterized RL algorithms and bound the generalization error of a set of algorithms on a distribution of RL environments given a sample of environments. Second, we develop a method for evaluating RL algorithms offline using a static collection of data. Our motivation is that real-life applications of RL often have properties that make online evaluation expensive (such as driving a robot car), ethically questionable (such as treating a disease), or simply impractical (such as challenging a human chess master). We compared several offline evaluation metrics and found our new metric (”relative Bellman update error”) addresses shortcomings in more standard approaches. Third, we examine the problem of evaluating behavior policies for individuals using observational data. Our focus is on quantifying the uncertainty that arises from multiple sources: population mismatch, data sparsity, and intrinsic stochasticity. We have applied our method to a collection of HIV treatment and non-profit fund-raising appeals data.
NotePh.D.
NoteIncludes bibliographical references
Noteby Vukosi N. Marivate
Genretheses, ETD doctoral
Persistent URLhttps://doi.org/doi:10.7282/T34X59H0
Languageeng
CollectionGraduate School - New Brunswick Electronic Theses and Dissertations
Organization NameRutgers, The State University of New Jersey
RightsThe author owns the copyright to this work.
Version 8.5.5
Rutgers University Libraries - Copyright ©2024