Nominated by the Rensselaer Polytechnic Institute as an outstanding Ph.D. thesis
Explains reinforcement learning through a range of problems by exploring what affects reinforcement learning and what contributes to a successful implementation
Includes a contemporary design of experiments methods, comprising of a novel sequential experimentation procedure that finds convergent learning algorithm parameter subregions and stochastic kriging for response surface metamodeling
Christopher Gatti received his PhD in Decision Sciences and Engineering Systems from Rensselaer Polytechnic Institute (RPI). During his time at RPI, his work focused on machine learning and statistics, with applications in reinforcement learning, graph search, stem cell RNA analysis, and neuro-electrophysiological signal analysis. Prior to beginning his graduate work at RPI, he received a BSE in mechanical engineering and an MSE in biomedical engineering, both from the University of Michigan. He then continued to work at the University of Michigan for three years doing computational biomechanics focusing on the shoulder and knee. He has been a gymnast since he was a child and is currently an acrobat for Cirque du Soleil.
This thesis takes an empirical approach to understanding of the behavior and interactions between the two main components of reinforcement learning: the learning algorithm and the functional representation of learned knowledge. The author approaches these entities using design of experiments not commonly employed to study machine learning methods. The results outlined in this work provide insight as to what enables and what has an effect on successful reinforcement learning implementations so that this learning method can be applied to more challenging problems.
GLOSSARYACKNOWLEDGMENTFOREWARD1. INTRODUCTION 2. REINFORCEMENT LEARNING2.1 Applications of reinforcement learning 2.1.1 Benchmark problems2.1.2 Games2.1.3 Real-world applications2.1.4 Generalized domains2.2 Components of reinforcement learning2.2.1 Domains2.2.2 Representations2.2.3 Learning algorithms 2.3 Heuristics and performance effectors 3. DESIGN OF EXPERIMENTS 3.1 Classical design of experiments3.2 Contemporary design of experiments3.3 Design of experiments for empirical algorithm analysis 4. METHODOLOGY 4.1 Sequential CART4.1.1 CART modeling 4.1.2 Sequential CART modeling4.1.3 Analysis of sequential CART 4.1.4 Empirical convergence criteria 4.1.5 Example: 2-D 6-hump camelback function4.2 Kriging metamodeling 4.2.1 Kriging 4.2.2 Deterministic kriging4.2.3 Stochastic kriging 4.2.4 Covariance function 4.2.5 Implementation 4.2.6 Analysis of kriging metamodels5. THE MOUNTAIN CAR PROBLEM 5.1 Reinforcement learning implementation 5.2 Sequential CART5.3 Response surface metamodeling5.4 Discussion 6. THE TRUCK BACKER-UPPER PROBLEM6.1 Reinforcement learning implementation6.2 Sequential CART 6.3 Response surface metamodeling6.4 Discussion 7. THE TANDEM TRUCK BACKER-UPPER PROBLEM 7.1 Reinforcement learning implementation7.2 Sequential CART 7.3 Discussion 8. DISCUSSION 8.1 Reinforcement learning8.2 Experimentation 8.3 Innovations 8.4 Future work APPENDICESA. Parameter effects in the game of Chung ToiB. Design of experiments for the mountain car problem C. Supporting tables
This thesis takes an empirical approach to understanding of the behavior and interactions between the two main components of reinforcement learning: the learning algorithm and the functional representation of learned knowledge. The author approaches these entities using design of experiments not commonly employed to study machine learning methods. The results outlined in this work provide insight as to what enables and what has an effect on successful reinforcement learning implementations so that this learning method can be applied to more challenging problems.
Introduction.- Reinforcement Learning. Design of Experiments.- Methodology.- The Mountain Car Problem.- The Truck Backer-Upper Problem.- The Tandem Truck Backer-Upper Problem.- Appendices.
Über den Autor
Christopher Gatti received his PhD in Decision Sciences and Engineering Systems from Rensselaer Polytechnic Institute (RPI). During his time at RPI, his work focused on machine learning and statistics, with applications in reinforcement learning, graph search, stem cell RNA analysis, and neuro-electrophysiological signal analysis. Prior to beginning his graduate work at RPI, he received a BSE in mechanical engineering and an MSE in biomedical engineering, both from the University of Michigan. He then continued to work at the University of Michigan for three years doing computational biomechanics focusing on the shoulder and knee. He has been a gymnast since he was a child and is currently an acrobat for Cirque du Soleil.
Inhaltsverzeichnis
GLOSSARY
ACKNOWLEDGMENT
FOREWARD
1. INTRODUCTION
2. REINFORCEMENT LEARNING
2.1 Applications of reinforcement learning
2.1.1 Benchmark problems
2.1.2 Games
2.1.3 Real-world applications
2.1.4 Generalized domains
2.2 Components of reinforcement learning
2.2.1 Domains
2.2.2 Representations
2.2.3 Learning algorithms
2.3 Heuristics and performance effectors
3. DESIGN OF EXPERIMENTS
3.1 Classical design of experiments
3.2 Contemporary design of experiments
3.3 Design of experiments for empirical algorithm analysis
4. METHODOLOGY
4.1 Sequential CART
4.1.1 CART modeling
4.1.2 Sequential CART modeling
4.1.3 Analysis of sequential CART
4.1.4 Empirical convergence criteria
4.1.5 Example: 2-D 6-hump camelback function
4.2 Kriging metamodeling
4.2.1 Kriging
4.2.2 Deterministic kriging
4.2.3 Stochastic kriging
4.2.4 Covariance function
4.2.5 Implementation
4.2.6 Analysis of kriging metamodels
5. THE MOUNTAIN CAR PROBLEM
5.1 Reinforcement learning implementation
5.2 Sequential CART
5.3 Response surface metamodeling
5.4 Discussion
6. THE TRUCK BACKER-UPPER PROBLEM
6.1 Reinforcement learning implementation
6.2 Sequential CART
6.3 Response surface metamodeling
6.4 Discussion
7. THE TANDEM TRUCK BACKER-UPPER PROBLEM
7.1 Reinforcement learning implementation
7.2 Sequential CART
7.3 Discussion
8. DISCUSSION
8.1 Reinforcement learning
8.2 Experimentation
8.3 Innovations
8.4 Future work
APPENDICES
A. Parameter effects in the game of Chung Toi
B. Design of experiments for the mountain car problem
C. Supporting tables
Klappentext
This thesis takes an empirical approach to understanding of the behavior and interactions between the two main components of reinforcement learning: the learning algorithm and the functional representation of learned knowledge. The author approaches these entities using design of experiments not commonly employed to study machine learning methods. The results outlined in this work provide insight as to what enables and what has an effect on successful reinforcement learning implementations so that this learning method can be applied to more challenging problems.
Nominated by the Rensselaer Polytechnic Institute as an outstanding Ph.D. thesis
rn
Explains reinforcement learning through a range of problems by exploring what affects reinforcement learning and what contributes to a successful implementation
rn
Includes a contemporary design of experiments methods, comprising of a novel sequential experimentation procedure that finds convergent learning algorithm parameter subregions and stochastic kriging for response surface metamodeling
rn
Includes supplementary material: sn.pub/extras
rn
