Research

Interests: Reinforcement Learning, Continual learning, Deep learning.

My research objective is to understand the computational principles underlying intelligent decision-making. To this end, most of my research is in reinforcement learning (RL). In the past I have done projects in a variety of settings including: deep RL, inverse RL, goal-conditioned RL, game-theoretic online learning for sequential decision-making, temporal abstraction, and RL for language model reasoning.

Talks | Code

Talks

  • Recent Insights in Value-based Deep Reinforcement Learning (Slides) | (Video)
    Amii AI Seminar surveying the Policy Churn, The Tandem effect, and the curse of diversity.

  • Revisiting Overestimation in Value-based Deep Reinforcement Learning (Slides) | (Video)
    Amii AI Seminar on a preliminary version of Deep Double Q-learning. Includes a tutorial on value overestimation.

  • Trajectory-Ranked Reward Extrapolation (Slides )
    Internal talk I gave on our ICML 2019 work at Preferred Networks.

  • Deterministic Implementations for Reproducibility in Deep Reinforcement Learning (Slides)
    Talk given at the AAAI 2019 Workshop on Reproducibility in AI for our paper.

Code

  • PFRL
    PFRL is a PyTorch deep reinforcement learning library. Part of my work at Preferred Networks was devoted to developing algorithms and infrastructure for PFRL.
  • Table-RL
    Table-rl is a library for tabular reinforcement learning in Python.
  • MAML
    A PyTorch implementation of Model Agnostic Meta Learning (MAML) on the Sinusoid task.
  • Atari Behavioral Cloning
    A Behavioral cloning implementation on Atari. This was adapted for our T-REX paper and modeled after DQfD's Behavioral cloning.
  • Bayesian Inverse RL
    A basic implementation of Bayesian Inverse Reinforcement Learning.