Research

Interests: Reinforcement Learning

My research objective is to understand the computational principles underlying intelligent decision-making. To this end, most of my research is in reinforcement learning (RL). In the past I have done projects in a variety of settings including: deep RL, inverse RL, goal-conditioned RL, game-theoretic online learning for sequential decision-making, temporal abstraction, and RL for language model reasoning.

Projects | Talks

Talks

  • Recent Insights in Value-based Deep Reinforcement Learning
    Amii AI Seminar
    (Slides) | (Video)
  • Revisiting Overestimation in Value-based Deep Reinforcement Learning
    Amii AI Seminar
    (Slides) | (Video)
  • Trajectory-Ranked Reward Extrapolation
    Internal talk at Preferred Networks
    (Slides)
  • Deterministic Implementations for Reproducibility in Deep Reinforcement Learning
    AAAI 2019 Workshop on Reproducibility in AI
    (Slides)

Projects

Deep Reinforcement Learning


My Master's thesis focused on reproducibility in deep reinforcement learning. Specifically, I studied the impact of nondeterminism in algorithm implementations on our ability to reproduce results.

Publications

Robotics


At Preferred Networks I have worked on targeted grasping for robotics. We combine techniques from goal-conditioned reinforcement learning (hindsight experience replay), deep reinforcement learning (QT-OPT), and distributed training to achieve target grasping on a human support robot.

Publications

Inverse Reinforcement Learning


We leverage ranked demonstrations to improve upon a suboptimal demonstrator in high-dimensional deep reinforcement learning tasks.

Publications