Systems Seminar Talk<br><br>Title: Large Batch Simulation for Deep Reinforcement Learning<br>Speaker: Brennan Shacklett, advised by Kayvon Fatahalian<br>Date: April 8<br>Time: 11:00am<br>Event link: <b><a href="https://stanford.zoom.us/j/97905322347?pwd=Z3B5Q1N5dW9BNTk1YS90SDZIUzN6Z... present a reinforcement learning system for visually complex 3D environments built around a custom simulator design that processes large batches of simulated environments simultaneously. This batch simulation strategy allows GPU resources to be efficiently leveraged by amortizing memory and compute costs across multiple simulated agents, dramatically improving the number of simulated environments per GPU and overall simulation throughput. Our implementation of navigation trains agents on the Gibson dataset at 19,000 frames of experience per second on a single GPU (and up to 72,000 frames per second on a single eight-GPU machine) – more than 100x faster than prior work in the same environments. In terms of end-to-end training, policies can be trained to convergence in 1.5 days on a single GPU to 97% of the accuracy of agents trained on a prior state-of-the-art system using a 64-GPU cluster over three days. This talk will describe the architecture of our batch simulator and our strategy of end-to-end optimization throughout the entire reinforcement learning system.