Andrej Karpathy
Stanford Ph.D. student, Computer Science

email: karpathy _at_ cs.stanford.edu
office: Gates 242
association: Stanford AI Lab

advisor: Fei-Fei Li
rotation advisors: Daphne Koller, Andrew Ng, Vladlen Koltun, Sebastian Thrun

Bio

2005-2009 Undergrad: CS, Physics and Math at University of Toronto.
2009-2011 Masters: CS at University of British Columbia. I worked with Michiel van de Panne on Motor Control in physically simulated articulated figures.
Summer 2011 and Summer 2013, Google Research: Large-Scale Deep Learning for Spatio-Temporal Feature Learning.

Research Interests

My main research interests are in Machine Learning (with emphasis on deep learning) and my preferred application domain is Vision. I like my data large, my algorithms simple, and my labels weak. I maintain a mostly academic blog for fun. I wish all researchers had one. (Btw: I recently upgraded to a new, Jekyll-based blog at karpathy.github.io. I will be publishing there from now on.)

Misc Interests

As far as my non-academic interests go I write quite a bit of Web/Javascript stuff, iOS/Android apps, random simulations of physical and biological systems, I have entrepreneurial inclinations, I think a lot about the brain/neuroscience/AI, evolution/genetics, tech/gadgets, design, education, robots, space exploration, lifelogging/personal analytics, healthy life style/diet, sustainable energy, and I solve Rubik's cubes under 20 seconds.

Publications


Deep Fragment Embeddings for Bidirectional Image Sentence Mapping
Andrej Karpathy, Armand Joulin, Li Fei-Fei
arXiv (2014). [pdf] [supplemental] [bibtex]

Large-Scale Video Classification with Convolutional Neural Networks
Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, Li Fei-Fei
CVPR 2014 (oral). [project] [pdf]
(includes Sports-1M dataset release of 1.1 million videos)

Grounded Compositional Semantics for Finding and Describing Images with Sentences
Richard Socher, Andrej Karpathy, Quoc V. Le, Christopher D. Manning, Andrew Y. Ng
TACL 2013 [pdf]

Object Discovery in 3D scenes via Shape Analysis
Andrej Karpathy, Stephen Miller, Li Fei-Fei
ICRA 2013. [pdf,code,data]
Emergence of Object-Selective Features in Unsupervised Feature Learning
Adam Coates, Andrej Karpathy, Andrew Ng
NIPS 2012. [PDF]
Curriculum Learning for Motor Skills
Andrej Karpathy, Michiel van de Panne
AI 2012. [full PDF] [short PDF] [video]
Locomotion Skills for Simulated Quadrupeds
Stelian Coros, Andrej Karpathy, Benjamin Jones, Lionel Reveret, Michiel van de Panne
SIGGRAPH 2011. [project web page] [video]
Teaching Assistant

Stanford Fall 2011, CS 229A: Machine Learning Online Class (assignments)
UBC Spring 2011, CPSC 540 : Graduate Probabilistic Machine Learning
UBC, 2010-2011, 3 x CPSC 121: Discrete Mathematics (taught tutorials)

 

Course Work

Coursera:
Genetics and Evolution (statement of accomplishment! :p)
Epigenetics (statement of accomplishment) and my rough class notes

Stanford: Maching Learning (Andrew Ng), Computer Vision (Fei Fei Li), Probabilistic Graphical Models (Daphne Koller, Kevin Murphy), Convex Optimization (Stephen Boyd), Advanced Probabilistic Graphical Models (Kevin Murphy), Startup (Peter Thiel), and Tennis :p

UBC: Machine Learning I (Nando de Freitas), Machine Learning II (Kevin Murphy), Computer Vision I (Bob Woodham), Computer Vision II (David Lowe)

 

Scaling for Multimodal 3D Object Detection (C++)
This was a fun course project for my computer vision class at Stanford. Full report is here in PDF.
Rubik's cube color extractor (Python)
Computer Vision class project for David Lowe's vision class at UBC.
Library for training binary Restricted Boltzmann Machines (MATLAB)
This course project ended up being incorporated into Kevin Murphy's pmtk3 Machine Learning toolbox.

 

Coding
side projects
ConvNetJS: Deep Learning in Javascript Lets you train Convolutional (or ordinary) Neural Networks entirely in your browser with Javascript. Demos show a CNN classifying MNIST, CIFAR-10, 1D/2D toy data classification/regression, Reinforcement Learning with Deep Q Learning.
svmjs: SVMs in Javascript SMO implementation, arbitrary kernels, and also pretty Canvas + HTML5 GUI for visualizing SVMs.
tsnejs: t-SNE visualization algorithm implementation in Javascript.
ulogme is a tool for quantifying productivity. It tracks your active windows / keystroke frequencies / notes throughout the entire day and visualizes the results in beautiful d3js timelines. #quantifiedself #d3js
Research Lei (Python, Javascript) is an Academic Papers Management and Discovery System. It helps researchers build, maintain, and explore academic literature more efficiently, in the browser. (deprecated since Microsoft Academic Search API was shut down :( )
NIPS 2012 accepted papers (Javascript+Python), in a much nicer format. Paper thumbnails, LDA topic analysis, tf-idf similar papers ranking. Helps with exploring newly published papers. This idea was adopted by @benhamner for ICML 2013 and Colin Lea for CVPR 2013.
New: now also NIPS 2013, and CVPR 2014.
A while ago I created a Google Glass Explorer's program winners page and the internet liked it. (Python, Javascript hack)
Evolutionary Artificial Life Simulator of Predator-Prey dynamics (C++)
simulates evolution of little organisms with neural network brains who can sense and hunt each other.
iPhone app (Objective C). Made over $20,000 on this so far :). I've also ported it to an Android app (phew!).
I also published a 2-4 player iPad game called Loud Snakes.
Sketcher [video] (Python) simulates a person drawing an image.
Tetris AI [video] (Python) is a Tetris AI player that can hold its own for few hours.
- Also check out my Multiplayer Co-op Tetris Lots of fun! :)
- Update: I now also ported the co-op Tetris to Javascript and to work over network in browser using node.js and socket.io. Play here.
4 seasons animation [video] (Python) simulates a tree through 4 seasons and looks pretty.

- Many more crappy projects I worked on long time ago
- and some more on my github


Lectures
A while ago I made some educational videos on YouTube for fun. I like explaining:
Solving the Rubik's cube :p (Beginner to Expert) (I'm qualified because I solve cubes in ~17seconds on average (example video from a competition here). These now have 5 million+ views, which is kinda nice.)
Game Programming in Python Tutorials. Just an experiment. I would rather teach Javascript + HTML5 for game making now.
Fractal Mathematics Tutorials.

Personal

- Advice for doing well in undergrad classes
- My Speedcubing page. This now averages about ~1200 views/day, I'm happy that people find it useful when learning to cube.
- OPML of some of the blogs I follow: RSS feeds (can import to RSS readers)