Stefan Hadjis

Digital Systems, High-Level Synthesis

• S. Hadjis, A. Canis, R. Sobue, Y. Hara-Azumi, H. Tomiyama, J. Anderson
  Profiling-driven multi-cycling in FPGA high-level synthesis
  ACM/IEEE Design Automation and Test in Europe Conference (DATE), Grenoble, France, March 2015.
  Selected as a long (30 minute) talk
  [bib]  [PDF]  [Slides]
  
  
• B. Fort, A. Canis, J. Choi, N. Calagar, R. Lian, S. Hadjis, Y.T. Chen, M. Hall, B. Syrowik, T. Czajkowski, S. Brown, J.H. Anderson
  Automating the Design of Processor/Accelerator Embedded Systems with LegUp High-Level Synthesis
  IEEE International Conference on Embedded and Ubiquitous Computing (EUC), Milan, Italy, August 2014.
  [PDF]
  
  
• S. Hadjis, A. Canis, J.H. Anderson, J. Choi, K. Nam, S. Brown, T. Czajkowski
  Impact of FPGA Architecture on Resource Sharing in High-Level Synthesis
  ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA), Monterey, CA, February 2012.
  [bib]  [PDF]  [Slides]  [Poster]
  

Accelerators and Compilers

• Stefan Hadjis, Kunle Olukotun
  TensorFlow to Cloud FPGAs: Tradeoffs for Accelerating Deep Neural Networks
  Proceedings of the 29th IEEE International Conference on Field Programmable Logic and Applications (FPL), Barcelona, Spain, September 2019.
  [bib]  [PDF]  [github]
  
  
• David Koeplinger, Matt Feldman, Raghu Prabhakar, Yaqi Zhang, Stefan Hadjis, Ruben Fiszel, Tian Zhao, Luigi Nardi, Ardavan Pedram, Christos Kozyrakis, Kunle Olukotun
  Spatial: A Language and Compiler For Application Accelerators
  Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Philadelphia, USA, June 2018.
  [PDF]
  
  
• Raghu Prabhakar, Yaqi Zhang, David Koeplinger, Matt Feldman, Tian Zhao, Stefan Hadjis, Ardavan Pedram, Christos Kozyrakis, Kunle Olukotun
  Plasticine: A Reconfigurable Architecture For Parallel Patterns
  Proceedings of the 44th International Symposium on Computer Architecture (ISCA), Toronto, Canada, June 2017.
  Also selected as IEEE Micro Top Picks 2018
  [PDF]
  

Machine Learning Systems

• Stefan Hadjis, Ce Zhang, Ioannis Mitliagkas, Christopher Ré
  Omnivore: An Optimizer for Multi-device Deep Learning on CPUs and GPUs
  Also mentioned in O'Reilly, HPCWire
  arXiv, 2016.
  [bib]  [PDF]  [github]  [FAQ]
  
  
• Stefan Hadjis, Firas Abuzaid, Ce Zhang, Christopher Ré
  Caffe con Troll: Shallow Ideas to Speed up Deep Learning
  ACM SIGMOD workshop on Data Analytics at Scale (DanaC), Melbourne, Australia, May 2015
  Also mentioned in this blog article
  [bib]  [PDF]  [Slides]  [github]
  

Machine Learning Theory

• Ioannis Mitliagkas, Ce Zhang, Stefan Hadjis, Christopher Ré
  Asynchrony begets Momentum, with an Application to Deep Learning
  arXiv, 2016.
  [PDF]
  
  
• Stefan Hadjis, Stefano Ermon
  Importance Sampling over Sets: A New Probabilistic Inference Scheme
  Conference on Uncertainty in Artificial Intelligence (UAI), Amsterdam, Netherlands, July 2015
  [bib]  [PDF]  [Code (16 MB)]  [Poster]  [Summary]
  

Medicine

• N'djin WA, Burtnyk M, Kobelevskiy I, Hadjis S , Bronskill M, Chopra R
  Coagulation of human prostate volumes with MRI-controlled transurethral ultrasound therapy: Results in gel phantoms
  International Journal of Medical Physics Research and Practice, 2012 Jul;39(7):4524-36.
  [PubMed] [Poster]
  
  

Research Software

• Spatial: A Language and Compiler for Application Accelerators [More information]
  Spatial is a language which allows programmers to design efficient accelerators from a high level of abstraction. Spatial can target CGRAs and FPGAs, including the F1 FPGAs on the Amazon Elastic Compute Cloud which I added support for (paper).
  
  
• The Spatial Multiverse [More information]
  The Spatial Multiverse exists around the Spatial Language and Compiler. It provides Spatial with front-end support from popular machine learning (ML) frameworks. The Spatial Multiverse contains (1) Spatial as a submodule, (2) tools to compile ML frameworks to Spatial in order to target hardware accelerators, and (3) examples of models and applications in these frameworks. This allows ML frameworks to be mapped to programmable hardware. In its initial version, the Spatial Multiverse supports mapping DNN models described in TensorFlow to FPGAs (paper).
  
  
• Omnivore: A Distributed Deep Learning Training Optimizer [More information]
  This is a deep learning training optimizer which reduces total training time by considering both (1) the time per iteration and (2) the number of iterations to convergence. A discovery of this work was that asynchrony in stochastic gradient descent training can be compensated for by decreasing momentum, such that there is reduced statistical penalty resulting from asynchronous training. While learning rate is often tuned during training, momentum is often fixed at 0.9. Another discovery was that it is often best to group machines such that within a group gradients are computed synchronously, but different groups compute gradients asynchronously. The optimizer selects learning hyper-parameters and the size of these compute groups (i.e. the degree of asynchrony in the system). Along with efficient hardware mapping and pruning of the hyper-parameter space, this leads to significant speedups over state-of-the-art tools, and scales to more than 100 GPUs (paper).
  
  
• Maximizing CPU Efficiency in Deep Learning [More information]
  Researchers often use GPUs for Deep Learning training and claim that GPUs are much faster than CPUs than is actually the case. Often this is because Deep Learning frameworks use sub-optimal CPU implementations which achieve a smaller percentage of the peak device throughput on the CPU than on the GPU. In this work we optimize DNN training for CPUs and, when targeting CPUs, show order-of-magnitude speedups compared to state-of-the-art frameworks. We also show that, with these optimizations, CPUs and GPUs can be used together to accelerate training (paper, paper, slides).

Software Tutorials

• BLAS-level CPU Performance in 100 Lines of C [More information]
  General Dense Matrix Multiplication.or GEMM is at the heart of many applications including Deep Learning. A common misconception is that BLAS code is very complicated, but in this tutorial I show that you can get BLAS-speed using only 100 lines of C (and no assembly programming).