-
Notifications
You must be signed in to change notification settings - Fork 12
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This reverts commit f34450e.
- Loading branch information
Showing
122 changed files
with
21,937 additions
and
4,202 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,4 +3,3 @@ | |
build | ||
.DS_Store | ||
.vscode | ||
source/api |
This file was deleted.
Oops, something went wrong.
Submodule Enzyme
deleted from
6b39d4
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
--- | ||
title: "{{ replace .Name "-" " " | title }}" | ||
date: {{ .Date }} | ||
draft: true | ||
--- | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
baseURL = "https://enzyme.mit.edu/" | ||
languageCode = "en-us" | ||
title = "Enzyme AD" | ||
theme = "mlir" | ||
|
||
hasCJKLanguage = true | ||
metaDataFormat = "yaml" | ||
|
||
pygmentsCodeFences = true | ||
pygmentsUseClasses = true | ||
|
||
defaultContentLanguage = "en" | ||
defaultContentLanguageInSubdir= false | ||
enableMissingTranslationPlaceholders = false | ||
|
||
disablePathToLower = true | ||
|
||
[Params] | ||
|
||
# Souce Code repository section | ||
description = "Enzyme Automatic Differentiation Framework" | ||
github_repository = "https://github.com/wsmoses/Enzyme" | ||
version = "" | ||
|
||
# Documentation repository section | ||
# documentation repository (set edit link to documentation repository) | ||
# github_doc_repository = "https://github.com/llvm/mlir-www/" | ||
|
||
# Analytic section | ||
google_analytics_id = "" # Your Google Analytics tracking id | ||
tag_manager_container_id = "" # Your Google Tag Manager container id | ||
google_site_verification = "" # Your Google Site Verification for Search Console | ||
|
||
# Theme settings section | ||
# Theme color | ||
custom_font_color = "" | ||
custom_background_color = "" | ||
|
||
# Documentation Menu section | ||
# Menu style settings | ||
menu_style = "slide-menu" # "open-menu" or "slide-menu" | ||
|
||
# Date format | ||
dateformat = "" # default "2 Jan 2006" | ||
# See the format reference https://gohugo.io/functions/format/#hugo-date-and-time-templating-reference | ||
|
||
# path name excluded from documentation menu | ||
menu_exclusion = [ | ||
"archives", | ||
"archive", | ||
"blog", | ||
"entry", | ||
"post", | ||
"posts" | ||
] | ||
|
||
# Global menu section | ||
[[menu.main]] | ||
name = "Community" | ||
weight = 3 | ||
|
||
[[menu.main]] | ||
name = "Discussion List" | ||
parent = "Community" | ||
url = "https://groups.google.com/d/forum/enzyme-dev" | ||
weight = 2 | ||
|
||
[[menu.main]] | ||
name = "Source" | ||
weight = 10 | ||
url = "https://github.com/wsmoses/Enzyme" | ||
|
||
[[menu.main]] | ||
name = "Bugs" | ||
weight = 20 | ||
url = "https://github.com/wsmoses/Enzyme/issues" | ||
|
||
[[menu.main]] | ||
name = "Try Online" | ||
weight = 30 | ||
url = "/explorer" | ||
|
||
[markup] | ||
[markup.goldmark] | ||
[markup.goldmark.renderer] | ||
hardWraps = false | ||
unsafe = true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,89 @@ | ||
--- | ||
title: "Installation" | ||
date: 2019-11-29T15:26:15Z | ||
draft: false | ||
weight: 5 | ||
--- | ||
|
||
You can install Enzyme using [Homebrew](https://brew.sh), or build it manually from source. To install using Homebrew, run | ||
|
||
``` | ||
brew install enzyme | ||
``` | ||
|
||
Currently, Homebrew has a pre-built binary (bottle) on macOS, but will build from source on Linux. To request a bottle for Linux, [file an issue](https://github.com/Homebrew/linuxbrew-core/issues) at Homebrew. | ||
|
||
The rest of these instructions explain how to build Enzyme from source. | ||
|
||
## Downloading Enzyme | ||
To start you should download Enzyme's code [Github](https://github.com/wsmoses/Enzyme). | ||
|
||
```sh | ||
git clone https://github.com/wsmoses/Enzyme | ||
cd Enzyme | ||
``` | ||
|
||
|
||
## Building LLVM | ||
|
||
Enzyme is a plugin for LLVM and consequently needs an existing build of LLVM to function. | ||
|
||
Enzyme is designed to work with a wide range of LLVM versions and is currently tested against LLVM 7, 8, 9, 10, 11, 12, and mainline. LLVM's plugin infrastructure can sometimes be flakey or not built by default. If loading Enzyme into an existing LLVM installation results in segfaults, we recommend building LLVM from source. | ||
|
||
Details on building LLVM can be found in for building LLVM can be found in the [LLVM Getting Started](https://llvm.org/docs/GettingStarted.html). A simple build command using Ninja is shown below: | ||
|
||
```sh | ||
cd /path/to/llvm/source/ | ||
mkdir build && cd build | ||
cmake -G Ninja ../llvm -DLLVM_TARGETS_TO_BUILD="host" -DLLVM_ENABLE_PROJECTS="clang" -DLLVM_ENABLE_PLUGINS=ON -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON | ||
ninja | ||
``` | ||
|
||
## Building Enzyme | ||
|
||
First enter the enzyme project directory inside the monorepo and create a build directory. Note that the source is inside a subdirectory of the repo called enzyme. If you get a CMakeLists.txt not found error, make sure you're pointing at the enzyme subdirectory. | ||
|
||
```sh | ||
cd /path/to/Enzyme/enzyme | ||
mkdir build && cd build | ||
``` | ||
|
||
From here, we can configure Enzyme via cmake, then build. Again, for ease we use Ninja (requiring the ninja build to to be installed). One can use make by omitting `-G Ninja` and running make instead of ninja. | ||
|
||
```sh | ||
cmake -G Ninja .. -DLLVM_DIR=/path/to/llvm/lib/cmake/llvm | ||
ninja | ||
``` | ||
|
||
This should create a file `Enzyme/LLVMEnzyme-<VERSION>.so` inside the build directory, which contains the LLVM analysis and optimization passes that implement Enzyme. | ||
|
||
|
||
|
||
## Verifying installation | ||
|
||
We can run Enzyme's unit tests by running the following command. They should run in less than a minute and verify that your build of Enzyme and LLVM interoperate properly. | ||
|
||
```sh | ||
ninja check-enzyme | ||
``` | ||
|
||
We can also run Enzyme's C/C++ integration tests. These tests require an existing installation of the Adept AD Engine (to compare against) and Eigen. Running these tests will take a moderate amount of time (about 6 minutes on a recent multicore). | ||
|
||
```sh | ||
ninja check-enzyme-integration | ||
``` | ||
|
||
Finally, we can also run Enzyme's benchmarking suite, which is composed of the reverse mode tests from Microsoft's [ADBench suite](https://github.com/microsoft/ADBench) and other interesting cases. Running these tests will potentially take a long time (about an hour on a recent multicore). | ||
|
||
```sh | ||
ninja bench-enzyme | ||
```` | ||
|
||
If you run Enzyme tests and get an error like `/bin/sh: 1: ../../: Permission denied` or ` ../../ not found`, it's likely that cmake wasn't able to find your version of llvm-lit, LLVM's unit tester. This often happens if you use the default Ubuntu install of LLVM as they stopped including it in their packaging. To remedy, find lit.py or lit or llvm-lit on your system and add the following flag to cmake: | ||
```sh | ||
cmake .. -DLLVM_EXTERNAL_LIT=/path/to/lit/lit.py | ||
``` | ||
## Developing inside a Container | ||
For debugging and testing purposes we have created [Docker images](https://github.com/tgymnich/enzyme-dev-docker), which closely resemble all of our CI environments. If you are using Visual Studio Code you can build and test Enzyme inside of a [Dev Container](https://code.visualstudio.com/docs/remote/containers). To change either the Ubuntu or the LLVM version in Visual Studio Code just edit the file at `.devcontainer/devcontainer.json` accordingly. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,85 @@ | ||
--- | ||
date: 2017-10-19T15:26:15Z | ||
lastmod: 2019-10-26T15:26:15Z | ||
publishdate: 2018-11-23T15:26:15Z | ||
--- | ||
|
||
# Enzyme Overview | ||
|
||
The Enzyme project is a tool that takes arbitrary existing code as LLVM IR and computes the derivative (and gradient) of that function. This allows developers to use Enzyme to automatically create gradients of their source code without much additional work. By working at the LLVM level Enzyme is able to differentiate programs in a variety of languages (C, C++, Swift, Julia, Rust, Fortran, TensorFlow, etc) in a single tool and achieve high performance by integrating with LLVM's optimization pipeline. | ||
|
||
```c | ||
#include <stdio.h> | ||
|
||
double square(double x) { | ||
return x * x; | ||
} | ||
|
||
double __enzyme_autodiff(void*, double); | ||
int main() { | ||
double x = 3.14; | ||
// Evaluates to 2 * x = 6.28 | ||
double grad_x = __enzyme_autodiff((void*)square, x); | ||
printf("square'(%f) = %f\n", x, grad_x); | ||
} | ||
``` | ||
By differentiating code after optimization, Enzyme is able to create substantially faster derivatives than existing tools that differentiate programs before optimization. | ||
<div style="padding:2em"> | ||
<img src="/all_top.png" width="500" align=center> | ||
</div> | ||
## Components | ||
Enzyme is composed of four pieces: | ||
* An optional preprocessing phase which performs minor transformations that tend to be helpful for AD. | ||
* A new interprocedural type analysis that deduces the underlying types of memory locations | ||
* An activity analaysis that determines what instructions or values can impact the derivative computation (common in existing AD systems). | ||
* An optimization pass which creates any required derivative functions, replacing calls to `__enzyme_autodiff` with the generated functions. | ||
## More resources | ||
For more information on Enzyme, please see: | ||
* The Enzyme [getting started guide](/getting_started/) | ||
* The Enzyme [mailing list](https://groups.google.com/d/forum/enzyme-dev) for any questions. | ||
* Previous [talks](/talks/). | ||
* You can try out Enzyme on our [Compiler Explorer instance](/explorer). | ||
## Citing Enzyme | ||
To cite Enzyme, please cite the following two papers (first for Enzyme as a whole, then for GPU+optimizations): | ||
``` | ||
@inproceedings{NEURIPS2020_9332c513, | ||
author = {Moses, William and Churavy, Valentin}, | ||
booktitle = {Advances in Neural Information Processing Systems}, | ||
editor = {H. Larochelle and M. Ranzato and R. Hadsell and M. F. Balcan and H. Lin}, | ||
pages = {12472--12485}, | ||
publisher = {Curran Associates, Inc.}, | ||
title = {Instead of Rewriting Foreign Code for Machine Learning, Automatically Synthesize Fast Gradients}, | ||
url = {https://proceedings.neurips.cc/paper/2020/file/9332c513ef44b682e9347822c2e457ac-Paper.pdf}, | ||
volume = {33}, | ||
year = {2020} | ||
} | ||
@inproceedings{10.1145/3458817.3476165, | ||
author = {Moses, William S. and Churavy, Valentin and Paehler, Ludger and H\"{u}ckelheim, Jan and Narayanan, Sri Hari Krishna and Schanen, Michel and Doerfert, Johannes}, | ||
title = {Reverse-Mode Automatic Differentiation and Optimization of GPU Kernels via Enzyme}, | ||
year = {2021}, | ||
isbn = {9781450384421}, | ||
publisher = {Association for Computing Machinery}, | ||
address = {New York, NY, USA}, | ||
url = {https://doi.org/10.1145/3458817.3476165}, | ||
doi = {10.1145/3458817.3476165}, | ||
booktitle = {Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis}, | ||
articleno = {61}, | ||
numpages = {16}, | ||
keywords = {CUDA, LLVM, ROCm, HPC, AD, GPU, automatic differentiation}, | ||
location = {St. Louis, Missouri}, | ||
series = {SC '21} | ||
} | ||
|
||
``` | ||
The original Enzyme is also avaiable as a preprint on [arXiv](https://arxiv.org/pdf/2010.01709.pdf). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
--- | ||
title: "Charter" | ||
date: 2019-11-29T15:26:15Z | ||
draft: false | ||
weight: 10 | ||
--- | ||
|
||
Enzyme is a project for high-performance automatic differentiation. The Enzyme community aims to be open and welcoming. If you'd like to participate, you can do so in a number of ways. | ||
|
||
* Join our [mailing list](https://groups.google.com/d/forum/enzyme-dev) | ||
* Join our weekly open-design call (see mailing list for meeting link) | ||
* Participate in development on [Github](https://github.com/wsmoses/Enzyme) | ||
|
||
|
||
# Abstract | ||
|
||
# Introduction | ||
|
||
# Type Analysis | ||
|
||
# Activity Analysis | ||
|
||
# Interoperability | ||
|
Oops, something went wrong.