Gradient Tracer

Gradient Tracer is a Python library designed to leverage automatic differentiation capabilities of established libraries (e.g. PyTorch) to compute gradients for tensor expressions, execute expressions and backpropagation, and store the results serving as reference data for testing other implementation. This library aims to simplify the process of tracing and logging gradients, making it easier to analyze and debug custom implemenation of machine learning models.

Features

• Automatic Differentiation : Utilizes PyTorch’s autograd to compute gradients for tensor expressions.

• Sample Processing: Processes samples written in Python following logical file and directories structure and naming, executes expressions, and performs backpropagation.

• Result Logging: Stores the computed gradients, operations directed graphs and results into following test structure files for easy analysis and debugging.

Project structure

gradient_tracer/
├── gradient_tracer/
│   ├── __init__.py
│   ├── core.py
│   ├── tests/
│   │   ├── __init__.py
│   │   └── test_core.py
│   ├── docs/
│   │   └── index.adoc
│   └── examples/
│       └── example_usage.py
├── README.adoc
├── LICENSE
├── setup.py
└── requirements.txt

Usage

Note	This library is still under heavy development.

• create your working folder

• add subfolders with test suites and test cases following file names convention (see more about Executable annotation in docs)

• create your virtual environment e.g. with pipenv --python 3.12

• install gradienttracer from source code pipenv install git+https://github.com/michalharakal/gradienttracer@main#egg=gradienttracer

• activate shell pipenv shell

• run cli command gt --help to commandline parameters

Attribution

This project is inspired by Andrej Karpathy’s work on micrograd. Check out the original project here: micrograd by Andrej Karpathy.

License

Gradient Tracer is released under the MIT License. See the link LICENSE file for more details.