Vitis Accelerator IP Flow #1134

steltze · 2024-11-20T13:28:28Z

WIP. Opening for feedback.

Vitis Accelerator based on the IP Flow for Zynq devices (in contrast to #991 ), similar to the Vivado Accelerator.

Inherits from the Vitis backend
Generates the AXI wrapper around the typical hls4ml IP and the test bench
Automatically creates a Vivado project with the block design, imports the IP, and generates the bitstream

Tested on

Vivado and Vitis HLS v2023.2
ZCU102 (pynq v3 ported on an ubuntu image) and PYNQ-Z2 (official v3.0.1 image) using AXI-Stream and io_stream and io_parallel with a small MLP
Tested the Vivado/VivadoAccelerator backends as well to check that nothing broke

To dos:

Integrate AXI-Master interface as an option
Include tar generation with the bitstream, design files for pynq and the python driver for pynq
Include FIFO Depth Optimization as a backend pass (similar to FIFO depth optimizer for Vitis backend #1037 )

Type of change

New feature (non-breaking change which adds functionality)

Tests

Pending unit tests

Test Configuration:

Checklist

I have read the guidelines for contributing.
I have commented my code, particularly in hard-to-understand areas.
I have made corresponding changes to the documentation.
My changes generate no new warnings. Some warnings occur due to ap_axi_sdata.h
I have installed and run pre-commit on the files I edited or added.
I have added tests that prove my fix is effective or that my feature works.

…rning#956) * Stage initial set of changes for the Catapult backend * applied some changes for issues reported by pre-commit. But pre-commit still reorders backends/__init__.py incorrectly * final changes for clean pre-commit * minor edits * Checkin * Add file * pre-commit format * add in nnet_utils files * format changes for pre-commit * run flows by netlist type * update design pragmas on some blocks. cleaned up TCL script * move AC submodules under hls4ml/templates/catapult * merged in latest changes from mainline * remove bad submodules * recreate AC submodules in hls4ml/templates/catapult * pre-commit fixes * pre-commit fixes * turn on Catapult backend testing * removed io_parallel testing for Catapult backend * add Catapult * added Catapult * added Catapult * added Catapult to some pytests * Added concept of ProjectDir to distinguish the project directory of the HLS tool from the ProjectName which is used for the cpp file and top function name * better handling of c++ testbench data files. enhanced directory naming. * fix syntax * workaround from Giuseppe * Add concept of ProjectDir for Catapult which is different from ProjectName that gets used for the top function name and the cpp files * add new file from Giuseppe * improvements to project management, reporting and testbench * include new file in generation of parameters.h * add hard_tanh for io_parallel. formatting * Full path to the header nnet_helpers.h is necessary in the include (check if this is not an issue with recent versions of Catapult) * Avoid ceiling function from the math library: ceil(n/d) ---> (n+d-1)/n * These are mostly workarounds for the BUP synyhesis of a testing model (should these changes make in something more general?) * revert format back to what clang-format currently enforces * simplification from Giuesspe * Fixes for bottom-up handling of libraries * pre-commit format fixes * fix loops * consolidate prj scripts * cleanup pragmas * switch from using ssh to https for submodules * fix include path for non-catapult install * update pytest environment * CL 1100381 * CL 1098112 * roll in latest changes. turn off Catapult variants of test_binary_cnn and test_cnn_mnist_qkeras for now * fix test failure * disable Catapult test for pytorch until it is supported * disable Catapult for pytorch tests * Simpler submodule initialization for CI --------- Co-authored-by: David Burnette <[email protected]> Co-authored-by: Giuseppe Di Guglielmo <[email protected]> Co-authored-by: Jovan Mitrevski <[email protected]> Co-authored-by: Vladimir Loncar <[email protected]>

updates: - [github.com/psf/black: 24.3.0 → 24.4.0](psf/black@24.3.0...24.4.0)

…p_flow

…rnel flow-versal

hls4ml/templates/vitis/nnet_utils/nnet_sepconv1d_stream.h

hls4ml/templates/vitis/nnet_utils/nnet_sepconv2d_stream.h

nghielme · 2025-02-22T23:20:59Z

hls4ml/backends/vitis_accelerator_ip_flow/vitis_accelerator_ip_flow_backend.py

+
+class VitisAcceleratorIPFlowBackend(VitisBackend):
+    def __init__(self):
+        super(VivadoBackend, self).__init__(name='VitisAcceleratorIPFlow')


Why calling super(VivadoBackend, self) and not super(VitisBackend, self)?

nghielme · 2025-02-22T23:23:14Z

hls4ml/backends/vitis_accelerator_ip_flow/vitis_accelerator_ip_flow_backend.py

+        # vivado_ip = 'vivado:ip'
+        writer_passes = ['make_stamp', 'vitisacceleratoripflow:write_hls']
+        self._writer_flow = register_flow('write', writer_passes, requires=['vitis:ip'], backend=self.name)
+        # self._default_flow = vivado_ip


I think the default flow should be vitis:ip, and should be set by calling super(VitisBackend, self) in the constructor of the class

nghielme · 2025-02-22T23:27:05Z

test/pytest/test_optimization/test_fifo_depth.py

I think a simple test running at least compile() (build() would be even better if possible) of a simple model would be great. This just to ensure that the backend can be used without FIFO depth optimisation

steltze and others added 30 commits April 19, 2024 10:31

Initial commit

312832f

Set change the backend

d2b5a15

Change the accelerator config script

02659dd

Set the vitis accelerator template

56296b6

Set vitis accelerator writer

7dd0173

Fix writes init

6f181b8

Include separable convolution resource implementation

bd2e52e

Separate depthwise resource strategy to 3 cases

b795240

Complete vitis accelerator wrapper for io_stream case

eeb04d4

Fix call to wrong backend writer

7e47c85

Fix vitis accelerator writer

5a2a38f

Fix include in axi wrapper header file writer

99f9429

Change python-cpp bridge writer

b9609dc

Fix tlast handling in axis wrapper writer

4f69c16

Extend convert_data to handle stream type, use that for the bridge

014a7b2

Add zcu102 to the supported boards json

723073e

Fix some c synthesis warnings

290896b

Group more tests per YAML to reduce the number of envs created

c9dfcf2

Support negative_slope in quantized_relu

d3b8e20

[pre-commit.ci] auto fixes from pre-commit hooks

b32984f

Fix activation check in profiling

98273a0

[pre-commit.ci] pre-commit autoupdate

2a71a83

updates: - [github.com/psf/black: 24.3.0 → 24.4.0](psf/black@24.3.0...24.4.0)

fix unwanted tested file change in fastmachinelearning#956

6ac964c

Fix SR backend synth missing variables

ec95e01

Test for SR backend config

5de1bf5

Upsampling support for PyTorch models

a6fec36

Split Catapult types into separate file

1b72b19

Split Quartus types into separate file

28521d0

Split Vivado types into separate file

a44707d

jmitrevs and others added 16 commits June 20, 2024 16:46

add parsing of depth multiplier for 1D depthwise conv

f12a7ea

Merge remote-tracking branch 'upstream/main' into vitis_accelerator_i…

4d24e4e

…p_flow

Finish resolving conficts with main

e2d270e

Supress removing tar for now

fa6bd66

Fix csynth and cosim

b42210d

Fix tcl script to find cosim report

1303bba

Correct PYNQ Z2 vivado tcl script, bitstream generated

8d3a1f2

Clean pynq tcl script

a8e0497

Fix compatibility of nnet helper functions with vitis axis

48686d3

Setup vivado tcl script for zcu102

bae450b

Rename backend to VitisAcceleratorIPFLow to prevent conflicts with ke…

dde9124

…rnel flow-versal

Fix compatiblity between axi stream and io parallel

663181f

Update pynq driver for zcu102

e32f4d0

Run pre-commit

c52ec75

Remove unused file

9d9e645

Remove unused xclbin generator

80697c0

nghielme reviewed Nov 21, 2024

View reviewed changes

hls4ml/templates/vitis/nnet_utils/nnet_sepconv1d_stream.h Outdated Show resolved Hide resolved

nghielme reviewed Nov 21, 2024

View reviewed changes

hls4ml/templates/vitis/nnet_utils/nnet_sepconv2d_stream.h Outdated Show resolved Hide resolved

steltze added 6 commits November 27, 2024 09:58

Clean backends init

f467829

Fix backend import sequence

4c74550

Start cleaning up code

542b950

Start integrating FIFO depth optimizer

c78aec2

Fix FIFO depth optimizer

62b5c27

Run precommit

d5f2192

steltze marked this pull request as ready for review February 20, 2025 14:25

steltze added 2 commits February 20, 2025 16:26

Merge branch 'main' into vitis_accelerator_ip_flow

34b0929

Update build_prj.tcl

14b413e

nghielme reviewed Feb 22, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Vitis Accelerator IP Flow #1134

Vitis Accelerator IP Flow #1134

steltze commented Nov 20, 2024 •

edited

Loading

nghielme Feb 22, 2025 •

edited

Loading

nghielme Feb 22, 2025

nghielme Feb 22, 2025

Vitis Accelerator IP Flow #1134

Are you sure you want to change the base?

Vitis Accelerator IP Flow #1134

Conversation

steltze commented Nov 20, 2024 • edited Loading

Type of change

Tests

Checklist

nghielme Feb 22, 2025 • edited Loading

Choose a reason for hiding this comment

nghielme Feb 22, 2025

Choose a reason for hiding this comment

nghielme Feb 22, 2025

Choose a reason for hiding this comment

steltze commented Nov 20, 2024 •

edited

Loading

nghielme Feb 22, 2025 •

edited

Loading