Development

Setting up the development environment

It is recommended to setup a dedicated devolpment python environment to avoid any conflicts with other packages on your computer. To do so either use virtualenv or a conda environment with Python >=3.8. With conda, this is achieved by running

conda create -n linien_dev python=3.10
conda activate linien_dev

We recommend to install packages for linting,code formatting, sorting imports, testing and checking formatting automatically by running

pip install black isort pytest pre-commit

Note that the code in gateware, i.e. the migen code should not be formatted with black: the syntax of migen is too different from standard python that applying black leads to horrible formatting. This is recorded in the pyproject.toml but it might not be honored by your IDE.

pre-commit install

from the repository's parent directory. If you don't adhere to these standards there is a bot roaming around on this repository that will complain!

Getting Linien

Linien consists of four packages, linien-common (shared between server and client), linien-client (client components for communication with the RedPitaya), linien-gui (graphical interface on the client side) and linien-server (the components that run on the RedPitaya) that all reside inside the Linien repository. Below, we describe how to install the individual packages on your computer and RedPitaya, respectively.

Branching

We use git flow as a branching model.

If you use the git flow addon: we use the standard settings except for a v as the tag prefix.

Install Git LFS and the GH CLI

We use Git LFS for storing the bitstream for the FPGA. Please make sure that it is installed prior to cloning the repository. Install it on RedPitaya OS 2.0 with

apt install git-lfs

Due to the very old Ubuntu version shipping with RedPitaya OS 1.0, Git LFS is not available there and it is thus necessary to manually copy linien.bin to the RedPitaya, see below.

For authentication, install the Github CLI following these instructions and log in with gh auth login.

On the development computer

On your computer, run

git clone https://github.com/linien-org/linien.git
cd linien
git checkout develop
pre-commit install # for automatically checking commits with isort, black and flake8

to clone the repostiory, checkout the development branch and configuring pre-commit. If you want to add a specific feature, please create a feature branch off of the develop branch (see git flow)

By default, Linien makes sure that the server- and client-side match. This check is skipped if you use a development version: to achieve this, chage the version in all four setup.py inside the linien-* directories to something that contains the string dev, e.g. 0.6.0.dev.

Next, install the three client-side packages in editable mode using pip:

cd linien-common
pip install -e .
cd ../linien-client
pip install -e .
cd ../linien-gui
pip install -e .

This allows you to modify the source code and automatically apply the chages without having the reinstall the packages every time.

If everything worked, you should be able to run the GUI by simply running

linien

Optional: Install fake server

There is the possibility to start a fake server, for example to test the GUI without a RedPitaya. Simply install the linien-server package on the computer via

cd linien-server
pip install -e .

and run the fake server with

linien-server --fake

On the RedPitaya

Login to your RedPitaya via SSH and clone the repository as above:

git clone https://github.com/linien-org/linien.git
cd linien
git checkout develop

Since currently Git LFS is not available on the RedPitaya, you have to manually copy the file linien-server/linien_server/linien.bin from the development computer to the RedPitaya in order for the server to run.

Again, change the version in the setup.py files inside linien-common and linien-server and install the packages in editable mode via

cd linien-common
pip3 install -e .
cd ../linien-server
pip3 install -e .

Note that pip3 has to be used specifically.

In addition, you need to install some other requirements by running

bash linien-server/linien_server/linien_install_requirements.sh

Once this is done, you should be able to run the server via

linien-server

on the RePitaya.

Testing

Our tests are based on pytest. Some tests have misoc as an additional requirement and we use the pytest-plt plugin. Install them via

pip install pytest-plt
pip install git+https://github.com/m-labs/misoc

To run all tests locally with your current environment, run

pytest . 

from the project root. Note that some tests are very slow! You can deselect the slow tests with the option-m "not slow"

Architecture

Linien contains three components:

• The client (linien-client): Connects to the server
• The server (linien-server): Handles connections from the client, runs long-running tasks like the autolock or the optimization algorithm. Connects to the acquisition process for communication with the FPGA.
• The acquisition process: Handles the low-level communication with the FPGA (reading / writing registers)

Furthermore, there is a graphical user interface (linien-gui) for the client and a shared library linien-common.

The communication between the components uses rpyc.

For development purposes, you can run the first two components on your local machine to simplify debugging. Only the acquisition process has to run on the RedPitaya. In a production version of linien, server and acquisition process run on the RedPitaya.

Running the code

After installation (see above), the Linien server can be started either by running linien-server or linien_start_server.sh. The latter disables ethernet blinking and runs linien-server inside a screen session. You can check whether this works by running screen -ls: there should be a session linien-server. To attach the session, use screen -r linien-server.

On the client-side, the GUI can be started by simply calling linien. The server will automatically be started upon connection. However, development versions have to be installed manually first (see above).

Run a fake server

For testing purposes, you can run a "fake" server on your computer with

linien-server --fake

after installation of the linien-server package on the computer (see above). Use localhost as the target address in the GUI.

Run server locally

For debugging it may be helpful to execute the server component on your machine (e.g. if you want to work on the autolock and want to plot the spectra). In order to make this work, you have to run python3 linien-server/linien_server/acquisition/service.py on your RedPitaya using SSH. This process provides remote access to the FPGA registers. Then, you can run the server locally and connect to the FPGA registers:

linien-server --remote-rp=root:[email protected]

Now, you can start the client.

Important: Be sure not to connect your client to the RedPitaya, but to "localhost" instead.

Building the FPGA image

For building the FPGA image, you need to install Xilinx Vivado first. Building the Linien gateware has only been tested with Vivado 2020.1 and 2020.2.

Then, call scripts/build_fpga_image.sh. In the end, the bitstream is located at linien/server/linien.bin.

Note: So far, this was tested only with Linux. It should work on Windows 10, though, when calling the script inside Windows Powershell.

Building the Linux client

The Linux version currently cannot be built with a Github action but has to be built manually, see #263.

Currently we use virtual machines based on ubuntu 20.04. It is important to install some dependencies:

sudo apt-get install qttools5-dev-tools binutils

Install linien-common, linien-client and linien-server as described above. In addition, install PyInstaller with pip install pyinstaller.

The GUI can then be built with

cd linien-gui
pyinstaller pyinstaller.spec