RODNet: Radar Object Detection Network

This is the official implementation of our RODNet papers at WACV 2021 and IEEE J-STSP 2021.

[Arxiv] [Dataset] [ROD2021 Challenge] [Presentation] [Demo]

Please cite our paper if this repository is helpful for your research:

@inproceedings{wang2021rodnet,
  author={Wang, Yizhou and Jiang, Zhongyu and Gao, Xiangyu and Hwang, Jenq-Neng and Xing, Guanbin and Liu, Hui},
  title={RODNet: Radar Object Detection Using Cross-Modal Supervision},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  month={January},
  year={2021},
  pages={504-513}
}

@article{wang2021rodnet,
  title={RODNet: A Real-Time Radar Object Detection Network Cross-Supervised by Camera-Radar Fused Object 3D Localization},
  author={Wang, Yizhou and Jiang, Zhongyu and Li, Yudong and Hwang, Jenq-Neng and Xing, Guanbin and Liu, Hui},
  journal={IEEE Journal of Selected Topics in Signal Processing},
  volume={15},
  number={4},
  pages={954--967},
  year={2021},
  publisher={IEEE}
}

Installation

Clone RODNet code.

cd $RODNET_ROOT
git clone https://github.com/yizhou-wang/RODNet.git

Create a conda environment for RODNet. Tested under Python 3.6, 3.7, 3.8.

conda create -n rodnet python=3.* -y
conda activate rodnet

Install pytorch. Note: If you are using Temporal Deformable Convolution (TDC), we only tested under pytorch<=1.4 and CUDA=10.1. Without TDC, you should be able to choose the latest versions. If you met some issues with environment, feel free to raise an issue.

conda install pytorch=1.4 torchvision cudatoolkit=10.1 -c pytorch  # if using TDC
# OR
conda install pytorch torchvision cudatoolkit=10.1 -c pytorch  # if not using TDC

Install cruw-devkit package. Please refer to cruw-devit repository for detailed instructions.

git clone https://github.com/yizhou-wang/cruw-devkit.git
cd cruw-devkit
pip install .
cd ..

Setup RODNet package.

pip install -e .

Note: If you are not using TDC, you can rename script setup_wo_tdc.py as setup.py, and run the above command. This should allow you to use the latest cuda and pytorch version.

Prepare data for RODNet

Download ROD2021 dataset. Follow this script to reorganize files as below.

data_root
  - sequences
  | - train
  | | - <SEQ_NAME>
  | | | - IMAGES_0
  | | | | - <FRAME_ID>.jpg
  | | | | - ***.jpg
  | | | - RADAR_RA_H
  | | |   - <FRAME_ID>_<CHIRP_ID>.npy
  | | |   - ***.npy
  | | - ***
  | | 
  | - test
  |   - <SEQ_NAME>
  |   | - RADAR_RA_H
  |   |   - <FRAME_ID>_<CHIRP_ID>.npy
  |   |   - ***.npy
  |   - ***
  | 
  - annotations
  | - train
  | | - <SEQ_NAME>.txt
  | | - ***.txt
  | - test
  |   - <SEQ_NAME>.txt
  |   - ***.txt
  - calib

Convert data and annotations to .pkl files.

python tools/prepare_dataset/prepare_data.py \
        --config configs/<CONFIG_FILE> \
        --data_root <DATASET_ROOT> \
        --split train,test \
        --out_data_dir data/<DATA_FOLDER_NAME>

Train models

python tools/train.py --config configs/<CONFIG_FILE> \
        --data_dir data/<DATA_FOLDER_NAME> \
        --log_dir checkpoints/

Inference

python tools/test.py --config configs/<CONFIG_FILE> \
        --data_dir data/<DATA_FOLDER_NAME> \
        --checkpoint <CHECKPOINT_PATH> \
        --res_dir results/