Skip to content

Commit 4a493c9

Browse files
authored
<docs>: Update README.md
1 parent feb88fe commit 4a493c9

File tree

1 file changed

+16
-12
lines changed

1 file changed

+16
-12
lines changed

README.md

+16-12
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,16 @@
1-
# LoHan ICDE 2025 Artifact
1+
# LoHan
22

3-
This artifact provides a guide to replicate the primary experiments in this paper. You can follow this repository to reproduce the experimental results about LoHan's maximum trainable model sizes, batch sizes and throughput in our paper. The documentation and auto-run script mainly focus on reproducing results in Subsection V-B and you can adjust the code to reproduce results in other sections.
3+
LoHan is a <ins>Lo</ins>w-cost <ins>H</ins>igh-perform<ins>an</ins>ce framework for large model fine-tuning. This repository now includes efficient data-parallel fine-tuning code (Ratel, ICDE 2025) and more exciting features are coming soon!
44

5-
## Environment Setup
5+
## Ratel ICDE 2025 Artifact
66

7-
### SSD Configuration
7+
This artifact provides a guide to replicate the primary experiments in this paper. You can follow this repository to reproduce the experimental results about Ratel's maximum trainable model sizes, batch sizes and throughput in our paper. The documentation and auto-run script mainly focus on reproducing results in Subsection V-B and you can adjust the code to reproduce results in other sections.
88

9-
LoHan aggregates the I/O bandwidth of multiple SSDs by configuring a RAID array for efficient model states and activation offloading. Therefore, we provide a script to configure this array.
9+
### Environment Setup
10+
11+
#### SSD Configuration
12+
13+
Ratel aggregates the I/O bandwidth of multiple SSDs by configuring a RAID array for efficient model states and activation offloading. Therefore, we provide a script to configure this array.
1014

1115
First, modify the `make_raid.sh` to meet your own needs. The script in this repo is used to configure the drives `/dev/nvme0n1` to `/dev/nvme11n1` into an array. You can adjust the line 23 to change the drives you want to set up.
1216

@@ -16,10 +20,10 @@ After configuring the script, you can run the script to set up the RAID array. Y
1620
./make_raid.sh
1721
```
1822

19-
### Installing the Python packages
23+
#### Installing the Python packages
2024

2125
```shell
22-
conda create -n lohan python=3.10
26+
conda create -n ratel python=3.10
2327
pip install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118
2428

2529
# If there are different CUDA versions, you should specify the CUDA version
@@ -31,15 +35,15 @@ pip install six==1.16.0
3135
pip install scikit-learn
3236
```
3337

34-
## Running LoHan
38+
### Running Ratel
3539

36-
We provide a script to run LoHan. You can adjust the script to reproduce the results.
40+
We provide a script to run Ratel. You can adjust the script to reproduce the results.
3741

3842
```shell
3943
bash run.sh
4044
```
4145

42-
### Limiting the Memory Size
46+
#### Limiting the Memory Size
4347

4448
Experiments in Subsection V-B require adjusting the main memory capacity. Instead of manually adding and removing the machine's DRAM, you can consider pinning the main memory via huge page so that these memory spaces cannot be utilized by Ratel.
4549

@@ -71,11 +75,11 @@ Hugepagesize: 2048 kB
7175
Hugetlb: 2097152 kB
7276
```
7377

74-
### Benchmark Results
78+
#### Benchmark Results
7579

7680
Please refer to [here](evaluation_data.md) for our raw evaluation data in our paper that might help for your reproduing.
7781

78-
## Acknowledgement
82+
### Acknowledgement
7983

8084
Some of the code in this project is modified from the [DeepSpeed](https://github.com/microsoft/DeepSpeed) repository, we appreciate the contributions of the original repository authors.
8185

0 commit comments

Comments
 (0)