Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add extract gpt result script #142

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

add extract gpt result script #142

wants to merge 3 commits into from

Conversation

ouyangyu
Copy link
Contributor

Filtered Result median value

case memory (MiB) lantency (ms) throuthput(sample/sec)
1n1g_dp1_mp1_pp1_mbs16_gbs16_na1_l24_hs1536_nah24_sl2048 30008 2223.93 7.19
1n1g_dp1_mp1_pp1_mbs1_gbs1_na1_l24_hs2304_nah24_sl2048 30130 286.76 3.49
1n1g_dp1_mp1_pp1_mbs2_gbs2_na1_l24_hs2304_nah24_sl2048 31080 489.99 4.08
1n1g_dp1_mp1_pp1_mbs4_gbs4_na1_l24_hs2304_nah24_sl2048 32984 896.91 4.46
1n4g_dp4_mp1_pp1_mbs1_gbs4_na1_l24_hs2304_nah24_sl2048 33870 305.59 13.09
1n4g_dp4_mp1_pp1_mbs2_gbs8_na1_l24_hs2304_nah24_sl2048 34808 510.87 15.66
1n4g_dp4_mp1_pp1_mbs4_gbs16_na1_l24_hs2304_nah24_sl2048 36724 917.67 17.44
1n8g_dp1_mp8_pp1_mbs32_gbs32_na1_l16_hs2304_nah16_sl2048 14110 977.59 32.73
1n8g_dp1_mp8_pp1_mbs32_gbs32_na1_l24_hs1536_nah24_sl2048 13214 1049.49 30.49
1n8g_dp1_mp8_pp1_mbs32_gbs32_na1_l24_hs2304_nah24_sl2048 17744 1516.38 21.10
1n8g_dp1_mp8_pp1_mbs64_gbs64_na1_l16_hs2304_nah16_sl2048 24748 1949.71 32.83
1n8g_dp1_mp8_pp1_mbs64_gbs64_na1_l24_hs1536_nah24_sl2048 23504 2063.59 31.01
1n8g_dp1_mp8_pp1_mbs64_gbs64_na1_l24_hs2304_nah24_sl2048 31270 3016.09 21.22
1n8g_dp2_mp4_pp1_mbs16_gbs32_na1_l24_hs1536_nah24_sl2048 12584 806.30 39.69
1n8g_dp2_mp4_pp1_mbs16_gbs32_na1_l24_hs2304_nah24_sl2048 16476 1174.57 27.24
1n8g_dp2_mp4_pp1_mbs32_gbs64_na1_l24_hs2304_nah24_sl2048 25402 2294.77 27.89
1n8g_dp4_mp2_pp1_mbs16_gbs64_na1_l24_hs1536_nah24_sl2048 19880 1331.34 48.07
1n8g_dp4_mp2_pp1_mbs16_gbs64_na1_l24_hs2304_nah24_sl2048 26304 1963.60 32.60
1n8g_dp8_mp1_pp1_mbs16_gbs128_na1_l24_hs1536_nah24_sl2048 32112 2263.92 56.54
1n8g_dp8_mp1_pp1_mbs1_gbs8_na1_l24_hs2304_nah24_sl2048 33870 312.95 25.56
1n8g_dp8_mp1_pp1_mbs2_gbs16_na1_l24_hs2304_nah24_sl2048 34820 518.97 30.83
1n8g_dp8_mp1_pp1_mbs4_gbs32_na1_l24_hs2304_nah24_sl2048 36730 928.78 34.45

ShawnXuan
ShawnXuan reviewed Aug 11, 2021
View reviewed changes
OneFlow/LanguageModeling/GPT/extract_gpt_result.py Outdated
from extract_util import extract_result


parser = argparse.ArgumentParser(description="flags for BERT benchmark")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BERT -> GPT

OneFlow/LanguageModeling/GPT/extract_gpt_result.py
Training...
| step | micro_batches | samples | throughput | latency | loss |
| -------- | --------------- | --------------- | ---------- | ---------- | ---------- |
| 1 | 1 | 32 | 3.65895 | 8.74569 | 11.27187 |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

every step?

OneFlow/LanguageModeling/GPT/extract_util.py Outdated
def compute_throughput(result_dict, args):
throughput = 0
latency = 0
for i in range(args.start_iter,args.end_iter):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

every step?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ShawnXuan ShawnXuan ShawnXuan left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ouyangyu @ShawnXuan