Data Collection Strategies for Mahjong Soul AI Training

[adsf0427]

Title: Data Collection Strategies for Mahjong Soul AI Training

Hello @Cryolite and the community,

I've been exploring this excellent repository for Mahjong Soul AI training and have some questions about data collection strategies. I'm hoping to start a discussion on this topic and get insights from those with experience.

Challenges

1. UUID Collection: Obtaining UUIDs for past game logs is proving difficult, limiting access to historical data.
2. Server Load: Concerns about potentially burdening Mahjong Soul's infrastructure when collecting large amounts of data.

Questions

1. What are some efficient methods for collecting data without significantly impacting Mahjong Soul's servers?
2. Are there any publicly available datasets suitable for training Mahjong AI?
3. Can anyone share best practices or effective techniques for data collection in this context?
4. @Cryolite , would it be possible to share a small subset of collected data for study purposes? This would be invaluable for understanding the structure of high-quality training data. If shared, it would be used strictly for personal learning and research, with full respect for confidentiality.

Community Input

I'd love to hear from others who have tackled these challenges. How have you approached data collection for Mahjong AI training?

Thank you all in advance for your insights, and special thanks to @Cryolite for creating this valuable resource for the Mahjong AI community.

[Cryolite]

Answers

1. Downloading game records directly from the Mahjong Soul server inevitably puts a load on the server. Mahjong Soul game records are approximately 50KB to 100KB per half-length game. For example, downloading game records on the order of 10 million games, which is necessary to train a model with modest strength through behavioral cloning, would total greater than 1GB of data. Having each individual directly download this order of data from the Mahjong Soul server would place a significant load on the server.

2. As far as I know, there are currently no publicly available datasets of that kind.

3. I'm currently developing a new version, v2. The simulator that comes with the new version is planned to include a feature to record detailed simulation progress in the form of training data. Using this, it will be possible to newly generate reasonably high-quality game records through simulations of matches between reasonably strong models without downloading any data from the Mahjong Soul server.

4. Previously, I consulted with a lawyer, and at least under Japanese law and the terms of service of the Japanese Mahjong Soul service, providing game records obtained with my own Mahjong Soul account to third parties may violate the terms of service. Therefore, regrettably, I do not intend to share the datasets I have. However, I believe that the simulation results of matches between trained models, as answered in 3., are not subject to the Mahjong Soul terms of service. I plan to make the training data generated through such simulations available on suitable platforms like Hugging Face.

[adsf0427]

Thank you, for your detailed and insightful response. Your guidance on server load, the absence of public datasets, and the promising development of v2's simulator feature are incredibly valuable. I understand the legal constraints and appreciate your commitment to sharing simulation-generated training data on platforms like Hugging Face.

Looking forward to the new developments and any further guidance you can provide.

[adsf0427]

Hello @Cryolite and the community,

Thank you for your insights.

Would it be possible to share the UUIDs of the Mahjong Soul games you have collected? I am asking for the UUIDs only, as they do not contain any game information. I have had some difficulty collecting them myself. I assure you that I have no intent to break the terms of service and will use them responsibly.

Thank you for considering my request.

[GrandDuke1106]

If you just need the uuid, check out this site
https://amae-koromo.sapk.ch

[adsf0427]

Thank you for your detailed response, @Cryolite.
I have some additional information that might be relevant to this discussion:
Recently, I've been working on another Mahjong AI project based on Mortal. I've trained a decent model and collected a substantial amount of self-play game data in the mjai format. I'm willing to share this data as a potential bootstrap for training Kanachan.
However, I'm facing a challenge in converting the mjai game data into the format that Kanachan requires (Mahjong Soul format). Could you provide some guidance on this conversion process? Having a way to convert between these formats could be beneficial for the community, allowing us to leverage data from various sources.
This approach might help address some of the data collection challenges we've discusse. It could potentially reduce the need for direct downloads from Mahjong Soul servers while still providing valuable training data.
I'm curious about your thoughts on this approach and whether you think it could be a viable strategy for data collection and cross-project collaboration in Mahjong AI development.

[Cryolite]

I am very interested to hear that you are working on a Mahjong AI project based on Mortal. I also appreciate your offer to provide self-play data in the mjai format. This could potentially enhance the training datasets not only for Kanachan but also for many other Mahjong AI projects.

I believe it makes a lot of sense to adopt the mjai format as the replay format for various Mahjong AI projects. Currently, the mjai format has become the de facto standard replay format used by many Mahjong AI projects, such as Mortal and RiichiLab. While Kanachan does not plan to directly support the mjai format, I am seriously considering implementing a conversion between the replay format Kanachan expects for training and the mjai format. This would allow, for example, indirect compatibility with the Tenhou replay format. In a way, the mjai format could function as a "hub format" for various replay formats, similar to how Unicode serves as a central standard for different character encodings.

With that said, I am planning to start implementing the conversion between the mjai format and the replay format expected by Kanachan for training. However, there is one issue. The mjai format specification is somewhat vague. In fact, the original specification page for the mjai format does not comprehensively cover all cases, and many Mahjong AI projects have added their own extensions to the mjai format. Therefore, I would like to ask everyone to share examples of mjai format replays used in various Mahjong AI projects or conversions from other formats, such as Tenhou or Mahjong Ichibangai replay formats, to mjai format using existing conversion tools.

On my end, I will also reach out to the creators of Mortal, the RiichiLab administrators, and the Haifuya administrators to encourage the establishment of a collaborative project to standardize the mjai format specification.

[adsf0427]

Thank you for your detailed message regarding the mjai format and its potential as a standardized format for Mahjong AI projects. I appreciate your insights on its growing adoption and the benefits it could bring to the community.
I'm glad to hear that you're considering implementing a conversion between Kanachan's expected training format and the mjai format.
Regarding your request for examples of mjai format replays, I'm happy to help. I can provide you with some mjai logs from Mortal's self-play sessions, along with corresponding Tenhou format logs that I've exported myself. These logs are compatible with the mortal reviewer. I believe these could be useful for your conversion implementation.

I fully support your initiative to reach out to other project creators and administrators to standardize the mjai format specification. This collaborative effort could greatly benefit the entire Mahjong AI community by establishing a more comprehensive and widely accepted standard.
If you need any additional information or have any questions about the logs I'm sharing, please don't hesitate to ask. I'm looking forward to seeing the progress on this standardization effort and how it might improve interoperability between different Mahjong AI projects.
tenhou_mjai.tar.gz