Skip to content

Commit

Permalink
Update GHA workflow for chart sync via issues (#16)
Browse files Browse the repository at this point in the history
* Update GHA workflow for chart sync via issues

* Update README

* Update README
  • Loading branch information
justinyoo authored Oct 23, 2024
1 parent 5635a84 commit 4649c55
Show file tree
Hide file tree
Showing 3 changed files with 74 additions and 8 deletions.
68 changes: 60 additions & 8 deletions .github/workflows/scrape.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ on:
workflow_dispatch:
schedule:
- cron: "30 20 * * *"
issues:
types:
- opened

jobs:
scrape:
Expand All @@ -14,36 +17,58 @@ jobs:
issues: write

steps:
- name: Check to proceed
id: check_to_proceed
shell: pwsh
run: |
$proceed = "false"
$chartSyncRequest = "false"
if ("${{ github.event_name }}" -eq "workflow_dispatch") {
$proceed = "true"
} elseif ("${{ github.event_name }}" -eq "schedule") {
$proceed = "true"
} elseif ("${{ github.event_name }}" -eq "issues" -and "${{ github.event.action }}" -eq "opened") {
if (${{ toJson(github.event.issue.body) }} -eq "### Issue Type\n\nSync Request") {
$proceed = "true"
$chartSyncRequest = "true"
}
}
"proceed=$proceed" >> $env:GITHUB_OUTPUT
"chartSyncRequest=$chartSyncRequest" >> $env:GITHUB_OUTPUT
- name: Checkout code
if: steps.check_to_proceed.outputs.proceed == 'true'
uses: actions/checkout@v4

- name: Run scraper
run: |
# Add your scraping command here
- name: Setup .NET
if: steps.check_to_proceed.outputs.proceed == 'true'
uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.x'

- name: Restore NuGet packages
if: steps.check_to_proceed.outputs.proceed == 'true'
shell: bash
run: |
dotnet restore
- name: Build solution
if: steps.check_to_proceed.outputs.proceed == 'true'
shell: bash
run: |
dotnet build -c Release
- name: Install Playwright
if: steps.check_to_proceed.outputs.proceed == 'true'
shell: pwsh
run: |
$playwright = Get-ChildItem -File Microsoft.Playwright.dll -Path . -Recurse
$installer = "$($playwright[0].Directory.FullName)/playwright.ps1"
& "$installer" install
- name: Run Melon app - Melon Top 100
if: steps.check_to_proceed.outputs.proceed == 'true'
shell: pwsh
run: |
$date = (Get-Date).ToUniversalTime().AddHours(9).ToString("yyyyMMdd")
Expand All @@ -57,6 +82,7 @@ jobs:
popd
- name: Run Spotify app - Melon Top 100
if: steps.check_to_proceed.outputs.proceed == 'true'
shell: pwsh
run: |
$appsettings = Get-Content -Path ./samples/SpotifyPlaylist.ConsoleApp/appsettings.Development.sample.json | ConvertFrom-Json
Expand All @@ -75,6 +101,7 @@ jobs:
popd
- name: Run Spotify app - Billboard Hot 100
if: steps.check_to_proceed.outputs.proceed == 'true'
shell: pwsh
run: |
$appsettings = Get-Content -Path ./samples/SpotifyPlaylist.ConsoleApp/appsettings.Development.sample.json | ConvertFrom-Json
Expand All @@ -93,6 +120,7 @@ jobs:
popd
- name: Upload data
if: steps.check_to_proceed.outputs.proceed == 'true'
uses: stefanzweifel/git-auto-commit-action@v5
with:
commit_message: "Update data"
Expand All @@ -101,15 +129,39 @@ jobs:
commit_user_email: "scraper+github-actions[bot]@users.noreply.github.com"
commit_author: "GitHub Actions <scraper+github-actions[bot]@users.noreply.github.com>"

- name: Check missing tracks
id: missing_tracks
if: steps.check_to_proceed.outputs.proceed == 'true'
shell: pwsh
run: |
$date = (Get-Date).ToUniversalTime().AddHours(9).ToString("yyyyMMdd")
$collection = Get-Content "./data/spotify100-$date.json" | ConvertFrom-Json
$missingTracks = ($collection.missingTracks.Count -gt 0).ToString().ToLowerInvariant()
"exist=$missingTracks" >> $env:GITHUB_OUTPUT
- name: Issue missing tracks
if: |
steps.check_to_proceed.outputs.proceed == 'true' &&
steps.missing_tracks.outputs.exist == 'true'
shell: pwsh
env:
GH_TOKEN: ${{ github.token }}
run: |
$date = (Get-Date).ToUniversalTime().AddHours(9).ToString("yyyyMMdd")
$collection = Get-Content "./data/spotify100-$date.json" | ConvertFrom-Json
if ($collection.missingTracks.Count -gt 0) {
$tracks = $collection.missingTracks | ConvertTo-Json -Depth 100
gh issue create --title "Missing tracks in Spotify playlist" --body "The following tracks are missing in the Spotify playlist:`r`n```````r`n$tracks`r`n```````r`n"
}
$tracks = $collection.missingTracks | ConvertTo-Json -Depth 100
gh issue create --title "Missing tracks in Spotify playlist" --body "The following tracks are missing in the Spotify playlist:`r`n```````jsonr`n$tracks`r`n```````r`n"
- name: Close sync request
if: |
steps.check_to_proceed.outputs.proceed == 'true' &&
steps.check_to_proceed.outputs.chartSyncRequest == 'true'
shell: pwsh
env:
GH_TOKEN: ${{ github.token }}
run: |
$issueNumber = ${{ github.event.issue.number }}
gh issue close $issueNumber -r completed -c "The chart has been synced up-to-date."
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,3 +85,17 @@ This is the Melon chart scraping library written in .NET - Top 100, Hot 100, Dai
return Results.Json(collection.Items);
});
```

## Samples

There are some samples in the `samples` directory.

### MelonChart on Spotify

There's a [playlist on Spotify](https://open.spotify.com/playlist/6zRtKdszUiBnJm4lIbC6h9?si=kCR3OHPVTHmI3jHEY1Qyrg) using this library. If you want to sync the Melon chart to Spotify up-to-date, create an [issue](../../issues), and it will automatically update the playlist.
![Melon Chart on Spotify](./assets/melon-chart-on-spotify.png)
## Issues?
If you find any issues, please [report them](../../issues).
Binary file added assets/melon-chart-on-spotify.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 4649c55

Please sign in to comment.