Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for exporting canvases and attachments #225

Closed
samdev11 opened this issue Jun 22, 2023 · 12 comments · Fixed by #401
Closed

Add support for exporting canvases and attachments #225

samdev11 opened this issue Jun 22, 2023 · 12 comments · Fixed by #401
Labels
API API related issue EDGE enhancement New feature or request
Milestone

Comments

@samdev11
Copy link

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Slack recently launched a new feature, Canvases. It looks like these are not currently being exported by slackdump.

Describe the solution you'd like
A clear and concise description of what you want to happen.

It would be great to add support to Slackdump for Canvases, so that the content and attachments can be exported.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

@rusq
Copy link
Owner

rusq commented Jun 22, 2023

Hey @samdev11 , currently there's no publicly available APIs to scrape canvases for the account, so it would not be possible at this point. My test workspaces don't have this feature, so I can't even test this. Most likely they are using the internal edge apis, implementing which is currently on on the radar.

I'll let this issue dangle until the smoke clears around those canvases.

@rusq rusq added EDGE API API related issue enhancement New feature or request labels Jun 22, 2023
@softyoda
Copy link

Hi, is the canvas API still no publicly available ? We wish to use slack new canvas but also perpetuate our data easily.

@rusq
Copy link
Owner

rusq commented Sep 20, 2023

Hey @softyoda , it doesn't look like it, there's nothing in this list: https://api.slack.com/methods

@kevinb9n
Copy link

kevinb9n commented Aug 15, 2024

I think canvases are available as a subset of files: https://api.slack.com/methods/files.info ?

The nice piece would be if the canvas could be exported as ordinary markdown instead of html or whatever.

@CodeBradley
Copy link

CodeBradley commented Jan 8, 2025

Any traction on this as of 2025 or workarounds to getting Canvases in an automated fashion?

I think canvases are available as a subset of files: https://api.slack.com/methods/files.info ?

The nice piece would be if the canvas could be exported as ordinary markdown instead of html or whatever.

This definitely has the links to the files, looks like curl could grab it easily via file.permalink, file.permalink_public with file.is_public = true && file.file_deleted = false and/or file.url_private, file.url_private_download.

{
    "ok": true,
    "file": {
        "id": "F0S43PZDF",
        "created": 1531763342,
        "timestamp": 1531763342,
        "name": "tedair.gif",
        "title": "tedair.gif",
        ...
        "is_public": true,
        ...
        "url_private": "https://.../tedair.gif",
        "url_private_download": "https://.../tedair.gif",
        ...
        "permalink": "https://.../tedair.gif",
        "permalink_public": "https://.../...",
        ...
     },
     ...
}

@rusq rusq added this to the v3.1 milestone Jan 8, 2025
@rusq
Copy link
Owner

rusq commented Jan 8, 2025

Hey @CodeBradley, yes, everything in the issues should eventually make it, chucked this into v3.1 milestone. Do you have an example of a canvas file structure? On my free workspace I can only have channel scope canvases.

@rusq rusq modified the milestones: v3.1, v3.0.3 Jan 10, 2025
@rusq
Copy link
Owner

rusq commented Jan 10, 2025

So, good news - canvases attached to messages are being downloaded as files already.

The only thing that remains - adding support for channel canvases.

@CodeBradley
Copy link

So, good news - canvases attached to messages are being downloaded as files already.

The only thing that remains - adding support for channel canvases.

This is awesome, thank you for the update. Question, if a download cuts out during a full Slack download, will it resume from where it left off if it restart again? I plan to try it but not sure if there's a setting or something that might exist for this.

@rusq
Copy link
Owner

rusq commented Jan 10, 2025

No problem :)

Re file downloads: I just made this change yesterday: #400, which retries on EOF errors (during file downloads it seems to be a common error), and also allows to redownload any missing files in "archive" type downloads.

There's no "resume" functionality, and if you run slackdump against an existing archive or directory - it will simply overwrite everything.

P.S> you can always convert "archive" format to a native slack export with "slackdump convert" command.

@CodeBradley
Copy link

Ok cool, just restarted it. Potential feature idea for large exports: Haven't looked into complexity; but it might make sense to attach an MD5 or SHA1 hash of each file/convo/object and if a backup has to restart and the hash changes then redownload it, otherwise if the hash matches then those files could be skipped.

Awesome repo overall.

@rusq
Copy link
Owner

rusq commented Jan 10, 2025

That shouldn't be hard to add, and I like the idea, but there are two things to consider: (1) in order to calculate the hash for comparison, the file needs to be downloaded anyway (2) If the hash mismatches, how to reliably tell which version is correct? Possibly the file with larger size would be "more correct" in most cases, the only proper way would be if the impossible would happen and Slack would expose the hash on the File object )

Right now, the "tool redownload" checks if the file exists and it's size > 0, otherwise it attempts the download again.

@CodeBradley
Copy link

That shouldn't be hard to add, and I like the idea, but there are two things to consider: (1) in order to calculate the hash for comparison, the file needs to be downloaded anyway (2) If the hash mismatches, how to reliably tell which version is correct? Possibly the file with larger size would be "more correct" in most cases, the only proper way would be if the impossible would happen and Slack would expose the hash on the File object )

Right now, the "tool redownload" checks if the file exists and it's size > 0, otherwise it attempts the download again.

Ahh, touche. Valid point, I think that's the best way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API API related issue EDGE enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants