Architecture overview

[honzajavorek]

It would be awesome if Crawlee had an architecture overview like this https://docs.scrapy.org/en/latest/topics/architecture.html and explanations of all the concepts. As a beginner Crawlee user, I use it as a black box, but then I'm surprised by certain limitations or certain behavior. I don't know how the storage folder works, or the integration with the platform. I can read the docs about how to use it and what's supposed to happen, but otherwise, it's all just magic. I'd like to understand better how it works internally, at least in terms of a simple flow chart, which breaks it down to several black boxes.

Not sure if Docusaurus supports Mermaid, but even if not, you can use Mermaid to draft the chart here in GitHub Issues, before it makes it to the docs. See apify/apify-docs#1015 how it can be used for iterative collaboration.