feat: allow using a pattern of file to specify multiple configurations of recurring tasks

[ThomasCrambert]

Having only one file to configure recurring tasks is not optimal when handling several engines within the same codebase that are isolated from each other.

With these changes, it will be possible to define a pattern of config file paths (eg: engines/*/config/solid_queue/recurring.yml).

Note:

• Should this new configuration option be available as well from the CLI? As it felt very advanced usage, I would say it should be configurable only via an initializer.
• Should this option be in addition of recurring_schedule_file? I can see a use case where transversal tasks are defined config/recurring.yml and others more oriented toward business topics are defined in engines/*/config/solid_queue/recurring.yml

[rosa]

Hey @ThomasCrambert, thanks for this! I don't quite understand the use-case here 🤔 Could you provide more details about why do you need this? How do you handle other configurations?

[ThomasCrambert]

Hello @rosa 👋

I don't quite understand the use-case here 🤔Could you provide more details about why do you need this?

Sure thing!

I've thought about this feature when I introduced solid_queue in our monolith application. There are more 50+ teams collaborating on it, and I don't think it makes much sense to have all of them defining business oriented recurring tasks within a single config/recurring.yml. As we have CODEOWNERS on every file of our codebase, which team should be in charge of validating the introduction of new recurring tasks? Does it even make sense to have one?

This is with that in mind that I thought about splitting this single configuration file, one where recurring tasks could be defined closer to the business needs (one configuration per engine is what I had in mind, another approach could be to have several configurations within the config folder). This way, you are sure that the right people are involved in the review process.

Does it make more sense to you?

How do you handle other configurations?

Which configurations are you referring to there? The ones for workers/dispatchers/schedulers?

[rosa]

Which configurations are you referring to there? The ones for workers/dispatchers/schedulers?

Yes, but also any other Rails configuration, because I didn't understand very well what you meant, so I thought your problem applied to any other config, like storage, DBs, etc 😅