Need cross-repo CI tests for docs pages

[garloff]

The docs pages pull in content from lots of other repositories.
This is good, as it leaves the documentation where the code is, vastly increasing the chances that docs are in sync with code changes, supporting a decentralized engineering model.

However, it comes with the downside that changes in one repo can easily break the docs pages build, making the whole build process fragile. Classic CI does not handle this well, as the PRs that break the docs are not triggering docs CI builds.

We should think about a design that avoids this fragility.

[garloff]

In my understanding, zuul has been designed to handle such cases. Correct @gtema?

[gtema]

right, it is. Very briefly: every such repo has a job building it's particular docs and invokes a general job that puts it all together. The general job is running with the specilative change under test.

[mbuechse]

Two simple ideas:

• pin versions (commits) for what is cloned
• add an explicit interface to each repo (a file that describes what is needed for the overall docs page)

[fkr]

@garloff indeed, thanks for opening this issue. I had that a few weeks ago with a different repo and forgot to open such an issue.

[garloff]

Two simple ideas:

* pin versions (commits) for what is cloned

When I look at node/ruby/python code with huge lists of pinned versions (down to minor versions and patchlevels!!), I personally consider this a huge failure in software engineering. The failure is two-fold: Code that's used by others (e.g. libraries) should be backwards compatible (and only allow breakage for major version updates). Code that uses the libraries should refrain from relying on implementation details and at most state minimum versions and possibly specific incompatibilities.
I accept pinning as exceptions, but not as the rule.
So I could accept a pin python < 3; python >= 2.6 (OK, python2 is of course not acceptable in 2025, but you get the idea), but these long lists with detailed pinned versions lead to disasters and conflicts that even with dependabot's help can often not be resolved well ... and you constantly need to look at those PRs ...

I believe a cleaner solution is making sure that breaking changes don't go unnoticed rather than allowing breakage to be merged, but then building stuff using old (pinned) working states.

[garloff]

• add an explicit interface to each repo (a file that describes what is needed for the overall docs page)

Currently, the knowledge is in the docs repo, pulling in content from other repos and including it in the build.
Changing this to have a "docs control file" in each contributing repo that describes how the content is being used and pulled in for the docs would probably be better. (The disadvantage of the current setup is that the fragility does not only have the risk of breaking things, but also the risk that content just disappears from visibility.)
The docs repo then only would need to have a list of contributing repos ...
We would need to be able to trigger docs CI builds whenever something is touched by PRs that are covered by the repo list (in docs repo) and the repo's docs control file. That's zuul magic then. (Well, zuul experts would consider this standard zuul usage.)
Or indeed go a step further (like @gtema suggested if I understood him correctly) where all repos have build processes in place that build their piece of the information for docs and when that succeeds, the chances for breakage in assembling it together becomes very low. (And even that would be found by triggering the assembly job with the speculative changes included, again zuul magic ...)

[mbuechse]

I suppose #312 is meant to address this issue (at least partially).

[garloff]

Yes, @gtema worked on this with me during the Hackathon.

The idea being:

• We have zuul do the docs builds
• Zuul would then take care of checking out all the repos that the docs depend upon (allowing to check out branches that are behind a PR) - currently the build process does it itself
• The docs building job would need to be added not just to the docs repo but to all repos that are consumed during doc building
• PRs on any of those repos would trigger the job
  • Indicating failure in case the docs build was broken by the proposed change
  • Producing a build that could^Wshould be reviewed by anyone who considers approving the PR (also confirming e.g. that docs have not vanished due to directory reorgs ...)
• Trouble is currently that docusaurus does not handle the case well where baseUrl is not /. So you can't just browse the directory or a swift bucket :-( Option to generate offline static HTML files usable without server facebook/docusaurus#3825

[garloff]

Here's a thought on working around docusaurus baseUrl issue.

1. In the build environment, set baseUrl to '/$(RANDOM_STRING)' (and possibly create a symlink with that name pointing to . in the buildDir if docusaurus consistency checks are otherwise triggered).
2. Upon publishing to a staging directory (on swift/s3), replace the $(RANDOM_STRING) with the directory name on the swift service ...
3. Upon publishing to the web server, replace the ${RANDOM_STRING) with /. That could even be done on the web server itself if needed.