Pants-internal tests that use pytest.mark.platform_specific_behaviour aren't required to have corresponding target tag

[huonw]

Describe the bug

We should have some sort of linting (or other tooling) that makes it impossible to define "platform-specific behaviour" (PSB) tests incorrectly (forgetting to mark the test or forgetting to tag the target).

Pants has a bunch of tests marked as "platform-specific behaviour" (PSB), defining a high-value subset of tests for running on new platforms. (i.e. the cheap/free Linux x86-64 runners run all tests, and our other runners only run the PSB ones.)

Marking a test like this requires two steps, to coordinate with the ./pants --tag=+platform_specific_behavior test :: -- -m platform_specific_behavior invocation used to run them:

1. apply pytest.mark.platform_specific_behavior to the test (within the ..._test.py file) so Pytest can select the test when running that file. Example:

   pants/src/python/pants/backend/python/lint/black/rules_integration_test.py

   Lines 107 to 112 in 6f8e16e

   	@pytest.mark.platform_specific_behavior
   	@pytest.mark.parametrize(
   	"major_minor_interpreter",
   	all_major_minor_python_versions(Black.default_interpreter_constraints),
   	)
   	def test_passing(rule_runner: PythonRuleRunner, major_minor_interpreter: str) -> None:

2. add platform_specific_behavior tag to the test's target (within a BUILD file) so Pants can select the files to run. Example:

   pants/src/python/pants/backend/python/lint/black/BUILD

   Lines 10 to 18 in 6f8e16e

   	python_tests(
   	name="tests",
   	overrides={
   	"rules_integration_test.py": {
   	"tags": ["platform_specific_behavior"],
   	"timeout": 240,
   	}
   	},
   	)

It's easy to do step 1 without step 2.

This shell pipeline estimates these mistakes, by finding any *_test.py that mention platform_specific_behavior where the adjacent BUILD does not (and vice versa):

comm -3 \
  <(rg --files-with-matches 'platform_specific_behavior' --glob '*_test.py' . | sed -E 's|(.*)/[^/]+|\1|' | sort | uniq) \
  <(rg --files-with-matches 'platform_specific_behavior' --glob 'BUILD' . | sed -E 's|(.*)/BUILD|\1|' | sort | uniq)

(NB. this isn't perfect, would be better to examine pants peek output, e.g. if there's two *_test.py files that have the tag in a directory, the BUILD file should have both tests tagged with it.)

At the time of writing these tests are missing the BUILD file tag (fixed in #21609):

• ./src/python/pants/backend/build_files/fmt/black/integration_test.py
• ./src/python/pants/backend/build_files/fmt/ruff/integration_test.py
• ./src/python/pants/backend/build_files/fmt/yapf/integration_test.py
• ./src/python/pants/backend/sql/lint/sqlfluff/rules_integration_test.py

Pants version
main / 6f8e16e

OS
N/A
Additional info
N/A