[proposal] How to Better Integrate Mid-Tier Runtime Hooks with Batch Resources?

[tan90github]

What is your proposal:

In light of the proposal #1762, we have been working on the implementation of the mid-tier resource runtimeHook recently. During development, we've encountered potential redundancies in the runtimeHook implementations between mid-tier and batch resources, with some details challenging to separate distinctly. We are seeking insights from the community regarding how compatibility between mid-tier and batch was initially addressed during the development of batch resources.

Why is this needed:

With the introduction of new features such as CPU normalization in batch resources, the integration of mid-tier resources may become increasingly complex. We are interested in any advice the community might have regarding the development of mid-tier and batch resources in terms of cgroup implementation.

Is there a suggested solution, if so, please add it:

We are considering several development approaches and would appreciate any recommendations from the community:

Plan 1: Mirroring the previous PR #1984, we aim to leverage the existing batchResource code extensively, including hooks, reconcilers, and rules. This approach might lead to a unified extended resource model, where feature gating cannot differentiate between batch and mid-tier resources, potentially causing issues with shared rules for mid-tier and batch, especially with the integration of mid-tier and batch resources.

Plan 2: Copy all code from the batchresource directory and rename 'batch' to 'mid'. This allows differentiation but results in high code redundancy.

Plan 3: Refactor the batchresource code to extract fundamental functions for use by both mid-tier and batch resources, which introduces greater risk.

Other potential plans that we may not have considered yet...

[tan90github]

cc @zwzhang0107 @j4ckstraw @yangfeiyu20102011

[stale]

This issue has been automatically marked as stale because it has not had recent activity.
This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, the issue is closed
  You can:
• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close
  Thank you for your contributions.

[tan90github]

We are expect to gradually implement the mid-level runtimeHook plugin according plan 3 as follows.

1. Support priority + qos runtimeHook reconciler filter, add priorityQosClass API.
   api: support priority+qos apis. #2280
2. Add mutating/validating webhooks to support mid resource.
3. Replace podQoSFilter with priorityQoSFilter in runtimeHook plugin (cpuset, cpunormalization and so on) .
4. Refactor the batch/mid runtimehook plugin, first support mid + BE, then support mid + LS.

cc @j4ckstraw @yangfeiyu20102011