Module Execution Config (#5741)

* Module Execution Config

* Module Execution Config

* Module Execution Config

* Remove mentioning the host-level config

* Getting back the host-level config

* update wording

* expanded the example

* lint

* adding module A/B test

@AntoxaAntoxic - can you please take a look at the new section 3.2 and make sure I got the A/B test parameter defaults correct? Thanks.

* lint

---------

Co-authored-by: Chris Huie <[email protected]>
Co-authored-by: bretg <[email protected]>

prebid-server/pbs-modules/index.md

@@ -12,7 +12,7 @@ The core of Prebid Server contains the foundational code needed for header biddi
 
 If you're looking for bidder adapter parameters, see [Bidders' Params](/dev-docs/pbs-bidders.html).
 
-* TOC
+- TOC
 {:toc}
 
 ## The Modules
@@ -54,20 +54,20 @@ mvn clean package --file extra/pom.xml
 
 The execution plan details:
 
-* which modules are used in your server
-* what order they're invoked in
-* how long modules have to run before timeout
-* whether any modules depend on each other
+- which modules are used in your server
+- what order they're invoked in
+- how long modules have to run before timeout
+- whether any modules depend on each other
 
 If you want the module to run on every request regardless of account, this is a
 host-level config you should place in `application.yaml`. If the module should
 be active only for certain accounts, you'll need to place the plan in the account-specific config.
 
 To define a plan, you'll need to know the following module details, which should be available in the module documentation:
 
-* urls: which PBS 'entry points' are relevant. e.g. /openrtb2/auction, /openrtb2/amp
-* stages: one or more of the 7 workflow stages where the module should be called: entrypoint, raw-auction-request, processed-auction-request, bidder-request, raw-bidder-response, processed-bidder-response, and/or auction-response.
-* hooks: for each stage where a module runs, its documentation will provide the hook function name.
+- urls: which PBS 'entry points' are relevant. e.g. /openrtb2/auction, /openrtb2/amp
+- stages: one or more of the 7 workflow stages where the module should be called: entrypoint, raw-auction-request, processed-auction-request, bidder-request, raw-bidder-response, processed-bidder-response, and/or auction-response.
+- hooks: for each stage where a module runs, its documentation will provide the hook function name.
 
 Here's an example application.yaml entry:
 
@@ -132,13 +132,131 @@ hooks:
   }
 ```
 
+{: .alert.alert-info :
+Execution plans can be placed in account configuration, but depending on how modules are enabled in your environment, it can be inconvenient to provide instructions to place the highly technical execution plan into the account config. Some organizations have
+chosen to keep all execution plans in host-level config, then enabling the `require-config-to-invoke` option as described in the next section.
+
 ### 3. Supply the module with configuration
 
 Modules may require configuration at startup or during the request:
 
-* If the module requires config at initialization, its documentation will
+- If the module requires config at initialization, its documentation will
 describe where the config file lives and what format it should take.
-* If the module requires runtime config, it should be passed via the account-conig mechanism.
+- If the module requires runtime config, it should be passed via the account-config mechanism.
+
+### 3.1 Module Execution Configuration
+
+PBS-Java 3.16 introduced new configurations that give the host company flexible control over which modules run for which accounts
+while still allowing all execution plans to be defined at the host-level.
+
+- `hooks.admin.module-execution` is a key-value map, where a key is a module name and a value is a boolean. It defines whether a module's hooks should be executed.
+This property can be configured on the host level at initialization as well as via account-config mechanism (a runtime config).
+- `settings.modules.require-config-to-invoke` is a host-level boolean property. When set to `true`, it requires a runtime config to exist for a module in order to actually run the execution plan.
+
+Here's how these work together:
+
+1. If `hooks.admin.module-execution` is defined at the host-level (application.yaml), it overrides all account config. No account can turn off a module flagged as true, and likewise they can’t turn on a module flagged as false.
+1. Essentially, setting false at the host level has the same effect as removing the module’s execution plan.
+1. If `hooks.admin.module-execution` is not defined at the host level, then normal precedence rules are in effect: any value in account config overrides what’s in default account config.
+1. If nothing is found for the module in the merged `hooks.admin.module-execution` and `require-config-to-invoke` is true, then account-level config is required.
+
+Example:
+
+```yaml
+# host-level config
+settings:
+  modules:
+    require-config-to-invoke: true
+hooks:
+  admin:
+    host-execution-plan: >
+      {"endpoints":{... define execution plans for module1, module3, and module4 here ...}}
+    module-execution:
+      module1: true         // don't allow accounts to turn off this module. Also don't worry about requiring config. Always run this one.
+      module2: false        // don't allow accounts to utilize this module at all, even if they define a plan in account config.
+```
+
+```json
+// account-level config
+// the end result is that module1, module3, and module5 are run.
+// module2 is not run even though a plan is defined in this account config because the host company has forbidden it above
+// module4 is not run because there's no config and require-config-to-invoke is true
+{
+  "hooks": {
+    "admin": {
+      "module-execution": {
+        "module1": false    // does nothing, since module1 is always on at the host level
+      }
+    },
+    "modules": {
+      "module3": { ... module 3 config ... },
+      "module5": { ... module 5 config ... },
+    },
+    "execution-plan": {
+       ... define an execution plan for module2 and module5 here ...
+    }
+  }
+}
+```
+
+### 3.2 A/B Testing Modules
+
+Host companies and accounts might want to try enabling a module on a small percentage of traffic before turning it on all the way.
+
+PBS-Java 3.16 introduced a new A/B testing framework that applies to any module. 
+
+```json5
+{
+    "hooks": {
+        "execution-plan": {
+           "abtests": [{
+              "accounts": [ 123, 456 ],                 // these are ignored if in account-level config
+              "module-code": "module1",
+              "enabled": true,                          // defaults to false
+              "percent-active": 5,                      // defaults to 100
+              "log-analytics-tag": true                 // defaults to true
+            },{
+               ... abtest config for other modules ...
+            }],
+            "endpoints": {
+                "/openrtb2/auction": {
+                    ...
+                 }
+            }
+        ]
+     ]
+]
+```
+
+These are the parameters accepted within the `abtests` object:
+
+{: .table .table-bordered .table-striped }
+| Parameter | Scope | Description | Type | Default |
+|-----------+-------+-------------+------+---------|
+| module-code | required | Which module is being tested. | string | none |
+| percent-active | required | What percent of the time the module will run. | integer | none |
+| accounts | optional | Defines which accounts this abtest block applies to. This is useful when the execution plan is defined at the host level and is ignored when the plan is at the account level. | array of int | none |
+| enabled | optional | Allows the abtest to be disabled without removing it. | boolean | true |
+| log-analytics-tag | optional | Directs PBS-core to log an analytics tag for reporting. | boolean | false |
+
+To get reporting on the test results, analytics adapters will need to read the [analytics tag](/prebid-server/developers/module-atags.html) created by the A/B test, which looks like this:
+
+```json5
+{
+   activities: [{
+     name: "core-module-abtests",
+     status: "success",
+     results: [{                      // one results object for each module in the abtests object
+         "status": STATUS,            // "run" or "skipped"
+         "values": {
+             "module": "module1"
+         }
+     }]
+  },
+          ... the status of other abtest decisions ...
+  }]
+}
+```
 
 ## Installing a PBS Privacy Module
 
@@ -147,6 +265,6 @@ relevant 'Activity' using the `privacyreg` directive as described in the [Activi
 
 ## Further Reading
 
-* [Developing a Prebid Server General Module](/prebid-server/developers/add-a-module.html)
-* [Developing a Prebid Server Privacy Module](/prebid-server/developers/add-a-privacy-module.html)
-* [Prebid Server Features](/prebid-server/features/pbs-feature-idx.html)
+- [Developing a Prebid Server General Module](/prebid-server/developers/add-a-module.html)
+- [Developing a Prebid Server Privacy Module](/prebid-server/developers/add-a-privacy-module.html)
+- [Prebid Server Features](/prebid-server/features/pbs-feature-idx.html)