Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: child modules telemetry check #4370

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

ReneHezser
Copy link
Contributor

Description

Child modules should be references with telemetry disabled.

Pipeline Reference

Pipeline

Type of Change

  • Update to CI Environment or utilities (Non-module affecting changes)
  • Azure Verified Module updates:
    • Bugfix containing backwards-compatible bug fixes, and I have NOT bumped the MAJOR or MINOR version in version.json:
      • Someone has opened a bug report issue, and I have included "Closes #{bug_report_issue_number}" in the PR description.
      • The bug was found by the module author, and no one has opened an issue to report it yet.
    • Feature update backwards compatible feature updates, and I have bumped the MINOR version in version.json.
    • Breaking changes and I have bumped the MAJOR version in version.json.
    • Update to documentation

Checklist

  • I'm sure there are no other open Pull Requests for the same update/change
  • I have run Set-AVMModule locally to generate the supporting module files.
  • My corresponding pipelines / checks run clean and green without any errors or warnings

@microsoft-github-policy-service microsoft-github-policy-service bot added Needs: Triage 🔍 Maintainers need to triage still Type: AVM 🅰️ ✌️ Ⓜ️ This is an AVM related issue labels Feb 3, 2025
# We only care about module templates
(Split-Path $involvedFilePath -Leaf) -eq 'main.bicep' -and
# only return child modules
(Test-Path -Path (Join-Path (Split-Path $involvedFilePath) 'version.json') -PathType Leaf)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either I recommend to rename the variable or remove this line, as a child module does not need a version.json file (as per its current definition).

return
}

$modulesWithChildReferences | ForEach-Object {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test should be written in a way where you collect all incorrect cases first, and then fail for that list not being empty. Otherwise, this test will fail on the very first incorrect case, and if you have more than one, you won't know until you fix the first, run the test again, and so on and so forth.
You can see this approach being implemented for other tests like

$incorrectParameters | Should -BeNullOrEmpty -Because ('parameters in the template file should be camel-cased. Found incorrect items: [{0}].' -f ($incorrectParameters -join ', '))

@@ -992,6 +1001,52 @@ Describe 'Module tests' -Tag 'Module' {
$telemetryDeploymentName = $telemetryDeployment.name # The AVM telemetry prefix
$telemetryDeploymentName | Should -Match "$expectedTelemetryIdentifier"
}

It '[<moduleFolderName>] Telemetry should be disabled for child modules.' -TestCases ($moduleFolderTestCases | Where-Object { $_.isTopLevelModule }) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would recommend to simplify this to

It '[<moduleFolderName>] Telemetry should be disabled for referenced modules with dedicated telemetry.' -TestCases $moduleFolderTestCases {

    param(
        [hashtable] $templateFileContent
    )

    $referencesWithTelemetry = @()
    $templateFileContent.resources.Keys | ForEach-Object {
        $elem = $templateFileContent.resources[$_]

        if ($elem.type -eq 'Microsoft.Resources/deployments' -and $elem.properties.template.parameters.Keys -contains 'enableTelemetry') {
            $elem['bicepRef'] = $_
            $referencesWithTelemetry += $elem
        }
    }

    if ($referencesWithTelemetry.Count -eq 0) {
        Set-ItResult -Skipped -Because 'no modules with dedicated telemetry are deployed.'
        return
    }

    # Analyze
    $incorrectReferences = @()
    foreach ($reference in $referencesWithTelemetry) {
        if ($reference.properties.parameters.Keys -notContains 'enableTelemetry' -or $reference.properties.parameters.enableTelemetry.value -ne "[variables('enableReferencedModuleTelemetry')]") {
            $incorrectReferences += $reference.bicepRef
        }
    }

    $incorrectReferences | Should -BeNullOrEmpty -Because ('module deployments that implement their own telemetry should pass `enableTelemetry: enableReferencedModuleTelemetry` to the referenced module. Found incorrect items: [{0}].' -f ($incorrectReferences -join ', '))
}

It does not rely on any function analyzing the repo and is purely using the compiled Bicep template to decide which references are relevant for testing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Result would look like

[-] [key-vault/vault] Telemetry should be disabled for referenced modules with dedicated telemetry. 11ms (10ms|1ms)
     Expected $null or empty, because module deployments that implement their own telemetry should pass `enableTelemetry: enableReferencedModuleTelemetry` to the referenced module. Found incorrect items: [keyVault_secrets, keyVault_privateEndpoints]., but got @('keyVault_secrets', 'keyVault_privateEndpoints').
     at $incorrectReferences | Should -BeNullOrEmpty -Because ('module deployments that implement their own telemetry should pass `enableTelemetry: enableReferencedModuleTelemetry` to the referenced module. Found incorrect items: [{0}].' -f ($incorrectReferences -join ', ')), C:\dev\ip\bicep-registry-modules\Upstream-Azure\utilities\pipelines\staticValidation\compliance\module.tests.ps1:620
     at <ScriptBlock>, C:\dev\ip\bicep-registry-modules\Upstream-Azure\utilities\pipelines\staticValidation\compliance\module.tests.ps1:620

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend also adding the following test for variables

It '[<moduleFolderName>] Variable `var enableReferencedModulesTelemetry = false` should exist if module references other modules with dedicated telemetry.' -TestCases $moduleFolderTestCases {

    param(
        [hashtable] $templateFileContent
    )

    $referencesWithTelemetry = $templateFileContent.resources.Values | Where-Object {
        $_.type -eq 'Microsoft.Resources/deployments' -and
        $_.properties.template.parameters.Keys -contains 'enableTelemetry'
    }

    if ($referencesWithTelemetry.Count -eq 0) {
        Set-ItResult -Skipped -Because 'no modules with dedicated telemetry are deployed.'
        return
    }

    $templateFileContent.variables.Keys | Should -Contain 'enableReferencedModulesTelemetry'
    $templateFileContent.variables.enableReferencedModulesTelemetry | Should -Be $false
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Result would look like

[-] [key-vault/vault] Variable 
ar enableReferencedModulesTelemetry = false should exist if module references other modules with dedicated telemetry. 26ms (17ms|8ms)
     Expected 'enableReferencedModulesTelemetry' to be found in collection @('copy', 'enableReferencedModuleTelemetry', 'builtInRoleNames'), but it was not found.
     at $templateFileContent.variables.Keys | Should -Contain 'enableReferencedModulesTelemetry', C:\dev\ip\bicep-registry-modules\Upstream-Azure\utilities\pipelines\staticValidation\compliance\module.tests.ps1:586
     at <ScriptBlock>, C:\dev\ip\bicep-registry-modules\Upstream-Azure\utilities\pipelines\staticValidation\compliance\module.tests.ps1:586
    [-] [key-vault/vault] Telemetry should be disabled for referenced modules with dedicated telemetry. 11ms (10ms|1ms)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs: Triage 🔍 Maintainers need to triage still Type: AVM 🅰️ ✌️ Ⓜ️ This is an AVM related issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants