Run ethernet/ping_interface unless hidden manifest set (e.g. lab runs) (new)

[pieqq]

Description

Use newly introduced hidden manifest (see #1699) to condition the run of the ethernet/ping_interface template jobs. This way, in the lab where devices with multiple Ethernet interfaces usually only have one of them connected, these jobs can be skipped and the generic ethernet/ping-with-any-cable-interface job can be run instead (validating that the Ethernet feature works to connect to ping the gateway). To that effect, ethernet/ping-with-any-cable-interface is added to the ethernet-automated and after-suspend-ethernet-automated test plans, alongside the ethernet/ping_interface template.

Resolved issues

https://warthogs.atlassian.net/browse/RTW-401

Documentation

The hidden manifest has a hidden-reason field which helps for documentation.

Tests

Create the following launcher (my-launcher):

[launcher]
launcher_version = 1
app_id = com.canonical.certification:RTW401
stock_reports = text

[test plan]
unit = com.canonical.certification::ethernet-automated
forced = yes

[test selection]
forced = yes

[manifest]
com.canonical.certification::_ignore_disconnected_ethernet_interfaces = true

and execute it:

$ checkbox-cli my-launcher

Observe that the ping_{interface} template jobs are skipped, and the ping-with-any-cable-interface is run:

==============[ Running job 3 / 4. Estimated time left: 0:00:08 ]===============
-----------[ Can ping another machine over Ethernet port enp0s31f6 ]------------
ID: com.canonical.certification::ethernet/ping_enp0s31f6
Category: com.canonical.plainbox::ethernet
Job cannot be started because:
 - resource expression "manifest._ignore_disconnected_ethernet_interfaces == 'False'" evaluates to false
Outcome: job cannot be started
==============[ Running job 4 / 4. Estimated time left: 0:00:04 ]===============
-----------[ Can ping the gateway with any cable Ethernet interface ]-----------
ID: com.canonical.certification::ethernet/ping-with-any-cable-interface
Category: com.canonical.plainbox::ethernet
... 8< -------------------------------------------------------------------------
Looking for all cable interfaces...
FAIL: Couldn't find any suitable cable interface.
------------------------------------------------------------------------- >8 ---

Remove the [manifest] section from the launcher and repeat the operation. This time, both ping_{interface} template jobs and ping-with-any-cable-interface are run.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 49.12%. Comparing base (bf5da28) to head (dd60642).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1709   +/-   ##
=======================================
  Coverage   49.12%   49.12%           
=======================================
  Files         372      372           
  Lines       40338    40338           
  Branches     6809     6809           
=======================================
  Hits        19816    19816           
  Misses      19799    19799           
  Partials      723      723           

Flag	Coverage Δ
provider-base	24.79% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Hook25]

I dont recall, even though I just wrote this, you can multiline this right? if so, please do

[mz2]

Since this is a recurrent source of SRU test hiccups, could you please...

• put in contribution guide a mention aimed at Canonical team members to update the lab machine manifests before merging a new test like this.
• rename the relevant repository of manifests to something that doesn't take a galaxy brain to remember it 😄 (https://github.com/canonical/ce-oem-dut-checkbox-configuration -- which is safe to mention in the public) and broadcast that. Renaming won't invalidate URLs, deploy keys etc immediately (or even very quickly I think), but they'll need to be done eventually.
• add a CI check in said repo to guard that expected manifest keys are included in all of the manifests included, so that people contributing new manifests know what to do.