Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ACI's Big Containers in SNP CI #6840

Merged
merged 18 commits into from
Feb 25, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
079e227
Print some CPU stats, so we have an early indication of what we did
eddyashton Feb 20, 2025
b8efdcb
Poke the canary
eddyashton Feb 20, 2025
53a6d06
Just blat a big request
eddyashton Feb 20, 2025
2f4c4bd
Also try a plausible region
eddyashton Feb 20, 2025
d4cbcf9
Enforce a minimum, rather than precise, attestation version (speculat…
eddyashton Feb 20, 2025
09cb919
Make core count and memory customisable through script
eddyashton Feb 20, 2025
128f977
Merge branch 'main' into try_big_containers
eddyashton Feb 20, 2025
e454223
Debugging: Verbose scp, skip CTest
eddyashton Feb 20, 2025
de2b624
Actually that might not work - run _one_ test
eddyashton Feb 20, 2025
e677df1
Merge branch 'try_big_containers' of github.com:eddyashton/CCF into t…
eddyashton Feb 20, 2025
edf0689
Merge branch 'main' of github.com:microsoft/CCF into try_big_containers
eddyashton Feb 20, 2025
c03ad7c
Merge branch 'main' into try_big_containers
eddyashton Feb 21, 2025
c2bc6ab
Comparative run - small containers with one job and logging
eddyashton Feb 21, 2025
5e54163
Merge branch 'try_big_containers' of github.com:eddyashton/CCF into t…
eddyashton Feb 21, 2025
0b98550
Well that was a waste of time. Ok, try everything again, verbosely
eddyashton Feb 21, 2025
80a5aa1
Oh duh, THIS should be the baseline test (in the old region)
eddyashton Feb 21, 2025
694ac05
Merge branch 'main' of github.com:microsoft/CCF into try_big_containers
eddyashton Feb 24, 2025
ef6efa0
Restore big-container params
eddyashton Feb 24, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion .azure-pipelines-templates/deploy_aci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,12 +61,14 @@ jobs:
python3 scripts/azure_deployment/arm_template.py deploy aci \
--subscription-id $(CCF_AZURE_SUBSCRIPTION_ID) \
--resource-group ccf-aci \
--region northeurope \
--region eastus \
--aci-type dynamic-agent \
--deployment-name ci-$(Build.BuildNumber) \
--aci-image ccfmsrc.azurecr.io/ccf/ci:pr-`git rev-parse HEAD` \
--managed-identity $(CCF_SNP_CI_MANAGED_IDENTITY_ID) \
--ports 22 \
--memory-gb 64 \
--core-count 16 \
--aci-setup-timeout 300 \
--aci-private-key-b64 $(sshKey) \
--out ~/aci_ips
Expand All @@ -80,6 +82,18 @@ jobs:
ACR_REGISTRY_RESOURCE_NAME: ccfmsrc
CCF_AZURE_SUBSCRIPTION_ID: $(CCF_AZURE_SUBSCRIPTION_ID)

- script: |
set -ex
set -o pipefail
mapfile -t IP_ADDR_LIST <<< $(cat ~/ipAddresses | awk '{print $2}')
ssh agent@${IP_ADDR_LIST[0]} -o "StrictHostKeyChecking=no" -o ConnectTimeout=100 '
nproc --all
lscpu
cat /proc/cpuinfo
'
name: print_cpu_stats
displayName: "Print CPU stats"

- script: |
set -ex
set -o pipefail
Expand Down
5 changes: 2 additions & 3 deletions .snpcc_canary
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,5 @@
O \ o | / |
/-xXx--//-----x=x--/-xXx--/---x-/--->>>--/
...
/\/\d(-_-)b/\/\/
----vmpl---

/\/\d(-_-)b/\/\
----vmpl----
20 changes: 18 additions & 2 deletions scripts/azure_deployment/arm_aci.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,15 +110,17 @@ def make_dev_container_command(args):
]


def make_dev_container(id, name, image, command, ports, with_volume):
def make_dev_container(
id, name, image, command, ports, with_volume, memory_gb, core_count
):
t = {
"name": f"{name}-{id}",
"properties": {
"image": image,
"command": command,
"ports": [{"protocol": "TCP", "port": p} for p in ports],
"environmentVariables": [],
"resources": {"requests": {"memoryInGB": 16, "cpu": 4}},
"resources": {"requests": {"memoryInGB": memory_gb, "cpu": core_count}},
},
}
if with_volume:
Expand Down Expand Up @@ -163,6 +165,18 @@ def parse_aci_args(parser: ArgumentParser) -> Namespace:
action="append",
default=[],
)
parser.add_argument(
"--memory-gb",
help="How many GB of memory to request for the container",
type=int,
default=16,
)
parser.add_argument(
"--core-count",
help="How many CPU cores to request for the container",
type=int,
default=4,
)

# SEV-SNP options
parser.add_argument(
Expand Down Expand Up @@ -245,6 +259,8 @@ def make_aci_deployment(args: Namespace) -> Deployment:
command,
args.ports,
with_volume=True,
memory_gb=args.memory_gb,
core_count=args.core_count,
)
]

Expand Down