Merge branch 'master' into markm-orch-type-guards

.github/actions/post-test/action.yml

@@ -30,7 +30,7 @@ runs:
         DATADOG_SITE: ${{ inputs.datadog-site }}
       continue-on-error: true
       run: |
-        ./scripts/ci-collect-testruns.sh
+        ./scripts/ci/collect-testruns.sh
         if [ "${{ inputs.datadog-token }}" != "" ]; then
           export DATADOG_API_KEY="${{ inputs.datadog-token }}"
           export DD_ENV="ci"

.github/workflows/integration.yml

@@ -229,6 +229,14 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 60
     steps:
+      - name: update job environment
+        run: |
+          if [ ":$HOME" = : ]; then
+            # Keep the first two path components, e.g. /home/runner.
+            HOME="(pwd | sed -E 's#^(/[^/]+/[^/]+).*#\1#')"
+            echo "HOME=$HOME" >> "$GITHUB_ENV"
+          fi
+          echo "SLOGFILE=$HOME/test.slog" >> "$GITHUB_ENV"
       - name: free up disk space
         run: |
           # Workaround to provide additional free space for testing.
@@ -279,30 +287,28 @@ jobs:
       - name: run proposals tests
         run: yarn test
         working-directory: a3p-integration
-      - name: collect all core eval scripts
-        # Core eval scripts will be copied under /tmp/core_eval_scripts directory
-        # colons in the parent directory name will be replaced with dashes
+      - name: copy a3p-integration
+        if: (success() || failure())
         run: |
-          find . -type d -path "./a3p-integration/proposals/*/submission" | while read -r dir; do
-            # Get the parent directory name
-            parent_dir=$(basename "$(dirname "$dir")")
-            # Replace colons with dashes in the parent directory name
-            sanitized_parent_dir=${parent_dir//:/-}
-            # Create the destination directory under /tmp/core_eval_scripts if it doesn't exist
-            destination_dir="/tmp/core_eval_scripts/$sanitized_parent_dir"
-            mkdir -p "$destination_dir"
-            # Copy everything from the $dir to the destination directory
-            cp -r "$dir"/* "$destination_dir/"
-            echo "Copied contents of $dir to $destination_dir"
-          done
-      - name: archive core eval scripts
-        # The core eval scripts can be found at the bottom of `Summary` page of
-        # `Integration tests` workflow once the workflow is completed.
-        # Ref: https://github.com/actions/upload-artifact?tab=readme-ov-file#where-does-the-upload-go
+          dir='/tmp/export/a3p-integration'
+          rm -rf "$dir"
+          scripts/ci/export-a3p.sh a3p-integration "$dir"
+      # Artifacts can be found at the bottom of `Summary` page of
+      # `Integration tests` workflow once the workflow is completed.
+      # Ref: https://github.com/actions/upload-artifact?tab=readme-ov-file#where-does-the-upload-go
+      - name: archive a3p-integration
+        if: (success() || failure())
+        uses: actions/upload-artifact@v4
+        with:
+          name: a3p-integration
+          path: /tmp/export/a3p-integration
+      - name: archive slogfile
+        if: (success() || failure())
         uses: actions/upload-artifact@v4
         with:
-          name: core-eval-scripts
-          path: /tmp/core_eval_scripts
+          name: slogfile
+          path: ${{ env.SLOGFILE }}
+          retention-days: 10
       - name: notify on failure
         if: failure() && github.event_name != 'pull_request'
         uses: ./.github/actions/notify-status

a3p-integration/README.md

@@ -153,12 +153,12 @@ make -C ../packages/deployment docker-build-sdk
 
 In a3p-integration, many core-eval proposals' `submission` content has to be
 generated from the local `agoric-sdk`, and must be rebuilt every time there is a
-change. The `scripts/build-all-submissions.sh` script contains commands to
-generate the core-eval content and move it to the expected proposal package's
-submission directory. The generation is executed as part of `a3p-integration`'s
-`build:submissions` step. Each proposal that requires such a build step should
-add an entry to the `sdk-generate` array in the `agoricProposal` section of
-`package.json`.
+change. This package's package.json `build:submissions` script runs
+`scripts/build-all-submissions.sh` to generate that content and move it into
+submission subdirectories under the corresponding [proposal
+directories](#package-layering). Each proposal that requires such a build step
+should have a corresponding entry in the `sdk-generate` array of its proposal
+directory's package.json `agoricProposal` section.
 
 Submissions that don't need to pass in options or generate references to source
 bundles can be written directly in a `foo-submission` subdirectory of the
@@ -172,24 +172,25 @@ provided, it should be written as two parts: a core eval (in one of the
 `.../proposals` directories) and a builder for it (under `.../builders/scripts`).
 
 The `build-all-submissions.sh` script reads instructions from
-`agoricProposal.sdk-generate` in `package.json`. That field contains a list of
-strings, each of which describes a single submission. If there is only one
-submission, it can use the default directory name `submission` by specifying
-only the name of the script file in `builders/scripts/vars`:
+`agoricProposal.sdk-generate` in package.json. That field contains a list of
+strings, each of which describes a single submission as a list of
+space-separated fields. If there is only one submission, it can use the default
+directory name `submission` by specifying just one field, containing the path of
+a builder script file relative to
+[packages/builders/scripts](../packages/builders/scripts):
 ```json
 "sdk-generate": ["test-localchain"],
 ```
-If there are multiple submissions, each entry has to specify the script name and
-a distinct directory name.
+If there are multiple submissions, each entry has to specify a distinct
+directory name in the second field:
 ```json
 "sdk-generate": [
   "probe-zcf-bundle probe-submission",
   "updatePriceFeeds priceFeed-submission",
   "add-auction newAuction-submission"
 ],
 ```
-A third argument can be used to provide additional parameters to the
-build script.
+Any remaining fields provide additional arguments to the build script:
 ```json
 "sdk-generate": [
   "inter-protocol/updatePriceFeeds.js submission/main main",

a3p-integration/debug-current.sh

@@ -1,8 +1,12 @@
-#!/bin/sh
+#!/bin/bash
+set -ueo pipefail
 
 # Convenience script to debug the current proposal being worked on.
 
-scripts/build-submission.sh proposals/z:acceptance testing/start-valueVow.js start-valueVow
-scripts/build-submission.sh proposals/z:acceptance testing/restart-valueVow.js restart-valueVow
+(
+  cd 'proposals/z:acceptance'
+  ../../scripts/build-submission.sh testing/start-valueVow.js start-valueVow
+  ../../scripts/build-submission.sh testing/restart-valueVow.js restart-valueVow
+)
 
 yarn test -m acceptance --debug

a3p-integration/package.json

@@ -13,7 +13,7 @@
     "doctor": "yarn synthetic-chain doctor"
   },
   "dependencies": {
-    "@agoric/synthetic-chain": "^0.3.0",
+    "@agoric/synthetic-chain": "^0.4.0",
     "@types/better-sqlite3": "^7.6.11"
   },
   "packageManager": "[email protected]",

a3p-integration/scripts/build-all-submissions.sh

@@ -1,17 +1,24 @@
 #!/bin/bash
 set -ueo pipefail
 
-# cd prints its target, so without the redirect, we get two copies
-SCRIPT_DIR=$(cd ${0%/*} > /dev/null && pwd -P)
+# Look in the "proposals" subdirectory of the working directory for
+# "$char:$name" subdirectories (cf. ../README.md#package-layering), and for each
+# one, extract from its package.json "agoricProposal" section a list of
+# "sdk-generate" entries corresponding to core-eval submission content that must
+# be generated (cf ../README.md#generating-core-eval-submissions) and then use
+# ./build-submission.sh to do so.
+
+SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &> /dev/null && pwd)
 
 IFS=$'\n'
 
 for proposal in ./proposals/?:*; do
+  echo >&2 "Building $proposal ..."
   cd $proposal
-  # build submission if proposal specifies an sdk-generate
   while read -r line; do
     IFS=' ' parts=($line)
-    $SCRIPT_DIR/build-submission.sh $proposal ${parts[@]}
+    "$SCRIPT_DIR"/build-submission.sh ${parts[@]}
   done < <(jq -r '.agoricProposal["sdk-generate"][]?' < package.json)
   cd -
+  echo >&2 "Built $proposal"
 done

a3p-integration/scripts/build-submission.sh

@@ -1,23 +1,34 @@
 #!/bin/bash
 set -ueo pipefail
 
-sdkroot=$(git rev-parse --show-toplevel)
+# Usage: $0 <builder script> [submission directory name] [builder script arg]...
+# Run the specified builder script from packages/builders/scripts and move the
+# output into the specified submission directory (defaulting to "submission")
+# relative to the working directory.
+# Must be run from inside agoric-sdk.
 
-cd "$sdkroot"
+builderScript=$1
+shift
+submissionDir=${1:-submission}
+shift || true
 
-a3pProposalDir=$1
-builderScript=$2
-submissionDirName=${3:-submission}
-submissionDir="./a3p-integration/$a3pProposalDir/$submissionDirName"
-extraParams=${4:-}
+# Run the builder script in a subshell at agoric-sdk.
+sdkroot=$(git rev-parse --show-toplevel)
+(
+  cd "$sdkroot"
+  yarn agoric run "packages/builders/scripts/$builderScript" "$@"
+)
 
-yarn agoric run "packages/builders/scripts/$builderScript" $extraParams
+# Create and populate the submission directory.
 mkdir -p "$submissionDir"
+echo >&2 "Populating $(basename -- "$(pwd -P)")/$submissionDir ..."
+ls "$sdkroot"/*-plan.json | while read plan; do
+  # Copy from the bundle cache.
+  cp $(grep -oh '/.*b1-.*.json' "$plan") "$submissionDir"
+
+  # Move from the root directory.
+  prefix=${plan%-plan.json}
+  mv "$prefix"* "$submissionDir"
 
-plans=*-plan.json
-for plan in $plans; do
-  base=${plan%-plan.json}
-  cp $(grep -oh '/.*b1-.*.json' "$base"-plan.json) "$submissionDir"
-  mv "$base"* "$submissionDir"
   ls -oS "$submissionDir"
 done

a3p-integration/yarn.lock

@@ -5,9 +5,9 @@ __metadata:
   version: 8
   cacheKey: 10c0
 
-"@agoric/synthetic-chain@npm:^0.3.0":
-  version: 0.3.0
-  resolution: "@agoric/synthetic-chain@npm:0.3.0"
+"@agoric/synthetic-chain@npm:^0.4.0":
+  version: 0.4.0
+  resolution: "@agoric/synthetic-chain@npm:0.4.0"
   dependencies:
     "@endo/zip": "npm:^1.0.7"
     better-sqlite3: "npm:^9.6.0"
@@ -16,7 +16,7 @@ __metadata:
     execa: "npm:^9.3.1"
   bin:
     synthetic-chain: dist/cli/cli.js
-  checksum: 10c0/17c6241bdc48b8a2a7608c9d4d7c0a0c76fb10d4ee44a31a1150104a792bcd1133f4b1a7e8ab26673a07450b3ceabccd9911999117568221b49221b6ee4306a1
+  checksum: 10c0/3cb31bba30f4c64ac459da39aed639a3ab0405aa1d867b4897bbad21cedd40f2142785be7f2e55b7198c58c6a31674fb534e115f7e20ebe20f4de80bfb55e7c8
   languageName: node
   linkType: hard
 
@@ -1034,7 +1034,7 @@ __metadata:
   version: 0.0.0-use.local
   resolution: "root-workspace-0b6124@workspace:."
   dependencies:
-    "@agoric/synthetic-chain": "npm:^0.3.0"
+    "@agoric/synthetic-chain": "npm:^0.4.0"
     "@types/better-sqlite3": "npm:^7.6.11"
   languageName: unknown
   linkType: soft

packages/ERTP/src/amountStore.js

@@ -1,6 +1,6 @@
 import { AmountMath } from './amountMath.js';
 
-/** @import {Amount, AssetKind, AmountValue, AssetKindForValue, AssetValueForKind, Brand, MathHelpers} from './types.js' */
+/** @import {Amount, AssetKind} from './types.js' */
 
 /**
  * @template {AssetKind} [K=AssetKind]

packages/ERTP/src/index.js

@@ -5,12 +5,5 @@ export * from './amountMath.js';
 export * from './issuerKit.js';
 export * from './typeGuards.js';
 
-/**
- * Importing Baggage from `@agoric/ertp` is deprecated. Import Baggage from
- * `@agoric/vat-data` instead
- *
- * @import {Baggage} from '@agoric/vat-data'
- */
-
 // eslint-disable-next-line import/export
 export * from './types-index.js';

packages/ERTP/src/legacy-payment-helpers.js

@@ -7,7 +7,7 @@ import { AmountMath } from './amountMath.js';
 
 /**
  * @import {ERef} from '@endo/far';
- * @import {Amount, AssetKind, AmountValue, AssetKindForValue, Payment, Brand, Purse} from './types.js';
+ * @import {Amount, AssetKind, Payment, Purse} from './types.js';
  */
 
 /**

packages/ERTP/src/purse.js

@@ -4,7 +4,7 @@ import { AmountMath } from './amountMath.js';
 import { makeTransientNotifierKit } from './transientNotifier.js';
 import { makeAmountStore } from './amountStore.js';
 
-/** @import {Amount, AssetKind, AmountValue, AssetKindForValue, RecoverySetsOption, Brand, Payment} from './types.js' */
+/** @import {AssetKind, RecoverySetsOption, Brand, Payment} from './types.js' */
 
 const EMPTY_COPY_SET = makeCopySet([]);
 

packages/ERTP/src/transientNotifier.js

@@ -6,7 +6,7 @@ import { makeNotifierKit } from '@agoric/notifier';
 
 /**
  * @import {Purse} from './types.js';
- * @import {LatestTopic, NotifierRecord} from '@agoric/notifier';
+ * @import {NotifierRecord} from '@agoric/notifier';
  */
 
 // Note: Virtual for high cardinality, but *not* durable, and so

packages/deploy-script-support/README.md

@@ -19,7 +19,10 @@ the proposal.
 
 Here's a simple example:
 
-```
+```js
+import { makeHelpers } from '@agoric/deploy-script-support';
+import { getManifestForGame1 } from '@agoric/smart-wallet/test/start-game1-proposal.js';
+
 /** @type {import('@agoric/deploy-script-support/src/externalTypes.js').CoreEvalBuilder} */
 const game1ProposalBuilder = async ({ publishRef, install }) => {
   return harden({
@@ -45,28 +48,42 @@ export default async (homeP, endowments) => {
 };
 ```
 
-The first element of `getManifestCall` is interpreted as the name of a function
-defining a behavior. The second element of `getManifestCall` produces the
-`options` argument passed to the function (`{ game1Red }` in the example
-above). A common thing to want to pass in `options` is a reference to code to be
-installed on-chain. The example above shows how. `publishRef(install(<path>))`
-is built from sources in agoric-sdk, and passed as a `bundleRef`, which contains
-a `bundleID` suitable for passing to Zoe (for contracts) or `vatAdminService`
-(for non-contract vat code).
-
-The CoreEvalBuilder says the proposal to run is
-'@agoric/smart-wallet/test/start-game1-proposal.js'. It says the manifest can be
-produced by running `getManifestForGame1`, and directs the creation of bundles
-from `@agoric/smart-wallet/test/gameAssetContract.js` which will be made
-available to the proposal as `game1Ref` in `options`.
-
-The manifest gives permissions for accessing objects in promise space, and
-passes installations to the proposalBuilder. Notice that `game1Ref` from the
-proposalBuilder is passed to `getManifestForGame1`, which adds it to
-`installations`, with the name `game1`. The name provided for installations will
-also be used to register the installation in `agoricNames`.
+This CoreEvalBuilder returns an object whose "sourceSpec" indicates that the
+proposal to run is "@agoric/smart-wallet/test/start-game1-proposal.js" and whose
+"getManifestCall" is a [functionName, ...args] array describing an invocation of
+`getManifestForGame1` exported from that file which is expected to return an
+object including a "manifest" as described below (but the actual invocation will
+insert as the first argument a "powers" object that includes functions such as
+`restoreRef`). A common thing to want to pass in `args` is a reference to code
+to be installed on-chain, and the example above shows how.
+`publishRef(install(...))` is built from sources in agoric-sdk, and passed as a
+`bundleRef`, which contains a `bundleID` suitable for passing to Zoe (for
+contracts) or `vatAdminService` (for non-contract vat code).
+
+The manifest from such an invocation is a JSON-serializable object in which each
+key is the name of a function to itself be invoked and the corresponding value
+is a "permit" describing an attenuation of the core-eval promise space to be
+provided as its first argument. A permit is either `true` or a string (_both
+meaning no attenuation of the respective subtree of the promise space, with a
+string serving as a grouping label for convenience and/or diagram generation_),
+or an object whose keys identify child properties and whose corresponding values
+are theirselves (recursive) permits. See `BootstrapManifiest` in
+[lib-boot.js](../vats/src/core/lib-boot.js).
+
+The object returned from a "getManifestCall" invocation may also include
+"installations" to register in `agoricNames`, as is done here with name "game1"
+using the reference passed in via the invocation argument:
+```js
+/** @type {import('@agoric/vats/src/core/lib-boot').BootstrapManifest} */
+const gameManifest = harden({
+  [startGameContract.name]: {
+    consume: {...},
+    brand: {...},
+    issuer: {...},
+    ...
+  },
+});
 
-```
 export const getManifestForGame1 = ({ restoreRef }, { game1Ref }) => {
   return harden({
     manifest: gameManifest,