Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Workflow failure: Run Example Code Tests #13885

Open
pulumi-bot opened this issue Jan 27, 2025 · 5 comments
Open

Workflow failure: Run Example Code Tests #13885

pulumi-bot opened this issue Jan 27, 2025 · 5 comments
Assignees
@thoward @toriancrane @dirien
Labels
area/blog Content issues on blog posts. area/docs Improvements or additions to documentation impact/flaky-test A test that is unreliable impact/regression Something that used to work, but is now broken impact/usability Something that impacts users' ability to use the product easily and intuitively kind/bug Some behavior is incorrect or out of spec kind/engineering Work that is not visible to an external user p1 A bug severe enough to be the next item assigned to an engineer

Comments

@pulumi-bot
Copy link
Collaborator

pulumi-bot commented Jan 27, 2025 •
edited
Loading

Workflow Failure

Run Example Code Tests has failed. See the list of failures below:
@pulumi-bot pulumi-bot added kind/engineering Work that is not visible to an external user needs-triage Needs attention from the triage team p1 A bug severe enough to be the next item assigned to an engineer labels Jan 27, 2025
@github-project-automation github-project-automation bot moved this to 🤔 Triage in Docs 📚 Jan 27, 2025
@thoward
Copy link
Contributor

thoward commented Jan 29, 2025

There are 38 failing example programs as of the most recent run:

The following projects failed:
- aws-acm-certificate-csharp
- aws-iampolicy-jsonparse-csharp
- aws-s3bucket-bucketobject-interpolate-csharp
- aws-s3bucket-bucketpolicy-csharp
- aws-s3websitebucket-oai-bucketpolicy-csharp
- aws-simulated-dbserver-database-csharp
- azure-native-import-resource-group-csharp
- azure-native-import-resource-group-go
- azure-native-import-resource-group-python
- azure-native-import-resource-group-yaml
- deepseek-ollama-csharp
- deepseek-ollama-go
- gcp-import-service-account-csharp
- gcp-import-service-account-go
- gcp-import-service-account-javascript
- gcp-import-service-account-python
- gcp-import-service-account-typescript
- gcp-import-service-account-yaml
- gcp-simple-webserver-yaml
- helm-kubernetes-csharp
- helm-kubernetes-go
- helm-kubernetes-javascript
- helm-kubernetes-part-two-csharp
- helm-kubernetes-part-two-go
- helm-kubernetes-part-two-python
- helm-kubernetes-part-two-typescript
- helm-kubernetes-part-two-yaml
- helm-kubernetes-python
- helm-kubernetes-typescript
- k8s-deployment-service-for-minikube-csharp
- k8s-deployment-service-for-minikube-go
- kubernetes-csi-driver-csharp
- kubernetes-csi-driver-go
- kubernetes-csi-driver-javascript
- kubernetes-csi-driver-python
- kubernetes-csi-driver-typescript
- kubernetes-deployments-csharp
- kubernetes-deployments-java
- kubernetes-deployments-typescript

I'm going to take a quick pass through to try to bring as many of these back to passing as I can, if they are quick.

@thoward thoward mentioned this issue Jan 30, 2025
Merged
@thoward
Copy link
Contributor

thoward commented Jan 30, 2025

Examples I was able to fix (see # )

  • aws-acm-certificate-csharp
  • aws-iampolicy-jsonparse-csharp
  • aws-s3bucket-bucketobject-interpolate-csharp
  • aws-s3bucket-bucketpolicy-csharp
  • aws-s3websitebucket-oai-bucketpolicy-csharp
  • aws-simulated-dbserver-database-csharp

Problem: .NET 7 specced
Resolution: Upgrade to .NET 8

  • deepseek-ollama-csharp
  • helm-kubernetes-part-two-csharp
  • kubernetes-csi-driver-csharp
  • kubernetes-deployments-csharp

Problem: .NET 9 specced
Resolution: Downgrade to .NET 8

  • deepseek-ollama-go

Problem: missing file open.rsa
Resolution: Copied file from csharp example into go example.

  • k8s-deployment-service-for-minikube-csharp

Problem: Missing comma causing syntax error
Resolution: Added comma

  • k8s-deployment-service-for-minikube-go

Problem: Two syntax errors (unused import, unused variable)
Resolution: Fixed those by removing the import and assigning to _

  • kubernetes-deployments-java

Problem: Misc minor bugs: no package name, class doesn't match filename, wrong type (DeploymentArgs vs DeploymentSpecArgs)
Resolution: Fixed minor code bugs.

  • kubernetes-deployments-typescript

Problem: Missing required configuration variables
Resolution: Updated the code example to use config.get(...) || "default"; instead of config.require(...); which allows it to run in our testing environment where configuration values aren't set externally.

@thoward
Copy link
Contributor

thoward commented Jan 30, 2025 •
edited
Loading

The ones I was not able to quick-fix (need some additional input from the authors @toriancrane , @dirien):

  • azure-native-import-resource-group-csharp
  • azure-native-import-resource-group-go
  • azure-native-import-resource-group-python
  • azure-native-import-resource-group-yaml

Problem: All failing due to the same underlying error:

azure-native:resources:ResourceGroup (imported-rg):
    error: Preview failed: resource '/subscriptions/***/resourceGroups/pulumi-tutorials' does not exist

Resolution: TBD ask original author(s)

  • gcp-import-service-account-csharp
  • gcp-import-service-account-go
  • gcp-import-service-account-javascript
  • gcp-import-service-account-python
  • gcp-import-service-account-typescript
  • gcp-import-service-account-yaml
  • gcp-simple-webserver-yaml

Problem: Insufficient permissions on IAM role/account

 error: Preview failed: refreshing urn:pulumi:dev::gcp-import-service-account-csharp::gcp:serviceaccount/account:Account::imported-tutorial-service-account: 1 error occurred:
    	* Error when reading or editing Service Account "projects/pulumi-devrel/serviceAccounts/pulumi-tutorial-service-accoun@pulumi-devrel.iam.gserviceaccount.com": Get "https://iam.googleapis.com/v1/projects/pulumi-devrel/serviceAccounts/pulumi-tutorial-service-accoun@pulumi-devrel.iam.gserviceaccount.com?alt=json&prettyPrint=false": oauth2/google: status code 403: {
      "error": {
        "code": 403,
        "message": "Permission 'iam.serviceAccounts.getAccessToken' denied on resource (or it may not exist).",
        "status": "PERMISSION_DENIED",
        "details": [
          {
            "@type": "type.googleapis.com/google.rpc.ErrorInfo",
            "reason": "IAM_PERMISSION_DENIED",
            "domain": "iam.googleapis.com",
            "metadata": {
              "permission": "iam.serviceAccounts.getAccessToken"
            }
          }
        ]
      }
    }

Resolution: TBD ask original author(s)

  • helm-kubernetes-csharp

Problem: Null object?

    error: Running program '/home/runner/work/docs/docs/static/programs/helm-kubernetes-csharp/bin/Debug/net8.0/helm-kubernetes-csharp.dll' failed with an unhandled exception:
    System.NullReferenceException: Object reference not set to an instance of an object.
       at new MyStack() in /home/runner/work/docs/docs/static/programs/helm-kubernetes-csharp/Program.cs:line 185
       at object RuntimeType.CreateInstanceDefaultCtor(bool publicOnly, bool wrapExceptions)
       at object RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, object[] args, CultureInfo culture)
       at Task<int> Pulumi.Deployment+Runner.Pulumi.IRunner.RunAsync<TStack>(IServiceProvider serviceProvider)+() => { }
       at Task<int> Pulumi.Deployment+Runner.RunAsync<TStack>(Func<TStack> stackFactory)

Resolution: TBD ask original author(s)

  • helm-kubernetes-go

Problem: Missing chart?

    kubernetes:helm.sh/v3:Release strimzi-kafka-operator  error: failed to pull chart: chart "strimzi-kafka-operator" version "latest" not found in https://strimzi.io/charts/ repository
 +  pulumi:pulumi:Stack helm-kubernetes-go-dev create 
    kubernetes:helm.sh/v3:Release strimzi-kafka-operator  1 error
Diagnostics:
  kubernetes:helm.sh/v3:Release (strimzi-kafka-operator):
    error: failed to pull chart: chart "strimzi-kafka-operator" version "latest" not found in https://strimzi.io/charts/

Resolution: TBD ask original author(s)

  • helm-kubernetes-javascript

Problem: Missing chart?

    kubernetes:helm.sh/v3:Release flux-operator  error: kubernetes:helm.sh/v3:Release resource 'flux-operator': property chart value {flux-operator} has a problem: looks like "oci://ghcr.io/controlplaneio-fluxcd/charts" is not a valid chart repository or cannot be reached: object required; check the chart name and repository configuration.
 +  pulumi:pulumi:Stack helm-kubernetes-javascript-dev create 
    kubernetes:helm.sh/v3:Release flux-operator  1 error
Diagnostics:
  kubernetes:helm.sh/v3:Release (flux-operator):
    error: kubernetes:helm.sh/v3:Release resource 'flux-operator': property chart value {flux-operator} has a problem: looks like "oci://ghcr.io/controlplaneio-fluxcd/charts" is not a valid chart repository or cannot be reached: object required; check the chart name and repository configuration.

Resolution: TBD ask original author(s)

  • helm-kubernetes-part-two-csharp
  • helm-kubernetes-part-two-go
  • helm-kubernetes-part-two-python
  • helm-kubernetes-part-two-typescript
  • helm-kubernetes-part-two-yaml

Problem: Missing chart?

 +  pulumi:pulumi:Stack helm-kubernetes-part-two-go-dev create error: kubernetes:helm.sh/v4:Chart resource 'cert-manager' has a problem: configured Kubernetes cluster is unreachable: unable to load Kubernetes client configuration from kubeconfig file. Make sure you have: 
 +  pulumi:pulumi:Stack helm-kubernetes-part-two-go-dev create 1 error
Diagnostics:
  pulumi:pulumi:Stack (helm-kubernetes-part-two-go-dev):
    error: kubernetes:helm.sh/v4:Chart resource 'cert-manager' has a problem: configured Kubernetes cluster is unreachable: unable to load Kubernetes client configuration from kubeconfig file. Make sure you have:
    
     	 • set up the provider as per https://www.pulumi.com/registry/packages/kubernetes/installation-configuration/
    
     invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable

Resolution: TBD ask original author(s)

  • helm-kubernetes-python

Problem: Incorrect type for opts?

  pulumi:pulumi:Stack (helm-kubernetes-python-dev):
    error: Program failed with an unhandled exception:
    Traceback (most recent call last):
      File "/home/runner/work/docs/docs/static/programs/helm-kubernetes-python/./__main__.py", line 5, in <module>
        ns = k8s.core.v1.Namespace(
      File "/home/runner/work/docs/docs/static/programs/helm-kubernetes-python/venv/lib/python3.8/site-packages/pulumi_kubernetes/core/v1/Namespace.py", line 133, in __init__
        __self__._internal_init(resource_name, *args, **kwargs)
      File "/home/runner/work/docs/docs/static/programs/helm-kubernetes-python/venv/lib/python3.8/site-packages/pulumi_kubernetes/core/v1/Namespace.py", line 143, in _internal_init
        opts = pulumi.ResourceOptions.merge(_utilities.get_resource_opts_defaults(), opts)
      File "/home/runner/work/docs/docs/static/programs/helm-kubernetes-python/venv/lib/python3.8/site-packages/pulumi/resource.py", line 664, in merge
        raise TypeError("Expected opts2 to be a ResourceOptions instance")
    TypeError: Expected opts2 to be a ResourceOptions instance

Resolution: TBD ask original author(s)

  • helm-kubernetes-typescript

Problem: Missing resource?

    kubernetes:helm.sh/v3:Release cloudnativepg  error: kubernetes:helm.sh/v3:Release resource 'cloudnativepg': property chart value {cloudnative-pg/cloudnative-pg} has a problem: repo cloudnative-pg not found; check the chart name and repository configuration.
    kubernetes:helm.sh/v3:Release cloudnativepg  1 error
 +  pulumi:pulumi:Stack helm-kubernetes-typescript-dev create 
Diagnostics:
  kubernetes:helm.sh/v3:Release (cloudnativepg):
    error: kubernetes:helm.sh/v3:Release resource 'cloudnativepg': property chart value {cloudnative-pg/cloudnative-pg} has a problem: repo cloudnative-pg not found; check the chart name and repository configuration.

Resolution: TBD ask original author(s)

  • kubernetes-csi-driver-go
  • kubernetes-csi-driver-javascript
  • kubernetes-csi-driver-python
  • kubernetes-csi-driver-typescript

Problem: unreachable cluster?

 +  pulumi:pulumi:Stack kubernetes-csi-driver-go-dev create error: kubernetes:helm.sh/v4:Chart resource 'secrets-store-csi-driver' has a problem: configured Kubernetes cluster is unreachable: unable to load Kubernetes client configuration from kubeconfig file. Make sure you have: 
 +  pulumi:pulumi:Stack kubernetes-csi-driver-go-dev create 1 error
Diagnostics:
  pulumi:pulumi:Stack (kubernetes-csi-driver-go-dev):
    error: kubernetes:helm.sh/v4:Chart resource 'secrets-store-csi-driver' has a problem: configured Kubernetes cluster is unreachable: unable to load Kubernetes client configuration from kubeconfig file. Make sure you have:
    
     	 • set up the provider as per https://www.pulumi.com/registry/packages/kubernetes/installation-configuration/
    
     invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable

Resolution: TBD ask original author(s)

@thoward thoward added area/docs Improvements or additions to documentation impact/usability Something that impacts users' ability to use the product easily and intuitively area/blog Content issues on blog posts. kind/bug Some behavior is incorrect or out of spec impact/flaky-test A test that is unreliable impact/regression Something that used to work, but is now broken and removed needs-triage Needs attention from the triage team labels Jan 30, 2025
@thoward thoward moved this from 🤔 Triage to 🔧 In Progress in Docs 📚 Jan 30, 2025
@dirien dirien mentioned this issue Jan 30, 2025
Merged
@dirien
Copy link
Contributor

dirien commented Jan 30, 2025

Hi @thoward,

Fixed the errors with this PR #13914

The examples:

  • kubernetes-csi-driver-go
  • kubernetes-csi-driver-javascript
  • kubernetes-csi-driver-python
  • kubernetes-csi-driver-typescript
  • helm-kubernetes-part-two-csharp
  • helm-kubernetes-part-two-go
  • helm-kubernetes-part-two-python
  • helm-kubernetes-part-two-typescript
  • helm-kubernetes-part-two-yaml

All need a local Kubernetes cluster to work with. We could run a kind cluster before the pulumi preview gets called. I would need time to make the changes in our Actions.

@thoward
Copy link
Contributor

thoward commented Jan 30, 2025

@dirien i recently added support for using a Makefile in example programs. Basically, include a Makefile in the root of the program directory with a test target and instead of trying to run pulumi preview to test your code it will run make test instead. You could use that to implement external setup, but you'd have to include your own pulumi preview code in the test target. You can see an example of that here: https://github.com/pulumi/docs/tree/master/static/programs/custom-policy-pack-integration-test-typescript/Makefile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@thoward thoward

@toriancrane toriancrane

@dirien dirien

Labels
area/blog Content issues on blog posts. area/docs Improvements or additions to documentation impact/flaky-test A test that is unreliable impact/regression Something that used to work, but is now broken impact/usability Something that impacts users' ability to use the product easily and intuitively kind/bug Some behavior is incorrect or out of spec kind/engineering Work that is not visible to an external user p1 A bug severe enough to be the next item assigned to an engineer
Projects
Docs 📚
Status: 🔧 In Progress
Milestone
No milestone
Development

No branches or pull requests
4 participants
@thoward @toriancrane @pulumi-bot @dirien