Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ansible playbook failing at "Deploy Database Directly" (Chap 01 / non-standard Vagrant box) #32

Open
jeff-tillinghast opened this issue Aug 25, 2023 · 3 comments

Comments

@jeff-tillinghast
Copy link

Hi all-- Because I'm running on an M1 Mac, I'm trying to make a non-standard local Vagrant configuration work. I'm running Vagrant with Parallels using box bento/ubuntu-22.04-arm64 (the recommended box per Parallels). In the Vagrantfile, I've changed line 12 to point to the according box, and line 18 to change from :virtual_box to :parallels. The box builds, and the playbook initiates through 'TASK [todo : python prerequisites]', but then it loses the plot-- it skips the next three steps and finally hits failure loop with multiple failures in 'deploy database directly', all referencing 'Invalid kube-config file. No configuration found.':

Output
TASK [todo : python prerequisites] *********************************************
changed: [host01]

TASK [todo : manifests directory] **********************************************
skipping: [host01]

TASK [todo : stage manifests] **************************************************
skipping: [host01] => (item=database.yaml)
skipping: [host01] => (item=application.yaml)
skipping: [host01] => (item=service.yaml)
skipping: [host01] => (item=scaler.yaml)
skipping: [host01]

TASK [todo : deploy database using operator] ***********************************
skipping: [host01]

TASK [todo : deploy database directly] *****************************************
FAILED - RETRYING: [host01]: deploy database directly (10 retries left).
[...]

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Could not create API client: Invalid kube-config file. No configuration found.
failed: [host01] (item=pvc.yaml.j2) => {"ansible_loop_var": "item", "attempts": 10, "changed": false, "item": "pvc.yaml.j2", "msg": "Could not create API client: Invalid kube-config file. No configuration found."}

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Could not create API client: Invalid kube-config file. No configuration found.
failed: [host01] (item=database-secret.yaml.j2) => {"ansible_loop_var": "item", "attempts": 10, "changed": false, "item": "database-secret.yaml.j2", "msg": "Could not create API client: Invalid kube-config file. No configuration found."}

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Could not create API client: Invalid kube-config file. No configuration found.
failed: [host01] (item=database-deploy.yaml.j2) => {"ansible_loop_var": "item", "attempts": 10, "changed": false, "item": "database-deploy.yaml.j2", "msg": "Could not create API client: Invalid kube-config file. No configuration found."}

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Could not create API client: Invalid kube-config file. No configuration found.
failed: [host01] (item=database-service.yaml.j2) => {"ansible_loop_var": "item", "attempts": 10, "changed": false, "item": "database-service.yaml.j2", "msg": "Could not create API client: Invalid kube-config file. No configuration found."}

PLAY RECAP *********************************************************************
host01 : ok=11 changed=10 unreachable=0 failed=1 skipped=3 rescued=0 ignored=0

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

/Output

The only other modification I made to the Vagrant was adding 'ansible.compatibility_mode = "2.0" to both of the 'config.vm.provision' blocks in response to output during my first test run. But adding/removing that condition doesn't affect the results or the output above.

Thanks for any help!

@jeff-tillinghast
Copy link
Author

Adding: I was able to use the image to continue through Chapter 1 until I got to the sample application. As a result of the failures above, K3S did not install properly. I manually installed K3S using the script at http://get.k3s.io, and it runs satisfactorily. But obviously the sample application did not install and start, so there are no pods running with the application or database.

@disposedtrolley
Copy link

@jeff-tillinghast I think it'll be a challenge to get this working, if it's at all possible.

The following change will allow k3s to install successfully and vagrant up to run without erroring:

diff --git a/setup/roles/k3s/defaults/main.yaml b/setup/roles/k3s/defaults/main.yaml
index 73fc135..5248305 100644
--- a/setup/roles/k3s/defaults/main.yaml
+++ b/setup/roles/k3s/defaults/main.yaml
@@ -1,3 +1,3 @@
 ---
-k3s_version: "v1.23.4+k3s1"
-k3s_url: https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/k3s
\ No newline at end of file
+k3s_version: "v1.30.1+k3s1"
+k3s_url: https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/k3s-arm64
\ No newline at end of file

The next issue occurs within the VM when K8s attempts to start the todo app:

vagrant@host01:~$ sudo k3s kubectl get pods
NAME                       READY   STATUS             RESTARTS      AGE
todo-db-66d6d87b67-psf98   1/1     Running            0             3m10s
todo-7c94fcc877-6zh69      0/1     CrashLoopBackOff   4 (88s ago)   3m9s
todo-7c94fcc877-5w5gd      0/1     CrashLoopBackOff   4 (80s ago)   3m9s
todo-7c94fcc877-xh7ph      0/1     CrashLoopBackOff   4 (73s ago)   3m9s

Examining the logs:

vagrant@host01:~$ sudo k3s kubectl logs todo-7c94fcc877-6zh69
exec /usr/local/bin/docker-entrypoint.sh: exec format error

which is due to the todo app's container image (referenced in setup/roles/todo/templates/application.yaml.j2) being built for amd64 only: https://hub.docker.com/r/bookofkubernetes/todo

@jdieg0
Copy link

jdieg0 commented Sep 12, 2024

Hi @disposedtrolley, @jeff-tillinghast,

thanks for the discussion. This is exactly where I got too with Vagrant and Parallels and the box bento/ubuntu-22.04 on an M1Pro before I started searching through the issues here in the GitHub repo.

vagrant@host01:~$ sudo su -
root@host01:~# k3s kubectl get pods
NAME                       READY   STATUS             RESTARTS      AGE
todo-db-585649889f-msvcg   1/1     Running            0             8m27s
todo-6bd859fdd5-s27ws      0/1     CrashLoopBackOff   6 (86s ago)   8m26s
todo-6bd859fdd5-zssqz      0/1     CrashLoopBackOff   6 (82s ago)   8m26s
todo-6bd859fdd5-bn7br      0/1     CrashLoopBackOff   6 (66s ago)   8m26s
root@host01:~# k3s kubectl logs todo-6bd859fdd5-s27ws
standard_init_linux.go:228: exec user process caused: exec format error
root@host01:~# k3s kubectl describe pod todo-6bd859fdd5-s27ws
Name:         todo-6bd859fdd5-s27ws
Namespace:    default
Priority:     0
Node:         host01/10.211.55.4
Start Time:   Thu, 12 Sep 2024 12:11:17 -0500
Labels:       app=todo
              pod-template-hash=6bd859fdd5
Annotations:  <none>
Status:       Running
IP:           10.42.0.6
IPs:
  IP:           10.42.0.6
Controlled By:  ReplicaSet/todo-6bd859fdd5
Containers:
  todo:
    Container ID:   containerd://adfcb2802330ee236c2da4f3f3735f702324322ff2c0376c689907041f07d955
    Image:          bookofkubernetes/todo:1.0.2
    Image ID:       docker.io/bookofkubernetes/todo@sha256:4e2d05617cce02695d0c8a0b8514176cbc586b3fce9f544cca2e7f263509df77
    Port:           5000/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Thu, 12 Sep 2024 12:23:23 -0500
      Finished:     Thu, 12 Sep 2024 12:23:23 -0500
    Ready:          False
    Restart Count:  7
    Limits:
      cpu:     50m
      memory:  128Mi
    Requests:
      cpu:     50m
      memory:  128Mi
    Environment:
      NODE_ENV:    production
      PREFIX:      /todo
      PGHOST:      todo-db
      PGDATABASE:  todo
      PGUSER:      <set to the key 'POSTGRES_USER' in secret 'todo-db'>      Optional: false
      PGPASSWORD:  <set to the key 'POSTGRES_PASSWORD' in secret 'todo-db'>  Optional: false
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-54bt5 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  kube-api-access-54bt5:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Guaranteed
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                 From               Message
  ----     ------     ----                ----               -------
  Normal   Scheduled  16m                 default-scheduler  Successfully assigned default/todo-6bd859fdd5-s27ws to host01
  Normal   Pulling    15m                 kubelet            Pulling image "bookofkubernetes/todo:1.0.2"
  Normal   Pulled     14m                 kubelet            Successfully pulled image "bookofkubernetes/todo:1.0.2" in 1m12.166504064s
  Normal   Pulled     13m (x4 over 14m)   kubelet            Container image "bookofkubernetes/todo:1.0.2" already present on machine
  Normal   Created    13m (x5 over 14m)   kubelet            Created container todo
  Normal   Started    13m (x5 over 14m)   kubelet            Started container todo
  Warning  BackOff    48s (x65 over 14m)  kubelet            Back-off restarting failed container
➜  nostarch-kubernetes git:(main) ✗ docker run --platform linux/arm64 -it bookofkubernetes/todo:1.0.2 /bin/sh
Unable to find image 'bookofkubernetes/todo:1.0.2' locally
1.0.2: Pulling from bookofkubernetes/todo
Digest: sha256:4e2d05617cce02695d0c8a0b8514176cbc586b3fce9f544cca2e7f263509df77
Status: Image is up to date for bookofkubernetes/todo:1.0.2
docker: image with reference docker.io/bookofkubernetes/todo:1.0.2 was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64.
See 'docker run --help'.

@AlanHohn Are there any ambitions to provide multi-architecture images, e.g. built with docker buildx? 😇

Best regards,
Diego

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants