Advanced Deploy HW - instance size check

[stencell]

The FTL grader is not checking instance size. In this example, the workers should be large and failing the check. They are not.

`[someone@clientvm 0 ~]$ oc get machine -n openshift-machine-api
NAME INSTANCE STATE TYPE REGION ZONE AGE
cluster-03bc-jgffs-master-0 i-0aaaaf56d5f5d1c07 running m5.xlarge eu-central-1 eu-central-1a 119m
cluster-03bc-jgffs-master-1 i-0bd2f672c9d9ec863 running m5.xlarge eu-central-1 eu-central-1b 119m
cluster-03bc-jgffs-master-2 i-0c8a049ce4e1f576e running m5.xlarge eu-central-1 eu-central-1c 119m
cluster-03bc-jgffs-worker-eu-central-1a-wrqlt i-002e96923d69ec3f8 running m5.xlarge eu-central-1 eu-central-1a 118m
cluster-03bc-jgffs-worker-eu-central-1b-cg78h i-0238ae75d795c1f0c running m5.xlarge eu-central-1 eu-central-1b 118m
infra-1a-x2wvf i-0dbcb8d82f6003e2c running m5.xlarge eu-central-1 eu-central-1a 21m
infra-1b-p9zbw i-061f4f3d13d19c54b running m5.xlarge eu-central-1 eu-central-1b 21m
[someone@clientvm 0 ~]$ cat /tmp/grading_report.txt
Course name: ocp_adv_infra
Lab number: hw
Student id: tom.gruenheit-opitz-consulting.c

FAIL: Check if cluster monitoring is running on infra nodes: Cluster monitoring is not running on infra nodes
FAIL: Check if Elasticsearch is running on infra nodes: Elasticsearch is not running on infra nodes
FAIL: Check if image registry is running on infra nodes: The image registry is not running on infra nodes
FAIL: Check if the CPU quota is 4 in the project george-test: CPU ResourceQuota is not set to 4 in the project george-test
FAIL: Check if the CPU request is 500m for Elasticsearch: CPU request is not 500m for Elasticsearch
FAIL: Check if the default CPU request is 500m in the LimitRange: Default CPU request is not 500m in the LimitRange project-limits
FAIL: Check if the default memory request is 500Mi in the LimitRange: Default CPU request is not 500Mi in the LimitRange project-limits
FAIL: Check if the memory request is 4Gi for Elasticsearch: Memory request is not 4Gi for Elasticsearch
FAIL: Check if the storage request quota is 20G in the project george-test: Storage requests quota is not set to 20G in the project george-test
FAIL: Check if the storage size is 20G for Elasticsearch: Storage size is not 20G for Elasticsearch
FAIL: Check LimitRange in project george-test: LimitRange project-limits doesn't exist in project george-test
FAIL: Check NetworkPolicy allow-from-openshift-ingress in project george-test: NetworkPolicy allow-from-openshift-ingress doesn't exist in project george-test
FAIL: Check NetworkPolicy allow-same-namespace in project george-test: NetworkPolicy allow-same-namespace doesn't exist in project george-test
FAIL: Check ResourceQuota in project george-test: ResourceQuota project-quota doesn't exist in project george-test
PASS: Check if 5 user identities exist
PASS: Check if john is a cluster admin
PASS: Check if the cluster has 2 infra nodes
PASS: Check if the cluster has 2 router pods
PASS: Check if the cluster has 2 worker nodes
PASS: Check if the cluster has 3 masters
PASS: Check if the routers are running on infra nodes

bc1b096a574bbe740de6fe0c48658ae31564401615b1973a41a056c702a760fe /tmp/grading_report.txt`

[stencell]

@pavelanni Did we fix this or decide to just skip it? I think skipping, but I'm not 100% sure.

[RobsonWatt]

Hi @pavelanni @stencell
I have the same issue on my homework grade review. let's take a look please:

Evidence 1:

[rwattcha-redhat.com@clientvm 0 ~]$ cat /tmp/grading_dir/grading_report.txt 
Course name: ocp4_advanced_deployment
Lab number: hw

Student id: 

PASS:  localhost                      Check if the cluster has 3 masters                
PASS:  localhost                      Check if the cluster has 2 infra nodes            
FAIL:  localhost                        Check if the cluster has 2 worker nodes The cluster doesn't have 2 worker nodes
PASS:  localhost                      Check if image registry is running on infra nodes 
PASS:  localhost                      Check if cluster monitoring is running on infra nodes
PASS:  localhost                      Check if Elasticsearch is running on infra nodes  
PASS:  localhost                      Check if the CPU request is 500m for Elasticsearch
PASS:  localhost                      Check if the memory request is 4Gi for Elasticsearch
PASS:  localhost                      Check if the storage size is 20G for Elasticsearch
PASS:  localhost                      Check if the cluster has 2 router pods            
PASS:  localhost                      Check if the routers are running on infra nodes   
PASS:  localhost                      Check if 5 user identities exist                  
PASS:  localhost                      Check if john is a cluster admin                  
PASS:  localhost                      Check LimitRange in project george-test           
PASS:  localhost                      Check if the default CPU request is 500m in the LimitRange
PASS:  localhost                      Check if the default memory request is 500Mi in the LimitRange
PASS:  localhost                      Check ResourceQuota in project george-test        
PASS:  localhost                      Check if the CPU quota is 4 in the project george-test
PASS:  localhost                      Check if the storage request quota is 20G in the project george-test
PASS:  localhost                      Check NetworkPolicy allow-same-namespace in project george-test
PASS:  localhost                      Check NetworkPolicy allow-from-openshift-ingress in project george-test
313cf8b87c00df39ef16c08604189352ab0de0b18ab150fb3e1d2bc0bb48f136  grading_report.txt

FAILED 1 Errors

Evidence 2:

[rwattcha-redhat.com@clientvm 0 ~]$ count=0;for node in $(oc get nodes | awk '{print $1}' |grep -v NAME);do if [[ $(oc describe node/$node |grep node-role.kubernetes.io| tr -d '[[:space:]]') == "node-role.kubernetes.io/worker=" ]]; then let count=count+1; fi; done; echo "Worker Nodes Count:" $count

Worker Nodes Count: 2

Evidence 3:

[rwattcha-redhat.com@clientvm 0 ~]$ oc get nodes |grep worker |wc -l
2

So, I've been reviewing playbook and I've find that it's passing by default a "4" into:

- name: Check if there are 2 workers
    include_role:
      name: grader_check_ocp_node_label
    vars:
      node_label: "node-role.kubernetes.io/worker="
      node_count: 4  # because infra are workers too
      task_description_message: Check if the cluster has 2 worker nodes
      student_error_message: "The cluster doesn't have 2 worker nodes"

So, when playbook try to validate it will always fail because node_count will be always a 4.:

- name: Check the node count
  set_fact:
    success: true
  when:
  - r_node.resources|length == node_count

I've changed it manually to:
node_count: 2 # because a worker should be removed
and it works.. but I don't want to fix it manually because I need the training official acreditation.

What do you think about it??? I've opened a RHU ticket with number RH0081956.

Regards,

Robson Watt

[stencell]

It is not ideal and is kind of a hack and should be better, but it does work as is. We've had literally hundreds of successful submissions. If we set it to 2, it should fail because, based on the assignment requirements, we should have FOUR workers. Two of them just happen to also be infra. You can ping me on gchat if you want me to take a look at your environment to see if there is some new behaviour with newer version of OCP, but otherwise this probably won't be changed for now.