device resgistry keeps crashing

[msatpathy26]

Getting the following error with CrashLoopBackOff while starting the cluster

{
 insertId: "1vafkt6ggu2m1r9sv"  
 labels: {
  k8s-pod/app: "device-registry"   
  k8s-pod/pod-template-hash: "54d64d4df9"   
 }
 logName: "projects/uptane-demo/logs/stderr"  
 receiveTimestamp: "2020-09-14T17:58:15.717960733Z"  
 resource: {
  labels: {
   cluster_name: "ota-ce"    
   container_name: "device-registry"    
   location: "us-central1-c"    
   namespace_name: "default"    
   pod_name: "device-registry-54d64d4df9-hdx9z"    
   project_id: "uptane-demo"    
  }
  type: "k8s_container"   
 }
 severity: "ERROR"  
 textPayload: "Exception in thread "main" java.lang.ExceptionInInitializerError
	at com.advancedtelematic.ota.deviceregistry.Boot.main(Boot.scala)
Caused by: akka.http.scaladsl.model.IllegalUriException: Illegal URI reference: Invalid input '/', expected userinfo-char, pct-encoded or '@' (line 1, column 22): http://localhost:tcp://10.7.240.236:80
"  
 timestamp: "2020-09-14T17:57:54.964280603Z"  
}

[alexhumphreys]

Will probably need more info to troubleshoot this, but that error message looks like it's trying to parse http://localhost:tcp://10.7.240.236:80 as a URI and failing.

At a guess some combination of config vars for HOST, PORT and SCHEME are incorrect/getting mixed up/combined. How are you starting the device registry, it looks like kubernetes? Would it be possible to send your config? You can also check the environment of the pod it's running in with kubectl exec $POD_NAME -- env and see if anything there looks out of line.

[msatpathy26]

@alexhumphreys , the env details cannot be retrieved as the container seems to have crashed.

NAME                                    READY   STATUS             RESTARTS   AGE

device-registry-54d64d4df9-rd4m8        0/1     CrashLoopBackOff   2          2m19s

Here is what the command returns

# kubectl exec device-registry-54d64d4df9-rd4m8 -- env
error: unable to upgrade connection: container not found ("device-registry")

Here are some additional information:

The YAML config:

apiVersion: v1
kind: Service
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"device-registry","namespace":"default"},"spec":{"ports":[{"port":80,"targetPort":9001}],"selector":{"app":"device-registry"}}}
  creationTimestamp: "2020-09-17T05:09:31Z"
  name: device-registry
  namespace: default
  resourceVersion: "1502"
  selfLink: /api/v1/namespaces/default/services/device-registry
  uid: 29e9d601-ddc3-4624-a527-e6043e722c3e
spec:
  clusterIP: 10.7.254.223
  ports:
  - port: 80
    protocol: TCP
    targetPort: 9001
  selector:
    app: device-registry
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

Also attaching the JSON log file for your reference.
null__logs__2020-09-17T10-52.LOG

[alexhumphreys]

Thanks for the logs! I ran them through jq with cat null__logs__2020-09-17T10-52.LOG | jq -r '.[] | .textPayload' and that gave me a reasonable looking stack trace:

	at com.advancedtelematic.ota.deviceregistry.Boot$.<clinit>(Boot.scala)
	at com.advancedtelematic.ota.deviceregistry.Boot$.<init>(Boot.scala:38)
	at com.advancedtelematic.ota.deviceregistry.Settings.$init$(Boot.scala:35)
	at akka.http.scaladsl.model.Uri$.apply(Uri.scala:188)
	at akka.http.scaladsl.model.Uri$.apply(Uri.scala:216)
	at akka.http.impl.model.parser.UriParser.parseUriReference(UriParser.scala:46)
	at akka.http.impl.model.parser.UriParser.fail(UriParser.scala:81)
	at akka.http.scaladsl.model.Uri$.fail(Uri.scala:831)

the fun part is probable at com.advancedtelematic.ota.deviceregistry.Settings.$init$(Boot.scala:35), which is this line here

ota-device-registry/src/main/scala/com/advancedtelematic/ota/deviceregistry/Boot.scala

Line 35 in d69ee9d

val directorUri = Uri(_config.getString("director.uri"))

So it looks like the config variables for director are in a twist. In #168 the env you provide has DIRECTOR_PORT=tcp://10.7.250.95:80 which gets used here: https://github.com/advancedtelematic/ota-device-registry/blob/master/src/main/resources/application.conf#L41. That would explain where the URI from your message http://localhost:tcp://10.7.240.236:80 appeared.

That variable should be a port, probably 9001 or 80 but could be something else depending on where your director service is running. Try update that and let me know how you get on.