Establish Docker-registry:2 with simple ui

Author: Springjunky

README.md

@@ -10,12 +10,15 @@ My personal solution is a local, docker-based CI/CD Build Environment ready in a
   * docker
 * Gitlab 
   * and a docker based gitlab-runner registrated
-  * suecured docker-registry ready (openssh certificate)
+  * secured docker-registry ready (openssh certificate)
   * push with ssh avialable at port 2222
 * Nexus 3
 * Sonar
-* Keykloak (as Single Sign or JWT playground)
-* Postgres (used by sonar, keycloak and YOUR applications)
+* Keykloak (as Single Sign On or JWT playground)
+* Postgres (used by Sonar, Keycloak and YOUR applications)
+* Standard Docker-Registry V2 with a simple UI 
+
+
 
 so you can be your own local "DevOp"; nearly every category of [periodic table of devoptools][4] has one tool in your local setup. 
 
@@ -30,10 +33,11 @@ so you can be your own local "DevOp"; nearly every category of [periodic table o
 
 |Port  |  Why  |
 |---|---|
-|80 |NGINX   |
-|5432 |postgres standard |
-|2222 |ssh port of gitlab, used to push via ssh connection |
-|5555 |Gitlab Docker-registry |
+|80 |NGINX, forwards requests ton internal docker-containers   |
+|5432 |postgres standard connection port |
+|2222 |ssh port of gitlab, used to push via ssh connection ssh://git@myHOST:2222/scott/foo.git |
+|5555 |Gitlab docker-registry secured with GitLab credentials |
+|5000 |Standard Docker-Registry v2 not secured|
 
 If your change the ports in the docker-compose.yml change them also in nginx-reverse/nginx.conf (stream {...} )
 ## Installation
@@ -78,6 +82,7 @@ Now you are ready to go with a little CI/CD Environment:
  Gitlab  http://<your-host-name>/gitlab
  Sonar  http://<your-host-name>/sonar
  Keycloak http://<your-host-name>/auth
+ Docker-Registry-Ui: http://<your-host-name>/regweb 
  Postgres: At standard listenport 5432 for your jdbc-connection-string 
            stream-passthrough to postgres-container.
 ```
@@ -99,6 +104,9 @@ See Readme in folder security-paranoia if you want to have some hints how to con
 |Postgres|postgres|admin|
 
 ## The Tools
+There are some configurations you have to do after setup.
+*Remember:* At every time use your real hostname and NOT localhost (locahost inside a container is the container itself).
+For example: If yuo configure a GitLab-Connection in Jenkins, you will reach Gitlab at http://&lt;you host&gt;/gitlab and not http://localhost/gitlab
 ### Jenkins
 * MAVEN_HOME is /opt/maven
 * JAVA_HOME is /usr/lib/jvm/java-8-openjdk-amd64
@@ -109,10 +117,10 @@ See Readme in folder security-paranoia if you want to have some hints how to con
 * the docker-registry from GitLab is at port 5555 (and secured with an openssl certificate ..thats part of
   setupEnvironment.sh), just create a project in gitlab and click at the registry tab to show
   how to login to the project registry and how to tag your images and upload them.
-* ssh cloning and pushing is at port 2222
+* ssh cloning and pushing is at port 2222 (ssh://git@myHOST:2222/scott/foo.git remeber to upload your public key before, should be ~/.ssh/id_rsa.pub )
 
 #### gitlab-runner
-The runner is a gitlab-multirunner image with a docker-runner (concurrent=1) , based on [gitlab/gitlab-runner][2]  at every startup any runner is removed and only ONE new runner ist registrated to avoid multiple runners  (the pipeline-history maybe lost.) docker-in-docker works :-)
+The runner is a gitlab-multirunner image with a docker-runner (concurrent=1) , based on [gitlab/gitlab-runner][2]  at every startup any runner is removed and only ONE new runner is registrated to avoid multiple runners  (the pipeline-history maybe lost.) docker-in-docker works :-)
 
 It takes a long time until gitlab is ready to accept a runner registration, if it fails, increase the REGISTER_TRYS in docker-compse.yml
 
@@ -136,34 +144,35 @@ _tl;dr_
 * create client "product-app" as openid-connect client with Valid Redirect URI's  http://&lt;your host&gt;:8081/*
 * create role "user"
 * create user "testuser" and map the role "user" to testuser (tab Role Mappings)
- 
-### Nexus
-Some ToDo for me described here
-[Unsecure docker-registry in Nexus][1]
-use GitLab as a secured registry
-..
-And _yes_ docker-plugin in jenkins works (docker in docker, usefull but not recommended)
+
+
 ### Postgres
-You can use any tool to connect to the database at locahost:5432 this is a pass through to the container so any
+You can use any tool to connect to the database at localhost:5432 this is a pass through to the container so any
 JDBC-Connection should work
 
+### The Docker-Registries
+* NEXUS-Docker-Registry ist NOT configured .. needs a pass trough and some more configs, see [Unsecure docker-registry in Nexus][1] feel free to provide a push-request
+* GitLab docker-registry is at port 5555 you have to use your GitLab Credentials from the corresponding git-respository
+* standard Docker-Registry v2 ist at standard port 5000 with no credentials, so eays-to-use
+
+
 ## Troubleshooting
 ##### check Hostname and IP
 In most cases a wrong HOSTNAME:HOSTIP causes trouble, to check this try the following.
 * log into the jenkins-fat container (with id)
 ```
   docker container ls
-  docker container exec -it dockerlocalbuildenvironment_jenkins_1 bash
+  docker container exec -it jenkins-fat bash
+  chmod a+rw /tmp
   apt-get update
   apt-get install -y --allow-unauthenticated iputils-ping
-  ping google.de
   ping jenkins
   ping gitlab
   ping <your local hostname>
 ```
 every ping must work, if not, check the .env file, is there the correct DC_HOSTNAME / DC_HOSTIP ?
 
-##### changed networks ?
+##### changed interface ip ?
   If you change your network (switching between home/office/lan/wifi) your ip-address
   could be change and the container is not able to resolve your host any more
   Check the .env file or just run the setup-Script again. 
@@ -185,7 +194,7 @@ every ping must work, if not, check the .env file, is there the correct DC_HOSTN
 * ~~apply git-lfs~~
 * ~~apply sonar~~
 * ~~apply keycloak~~
-* apply a better registry
+* ~~apply a better registry~~
 
 
 

docker-compose-keycloak.yml

@@ -6,6 +6,7 @@ version: "3"
 services:
   keycloak:
     image: jboss/keycloak
+    container_name: jboss-keycloak
     environment:
       - KEYCLOAK_USER=admin
       - KEYCLOAK_PASSWORD=admin

docker-compose-sonar.yml

@@ -7,6 +7,7 @@ version: "3"
 services:
   sonar:
     image: sonarqube
+    container_name: sonarcube
     depends_on: # start  proxy after all the others
       - postgres
     extra_hosts:

docker-compose.yml

@@ -14,21 +14,46 @@ networks:
 services:
 # ------------------------------------------------------------------------------
   ngnix:
+     container_name: nginx-reverse
      build: nginx-reverse
      ports:
-       - "80:80"             #http://
-       - "5432:5432"       # Default-Port of the Postges DB passing jdbc-Connections to the postgres-Container 
-       - "2222:2222"     #ssh port of gitlab (ssh://git@myHOST:2222/scott/foo.git)
-       - "5555:5555"  #Gitlab Docker Registry do NOT use 5000, this is an internal PORT of the gitlab-ce Image
-     command: ["nginx-debug", "-g", "daemon off;"]  # Start nginx in debug to see whats going on    
+       - 80:80      # http://
+       - 5432:5432  # Default-Port of the Postges DB passing jdbc-Connections to the postgres-Container 
+       - 2222:2222  # ssh port of gitlab (ssh://git@myHOST:2222/scott/foo.git)
+       - 5555:5555  # Gitlab Docker Registry do NOT use 5000, this is an internal PORT of the gitlab-ce Image
+     #command: ["nginx-debug", "-g", "daemon off;"]  # Start nginx in debug to see whats going on    
      depends_on: # start  proxy after all the others
        - gitlab
        - jenkins
        - nexus
      networks:
        - devstacknetwork
+# ------------------------------------------------------------------------------
+  regweb:
+    container_name: docker-registry-web
+    image: hyper/docker-registry-web
+    networks:
+      - devstacknetwork
+    extra_hosts:
+      - ${DC_HOSTNAME}:${DC_HOSTIP}
+    environment:
+      - REGISTRY_URL=http://${DC_HOSTNAME}:5000/v2  
+      - REGISTRY_NAME=Your docker-local-build-registry@${DC_HOSTNAME}:5000
+      - REGISTRY_READONLY=false 
+      - REGISTRY_CONTEXT_PATH=/regweb
+# ------------------------------------------------------------------------------
+  registry:
+    container_name: registry
+    image: registry:2
+    network_mode: "host"
+    ports:
+      - 5000:5000
+    volumes:
+      - ${DC_BASE_DATA_DIR}/docker-registry/config.yml:/etc/docker/registry/config.yml
+      - ${DC_BASE_DATA_DIR}/docker-registry/images:/var/lib/registry
 # ------------------------------------------------------------------------------
   postgres:
+    container_name: postgres
     build: postgres
     extra_hosts:
       - ${DC_HOSTNAME}:${DC_HOSTIP}
@@ -48,6 +73,7 @@ services:
 
   jenkins:
     build: jenkins-fat
+    container_name: jenkins-fat
     extra_hosts:
       - ${DC_HOSTNAME}:${DC_HOSTIP}
     networks:
@@ -59,6 +85,7 @@ services:
 # ------------------------------------------------------------------------------
   gitlab:
     image: 'gitlab/gitlab-ce:latest'
+    container_name: gitlab-ce
     extra_hosts:
       - ${DC_HOSTNAME}:${DC_HOSTIP}
     networks:
@@ -87,6 +114,7 @@ services:
 # ------------------------------------------------------------------------------
   nexus:
     image: sonatype/nexus3
+    container_name: sonatype-nexus3
     extra_hosts:
       - ${DC_HOSTNAME}:${DC_HOSTIP}
     networks:
@@ -99,6 +127,7 @@ services:
 # ------------------------------------------------------------------------------
   gitlabrunner:
     build: gitlabrunner
+    container_name: gitlabrunner
     network_mode: "host"
     extra_hosts:
       - ${DC_HOSTNAME}:${DC_HOSTIP}

jenkins-fat/.gitignore

@@ -0,0 +1 @@
+/active-plugins.txt_SAVE

jenkins-fat/Dockerfile

@@ -6,7 +6,6 @@ ENV JAVA_VERSION=8 \
     JAVA_HOME="/usr/lib/jvm/default-jvm"
 
 # update dpkg repositories and install tools
-
 RUN apt-get update
 #------------ Open JDK
 RUN apt-get install -y openjdk-8-jdk

nginx-reverse/reverse-proxy.conf

@@ -25,6 +25,12 @@ server {
       proxy_pass         http://nexus:8081;
       include /etc/nginx/conf.d/proxy-settings.conf;
     }
+
+  
+    location ^~/regweb/   {
+      proxy_pass         http://regweb:8080/regweb/;
+      include /etc/nginx/conf.d/proxy-settings.conf;
+    }
 
     # Same like sonar.web.context=/sonar in preconfig/sonar/sonar.properties
     location ^~/sonar {

preconfig/docker-registry/config.yml

@@ -0,0 +1,20 @@
+version: 0.1
+log:
+  fields:
+    service: registry
+storage:
+  delete:
+    enabled: true
+  cache:
+    blobdescriptor: inmemory
+  filesystem:
+    rootdirectory: /var/lib/registry
+http:
+  addr: :5000
+  headers:
+    X-Content-Type-Options: [nosniff]
+    Access-Control-Allow-Origin: ['*']
+    Access-Control-Allow-Methods: ['HEAD', 'GET', 'OPTIONS', 'DELETE']
+    Access-Control-Allow-Headers: ['Authorization']
+    Access-Control-Max-Age: [1728000]
+    Access-Control-Expose-Headers: ['Docker-Content-Digest']

setupEnvironment.sh

@@ -77,6 +77,7 @@ mkdir -p $USER_DATA_DIR/sonar/sonarqube_conf
 mkdir -p $USER_DATA_DIR/jenkins
 mkdir -p $USER_DATA_DIR/gitlab/config/ssl
 mkdir -p $USER_DATA_DIR/nexus
+mkdir -p $USER_DATA_DIR/docker-registry/images
 chown -R 200 $USER_DATA_DIR/nexus
 #----------------------------------
 
@@ -127,6 +128,10 @@ fi
 #Copy and modify predefined Jobs and Configs
 cp -r preconfig/jenkins/* $USER_DATA_DIR/jenkins/
 
+# Copy Registry Config
+cp preconfig/docker-registry/config.yml $USER_DATA_DIR/docker-registry
+
+# Prepare the sample-project to run on <your-host>
 sed -i s#HOSTNAME#${HOSTNAME}#g  spring-boot-keycloak-sample/src/main/resources/application.properties 
 sed -i s#HOSTNAME#${HOSTNAME}#g  spring-boot-keycloak-sample/src/main/resources/static/index.html 
 
@@ -148,19 +153,20 @@ echo "Environment for docker-compose.yml created"
 echo " "
 echo "use the following URL"
 BASE_URL="http://"$(hostname)"/"
-echo "Jenkins: ${BASE_URL}jenkins"
-echo "Nexus  : ${BASE_URL}nexus"
-echo "Gitlab : ${BASE_URL}gitlab"
-echo "Sonar: ${BASE_URL}sonar (optional)"
-echo "Keycloak: ${BASE_URL}auth (optional)"
+echo "Jenkins:            ${BASE_URL}jenkins"
+echo "Nexus  :            ${BASE_URL}nexus"
+echo "Gitlab :            ${BASE_URL}gitlab"
+echo "Docker-Registry-Ui: ${BASE_URL}regweb"
+echo "Sonar:              ${BASE_URL}sonar (optional)"
+echo "Keycloak:           ${BASE_URL}auth (optional)"
 echo "Feel free to provide push-requests :-)"
 pause 
 echo " "
 
 echo "Setup finished, just type the following commands to start and see the logs of your environment"
 echo "docker-compose up --build -d "
 echo "docker-compose logs -f"
-
+echo "be patient ...10 docker-containers needs time to start up "