Skip to content

Latest commit

 

History

History
100 lines (90 loc) · 5.71 KB

README.md

File metadata and controls

100 lines (90 loc) · 5.71 KB

Repo to aid with reproducing issues on Nomad

Nomad infra

overview

Cluster Nomad Regions Nomad Datacenters Nomad Servers Nomad Clients Nomad Allocations Nomad Total CPU Nomad Total Memory Consul Datacenters
Development 1 1 3 2 27 9 Ghz 16 GiB 1
Staging/UAT 2 4 9 7 190 200 Ghz 240 GiB 2
Production 2 3 9 17 410 2655 Ghz 1873 GiB 2

setup

  • Ubuntu (22.04)
  • servers/masters -> Nomad + Consul + Unbound (service.consul) + Traefik + Keepalived (vm)
    • keepalived for centralized UI and APP differentation.
      • example
        • nomad-development.company.com IN A -> nomad-development-keepalived-VIP *.nomad-development.company.com IN A -> nomad-development-keepalived-VIP
        • nomad-staging.company.com IN A -> nomad-staging-keepalived-VIP *.nomad-staging.company.com IN A -> nomad-staging-keepalived-VIP
        • nomad-production.company.com IN A -> nomad-production-keepalived-VIP *.nomad-production.company.com IN A -> nomad-production-keepalived-VIP
          • why?
          • OIDC / Oauth like to have consistent callback-URL's, so if you login to Nomad-UI with OIDC, and a server fails, users can just keep using same login-url.
          • in jobspecs, you can work with templated traefik-rules to more easily automate a staging-job to get templated to jobname.nomad-staging.company.com
          • and if you leverage Letsencrypt (natively supported by Traefik) less hassle with SSL certificates.
            • of course, traefik also supports including custom certificates if preferred.
          • developers can work without too much manual interference needed.
  • Clients/Workers -> Nomad/Consul + Dockerd
    • DNS resolvers set to masters (service.consul)
  • Job deployments through Gitlab CI/CD
    • custom templating on gitlab-side & on nomad side with Levant

default nomad job stack per environment

  • may indicate jobs that do not run exactly on all environments.