start server via docker #373

de-code · 2019-05-31T09:44:38Z

No description provided.

de-code · 2019-05-31T09:59:58Z

@giorgiosironi
In the current deployment we run npm run bundle for the client as part of the formula. That will inject HTML into the index.html for NewRelic and Google Tag Manager. The PeerScout Python server, serving the API, also serves the client as static files (it only needs the JavaScript bundle and the index.html).

The problem is, that the peerscout image doesn't include node, and therefore we wouldn't be able to run npm run bundle. We could run the client container separately (and run npm run bundle as part of the deployment). Then we would use something like nginx to serve client and API under the same port.

However, it seems we should probably already include the bundled JavaScript in some container. We just need to inject the HTML into the index.html (into head, end of body and end of file). We could write a Python script to do that. I am just wondering whether that is already a solved problem?
How do we do that for other Single Page App projects? (the client could still be served by nginx - we could remove it from the peerscout image)

giorgiosironi · 2019-05-31T13:45:44Z

How do we do that for other Single Page App projects? (the client could still be served by nginx - we could remove it from the peerscout image)

Maybe the xpub people know of a solution as they integrated New Relic as well. There's ids derived from the account in the generated HTML/JavaScript so I'd avoid building it into the image as it should be configuration. We may have to mount an HTML file or a fragment of it as a volume?

de-code · 2019-05-31T13:57:57Z

How do we do that for other Single Page App projects? (the client could still be served by nginx - we could remove it from the peerscout image)

Maybe the xpub people know of a solution as they integrated New Relic as well. There's ids derived from the account in the generated HTML/JavaScript so I'd avoid building it into the image as it should be configuration. We may have to mount an HTML file or a fragment of it as a volume?

Yes, mounting the HTML fragements (that we already provide). It's just the small "script" (or similar) that bakes it into the correct place of the index.html (which currently is part of the webpack bundle process).

@diversemix how do you manage that in xpub?

diversemix · 2019-06-03T13:00:33Z

How do we do that for other Single Page App projects? (the client could still be served by nginx - we could remove it from the peerscout image)

Maybe the xpub people know of a solution as they integrated New Relic as well. There's ids derived from the account in the generated HTML/JavaScript so I'd avoid building it into the image as it should be configuration. We may have to mount an HTML file or a fragment of it as a volume?

Yes, mounting the HTML fragements (that we already provide). It's just the small "script" (or similar) that bakes it into the correct place of the index.html (which currently is part of the webpack bundle process).

@diversemix how do you manage that in xpub?

Because we are a SPA ... then its quite simple for us... https://github.com/elifesciences/elife-xpub/blob/cbb9bb9067d688090ea482cb634846dad4a61602/app.js#L1

The newrelic.js then pulls in the relevant stuff from the config:

  newrelic: {
    licenseKey: 'xxx',
    applicationID: '162979288',
  },

You could make the configuration then pull from the environment. We don't do this with the new relic config but do do it for other variables in config.

de-code · 2019-06-03T13:18:32Z

Because we are a SPA ... then its quite simple for us... https://github.com/elifesciences/elife-xpub/blob/cbb9bb9067d688090ea482cb634846dad4a61602/app.js#L1

Okay, thank you. I am guessing that is hooking into the express server or similar that pubsweet-server is using, to modify the HTML sent by the server. I could do something similar to the Python server (or in express if we changed it to serve the client separately). But might prefer re-using the existing HTML fragments.

de-code · 2019-06-12T08:27:03Z

Will implement the use of html fragments separately #387

de-code · 2019-06-12T08:28:30Z

Dockerfile

@@ -26,7 +26,7 @@ ARG install_dev
 COPY requirements.dev.txt ./
 RUN if [ "${install_dev}" = "y" ]; then pip install -r requirements.dev.txt; fi

-COPY --from=client --chown=elife:elife /home/node/client/ ${PROJECT_FOLDER}/client/
+COPY --from=client --chown=elife:elife /home/node/client/dist/ ${PROJECT_FOLDER}/client/dist/


We only need dist when running the server (no need to copy node_modules & co)

de-code · 2019-06-12T08:29:39Z

Dockerfile

@@ -35,7 +35,7 @@ COPY --chown=elife:elife app-defaults.cfg ${PROJECT_FOLDER}/

 USER root
 RUN mkdir .data && chown www-data:www-data .data
-RUN mkdir logs && chown www-data:www-data logs
+RUN mkdir logs && chown www-data:www-data logs && chmod -R a+w logs


It will be www-data and elife writing to the logs

Shouldn't it be just www-data if we run everything in the container? But fine to keep this backward compatibility if needed.

The image is (or would be) used to run the server as well as the "ETL" scripts (migrate schema, load data etc.). The latter is currently meant to be run via the elife user. But we could use www-data for both if you think that would be better.

de-code · 2019-06-12T08:31:26Z

@giorgiosironi any objections to merging this PR?

giorgiosironi · 2019-06-13T08:59:51Z

docker-compose.override.yml

+      dockerfile: Dockerfile
+    image: elifesciences/peerscout_init:${IMAGE_TAG}
+    volumes:
+      - config-aws:/home/elife/volume-config-aws


some weird copying from one folder to the other to avoid ownership clashes?

No problem with that, but why needing a volume if the copying is executed every time the container starts anyway?

I find that when the container is not run as root or the developer's user, then just mounting the credentials won't work because the permissions are meant to set up that only the user can read them. This makes the developers credentials available via the volume. (The init container is run as root, and therefore has permissions)

One alternative would maybe be to run the container using the developer's user (which seem to also be more complicated than it should). I experimented with a few approaches across the projects. Not sure which one is best. Any suggestions?

giorgiosironi

No particular objections

de-code · 2019-06-20T15:40:06Z

Also updated the base image because the previous base Python image had LANG set to en_US.UTF-8 which didn't actually exist in the config. Only C.UTF-8 was present. Which is also what LANG is set to in the latest version of our Python base image as well as other Python base images.

de-code added 3 commits May 31, 2019 16:04

start server via docker

abb59a2

run client bundle

1eb60cf

just copy client dist

1166161

de-code requested a review from giorgiosironi May 31, 2019 10:00

de-code self-assigned this May 31, 2019

de-code added 3 commits May 31, 2019 19:37

added update-data-and-reload target

fd7be07

create .data directory if it doesn't exist

6839fa2

updated README

61c665e

de-code mentioned this pull request Jun 12, 2019

allow use of html fragments when using docker image #387

Open

de-code changed the title ~~[wip] start server via docker~~ start server via docker Jun 12, 2019

de-code commented Jun 12, 2019

View reviewed changes

giorgiosironi reviewed Jun 13, 2019

View reviewed changes

giorgiosironi approved these changes Jun 13, 2019

View reviewed changes

de-code added 3 commits June 20, 2019 16:25

added unicode cache test

77e6460

updated base image to fix unicode issue due to LANG=en_US.UTF-8

96a4229

added server-test-pytest and run-preprocessing-step target

9e31d4d

de-code merged commit 94dcc14 into develop Jun 20, 2019

de-code deleted the docker-server-start branch June 20, 2019 15:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

start server via docker #373

start server via docker #373

de-code commented May 31, 2019

de-code commented May 31, 2019

giorgiosironi commented May 31, 2019

de-code commented May 31, 2019

diversemix commented Jun 3, 2019 •

edited

Loading

de-code commented Jun 3, 2019

de-code commented Jun 12, 2019

de-code Jun 12, 2019

de-code Jun 12, 2019

giorgiosironi Jun 13, 2019

de-code Jun 13, 2019

de-code commented Jun 12, 2019

giorgiosironi Jun 13, 2019

giorgiosironi Jun 13, 2019

de-code Jun 13, 2019

giorgiosironi left a comment

de-code commented Jun 20, 2019

start server via docker #373

start server via docker #373

Conversation

de-code commented May 31, 2019

de-code commented May 31, 2019

giorgiosironi commented May 31, 2019

de-code commented May 31, 2019

diversemix commented Jun 3, 2019 • edited Loading

de-code commented Jun 3, 2019

de-code commented Jun 12, 2019

de-code Jun 12, 2019

Choose a reason for hiding this comment

de-code Jun 12, 2019

Choose a reason for hiding this comment

giorgiosironi Jun 13, 2019

Choose a reason for hiding this comment

de-code Jun 13, 2019

Choose a reason for hiding this comment

de-code commented Jun 12, 2019

giorgiosironi Jun 13, 2019

Choose a reason for hiding this comment

giorgiosironi Jun 13, 2019

Choose a reason for hiding this comment

de-code Jun 13, 2019

Choose a reason for hiding this comment

giorgiosironi left a comment

Choose a reason for hiding this comment

de-code commented Jun 20, 2019

diversemix commented Jun 3, 2019 •

edited

Loading