Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added custom docker environment setting #60

Merged
merged 13 commits into from
Jan 10, 2022
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04
RUN apt-get update
RUN apt-get install ffmpeg libsm6 libxext6 -y
RUN apt-get update && apt-get install -y python3-opencv
RUN pip install opencv-python
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
{
"name": "environment_inference_dockerfile",
"version": "oct21",
"environmentVariables": {
"EXAMPLE_ENV_VAR": "EXAMPLE_VALUE"
},
"python": {
"userManagedDependencies": false,
"interpreterPath": "python",
"condaDependenciesFile": null,
"baseCondaEnvironment": null
},
"docker": {
"enabled": false,
"baseImage": null,
"sharedVolumes": true,
"shmSize": null,
"arguments": [],
"baseImageRegistry": {
"address": null,
"username": null,
"password": null,
"registryIdentity": null
},
"platform": {
"os": "Linux",
"architecture": "amd64"
}
},
"spark": {
"repositories": [],
"packages": [],
"precachePackages": true
},
"databricks": {
"mavenLibraries": [],
"pypiLibraries": [],
"rcranLibraries": [],
"jarLibraries": [],
"eggLibraries": []
},
"r": null,
"inferencingStackVersion": null
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
channels:
- conda-forge
dependencies:
- python=3.8
- pip:
- azureml-core==1.35.0
- azureml-defaults==1.35.0
- pandas
- joblib

name: basic_env
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04
RUN apt-get update
RUN apt-get install ffmpeg libsm6 libxext6 -y
RUN apt-get update && apt-get install -y python3-opencv
RUN pip install opencv-python
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
{
"name": "environment_training_dockerfile",
"version": "oct21",
"environmentVariables": {
"EXAMPLE_ENV_VAR": "EXAMPLE_VALUE"
},
"python": {
"userManagedDependencies": false,
"interpreterPath": "python",
"condaDependenciesFile": null,
"baseCondaEnvironment": null
},
"docker": {
"enabled": false,
"baseImage": null,
"sharedVolumes": true,
"shmSize": null,
"arguments": [],
"baseImageRegistry": {
"address": null,
"username": null,
"password": null,
"registryIdentity": null
},
"platform": {
"os": "Linux",
"architecture": "amd64"
}
},
"spark": {
"repositories": [],
"packages": [],
"precachePackages": true
},
"databricks": {
"mavenLibraries": [],
"pypiLibraries": [],
"rcranLibraries": [],
"jarLibraries": [],
"eggLibraries": []
},
"r": null,
"inferencingStackVersion": null
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
channels:
- conda-forge
dependencies:
- python=3.8
- pip:
- azureml-core==1.35.0
- azureml-defaults==1.35.0
- pandas
- joblib

name: basic_env
17 changes: 17 additions & 0 deletions docs/how-to/AdjustingDockerfile.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Customizing the base docker image (mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20210922.v1)
mariamedp marked this conversation as resolved.
Show resolved Hide resolved
You have the option to specify your custom docker images for training and inference separately.

## First Step: Define your dockerfile
In case you need to upgrade/change packages in Linux (using for example apt-get install / apt-get update) or define additional commands you have the option to configure the Dockerfile on your own.

**[Environments](../../configuration/environments/)** contains two docker-related folders "environment_inference_dockerfile" and "environment_training_dockerfile" with the corresponding Dockerfile (BaseDockerfile).

## Second Step: Change the environment path
After having set up your Dockerfile, you need to change the paths here: **[Configurations](../../configuration/configuration-aml.variables.yaml)**

Replace "AML_TRAINING_ENV_PATH" with "configuration/environments/environment_training_dockerfile" if you need the adjusted dockerfile for training.
Replace "AML_BATCHINFERENCE_ENV_PATH" with "configuration/environments/environment_inference_dockerfile" if you need the adjusted dockerfile for training.

In case you want to deploy your own webservice, you can customize your docker images differently. Issue [#29](https://github.com/microsoft/dstoolkit-mlops-base/issues/29) describes this enhancement.

For more detail, please refer to *[GettingStarted](GettingStarted.md)*
7 changes: 6 additions & 1 deletion docs/how-to/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,9 @@

If you see the error below. You have to ensure that the service connection is created at the Azure Machine Learning Workspace level and not Subscription level

![faq-error-invoke](../media/FAQ-invoke.error.png)
![faq-error-invoke](../media/FAQ-invoke.error.png)

## I would like to adjust the Docker Image. How can I change the Dockerfile?

Documentation can be found here:
**[Custom Dockerfile](../../docs/how-to/AdjustingDockerfile)**