clearml.conf environment variables passed into docker image are not in VS Code session

[johnml1135]

I pass environment variables into the docker image ['--env', 'MY_VARIABLE=my_value'], etc. When I run the clearml-session and log into VS Code through the SSH connection, the variables are no longer there.

These environment variables are needed (they contain AWS credentials, etc.). Can they (somehow) be passed to the SSH instance starting up?

[johnml1135]

I see in interactive_session.py, that the variables have to be manually exported. Here:

clearml-session/clearml_session/interactive_session_task.py

Lines 485 to 489 in ff8d97f

	'echo "export CLEARML_CONFIG_FILE={trains_config_file}" >> /etc/profile'.format(
	password=ssh_password,
	port=port,
	trains_config_file=os.environ.get("CLEARML_CONFIG_FILE") or os.environ.get("CLEARML_CONFIG_FILE"),
	)

Could somehow the added environment variables be appended to this list? Are the variables in task._get_runtime_properties()?

[jkhenning]

Hi @johnml1135 ,

Sorry, missed this 🙁

What clearml-agent version are you using?

[johnml1135]

I believe we are using the most recent, v1.1.1 (released in August).

[bmartinn]

Hi @johnml1135

I pass environment variables into the docker image ['--env', 'MY_VARIABLE=my_value'], etc. When I run the clearml-session and log into VS Code through the SSH connection, the variables are no longer there.

if you are massing variables into the docker, any process should have them, this means any process (including the SSH session) should have them. That said, it could be that they are overwritten, is that the case ?
What is the command line you are using to run celarml-session ? Are you passing --docker-args "--env MY_VARIABLE=my_value" ?

These environment variables are needed (they contain AWS credentials, etc.). Can they (somehow) be passed to the SSH instance starting up?

Wouldn't it make sense to have the credntials set in the clearml.conf file on the agent machine, so that any Task will have those credentials ?

Can you verify the issue with clearml-session==0.3.5 we have added a few features and this might have also got sorted.

[johnml1135]

The credentials are set in clearml.conf:

    # optional arguments to pass to docker image
    # these are local for this agent and will not be updated in the experiment's docker_cmd section
    extra_docker_arguments: ["--env","AWS_REGION=...",
        "--env","AWS_ACCESS_KEY_ID=.....",
        "--env","AWS_SECRET_ACCESS_KEY=....",
        "-v","C:\Users\....:/root/.cache/pypoetry",
        ]

These extra_docker_arguments are not passed to the ssh session (as they have to be explicitly stated as per the previous post I made). For the extra args to be passed, they would have to be added to:

clearml-session/clearml_session/interactive_session_task.py

Lines 485 to 489 in ff8d97f

	'echo "export CLEARML_CONFIG_FILE={trains_config_file}" >> /etc/profile'.format(
	password=ssh_password,
	port=port,
	trains_config_file=os.environ.get("CLEARML_CONFIG_FILE") or os.environ.get("CLEARML_CONFIG_FILE"),
	)

That is, there would need to be a new line that echo export MYVAR=MYVALUE for each "--env","AWS_ACCESS_KEY_ID=.....".

[jkhenning]

Can you verify the issue with clearml-session==0.3.5 we have added a few features and this might have also got sorted.

@johnml1135 just making sure to ask if you did 🙂