Dagster Helm, run a job in the same code-location server.

[andrescolodrero]

Hi,
can i launch a job exactly in the same container running user-code-location?
I´like to do same than when we are running "dagster dev", run another pythonm thread. can be done with custom launcher?
I want to do this for debugging purporsal

[dpeng817]

The default run launcher executes runs in a subprocess. If you wanted to use a python thread, you would have to write a custom run launcher.

[andrescolodrero]

What does it mean a "subprocess"? a OS subprocess? Im getting an strange behaviour when using a connection to database, all works from calling a script in the terminal, but not from the Dagster Job. #19429 When i execute the job in another container usinh k8srunlauncher i see: UID PID PPID C STIME TTY TIME CMD app 1 0 14 22:56 ? 00:00:02 /usr/local/bin/python /usr/local/bin/dagster api execute_run {"__class app 14 1 0 22:56 ? 00:00:00 /usr/local/bin/python -c from multiprocessing.resource_tracker import app 15 1 15 22:56 ? 00:00:02 /usr/local/bin/python -c from multiprocessing.spawn import spawn_main I will test a custom launcher, but i´d like to know what is happening. El lun, 29 ene 2024 a las 22:37, Christopher DeCarolis (< ***@***.***>) escribió:

…

The default run launcher executes runs in a subprocess. If you wanted to use a python thread, you would have to write a custom run launcher. — Reply to this email directly, view it on GitHub <#19460 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABDMUJRWCMD2LDX33TNIIJDYRAQDLAVCNFSM6AAAAABCPC6T3GVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DEOJSGMYTI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[dpeng817]

That's right, an OS subprocess opened by python's  subprocess module.
That latter bit seems like it should be a separate github discussion.

[andrescolodrero]

From a container im running the script with pydobc command + kerberos auth with no problem. I also tested with python subprocess and works fine.
There is something in the k8slauncher (and celery too) that is causing this error, very hard to analize.
i will try customLauncher, but im pretty new in dagster and there is no much documentation out there.

i created a custom launcher in my project
my_dagster_project/
|-- tutorial/
| |-- init.py
| |-- customlauncher/
| |-- init.py
| |-- pythonlauncher.py (with class

and then i reference:

 class: MyThreadedRunLauncher
      config: {}
      module: tutorial.customlauncher.pythonlauncher

but always get:
Caught an unrecoverable error while dequeuing the run. Marking the run as failed and dropping it from the queue
dagster._check.CheckError: Failure condition: Couldn't import module pythonlauncher when attempting to load the configurable class tutorial.customlauncher.pythonlauncher.MyThreadedRunLauncher