Skip to content

Procedure support #74

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 15 commits into from
Closed

Procedure support #74

wants to merge 15 commits into from

Conversation

meatballhat
Copy link
Contributor

@meatballhat meatballhat commented Apr 16, 2025 •
edited
Loading

I'm able to run procedures by switching to python 3.13 and then running the server like this:

ORIG_PYTHONPATH="${PYTHONPATH}"
PYTHONPATH="."
# `pipelines-runtime` python code
PYTHONPATH+=":${PWD}/../pipelines-runtime"
# `cog-runtime` python code
PYTHONPATH+=":${PWD}/python"
# `cog-runtime` virtualenv
PYTHONPATH+=":${PWD}/.venv/lib/python3.13/site-packages"
# `pipelines-runtime` virtualenv
PYTHONPATH+=":${PWD}/../pipelines-runtime/.venv/lib/python3.13/site-packages"

PYTHONPATH+=":${ORIG_PYTHONPATH}"
export PYTHONPATH

printf 'PYTHONPATH:\n'
printenv PYTHONPATH | tr ':' '\n'

# not critical but yeah
export LOG_FORMAT=development
export LOG_LEVEL=debug

./cog-server server --use-procedure-mode --await-explicit-shutdown

and then making procedure requests like this (typical for procedures):

curl -s \
  -H 'Content-Type: application/json' \
  -X POST \
  -d '{"input":{"procedure_source_url":"file:///Users/me/code/replicate/pipelines-runtime/fixtures/hello-pipelines","token":"<REDACTED-REPLICATE-API-TOKEN>","inputs_json":"{\"word\":\"young sheldon\"}"}}' \
  http://127.0.0.1:5000/predictions

meatballhat and others added 15 commits April 15, 2025 00:02
@meatballhat
Define Runner as an interface
9d9b71a 
- to clarify the necessary contract
- so that a different implementation may be used for procedures
@meatballhat
Move server config type into ./internal/server
8a03a11
@meatballhat
Accept server config in NewRunner
a977f17
@meatballhat
Renames to put Runner and NewRunner in own file
8e5a1e4
@meatballhat
Stub in ProcedureRunner as (mostly) copy of PredictionRunner
c6f4b94
@nevillelyh
Support class-less predict fn for procedures
058f51c
@nevillelyh
Rename class_name to predictor_name
7b13b60
@meatballhat
Merge remote-tracking branch 'origin/neville/predict' into moar-proce…
3f2890b 
…dure-like
@meatballhat
Too many changes around procedure support
3f615bb 
- handling and bubbling up errors instead of panicking
- switching some function signatures to use pointers
- context-related resiliency
@meatballhat
A few smallish fixes to get procodure runner running
372b8f6
@meatballhat
Not modifying coglet for procedures!
1a47c96
@meatballhat
Getting procedure runner environment recycling happy
c989365
@meatballhat
Fix output handling for procedures
b67893e
@meatballhat
HACKS to get procedures running oh no
e69901e
@meatballhat
Whoops formatting and typing fixes
3668d23
@nevillelyh
Copy link
Contributor

This changes a lot of things. What exactly is needed for procedures to work on top of #73?

@nevillelyh nevillelyh mentioned this pull request May 7, 2025
Merged
@nevillelyh
Copy link
Contributor

Superceded by #81 🙇

@nevillelyh nevillelyh closed this Jun 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@meatballhat @nevillelyh