Run the template build on the infrastructure #782
Conversation
dobrac
force-pushed
the
experiment-run-the-docker-build-inside-of-firecracker-e2b-2147
branch
2 times, most recently
from
d06fdc7 to
251dcc1
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
dobrac
force-pushed
the
experiment-run-the-docker-build-inside-of-firecracker-e2b-2147
branch
from
f571c59 to
ce82650
Compare
dobrac
force-pushed
the
experiment-run-the-docker-build-inside-of-firecracker-e2b-2147
branch
from
61b0a20 to
a19207e
Compare
dobrac
changed the title
dobrac
force-pushed
the
experiment-run-the-docker-build-inside-of-firecracker-e2b-2147
branch
from
a19207e to
2bdcbd2
Compare
…tracting an archive containing symbolic links Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
dobrac
force-pushed
the
experiment-run-the-docker-build-inside-of-firecracker-e2b-2147
branch
2 times, most recently
from
a84461d to
33031ce
Compare
dobrac
changed the title
| "500": | ||
| $ref: "#/components/responses/500" | ||
|
|
||
| /templates/{templateID}/files/{hash}: |
There was a problem hiding this comment.
Maybe it makes sense to prefix this one also with V2, because it's part of the new feature that comes by V2?
There was a problem hiding this comment.
I've made v2 only the endpoints, that have version two (we have the same example with sandboxes list), this one is new, so no v2 required here. V2 has nothing to do with the Build System V2, but I understand the question
| @@ -36,6 +37,10 @@ func (fs *FileSystemStorageProvider) GetDetails() string { | |||
| return fmt.Sprintf("[Local file storage, base path set to %s]", fs.basePath) | |||
| } | |||
|
|
|||
| func (fs *FileSystemStorageProvider) SignedURL(_ context.Context, _ string, _ time.Duration) (string, error) { | |||
There was a problem hiding this comment.
How will the testing of the new build system work in integration tests/environments without object storage?
| @@ -75,6 +78,20 @@ func (a *AWSBucketStorageProvider) GetDetails() string { | |||
| return fmt.Sprintf("[AWS Storage, bucket set to %s]", a.bucketName) | |||
| } | |||
|
|
|||
| func (a *AWSBucketStorageProvider) SignedURL(ctx context.Context, path string, ttl time.Duration) (string, error) { | |||
There was a problem hiding this comment.
Can we maybe name it with something like UploadSignedURL?
| @@ -75,4 +113,7 @@ service TemplateService { | |||
|
|
|||
| // todo (2025-05): this is deprecated, please use InfoService that is used for both orchestrator and template manager | |||
| rpc HealthStatus (google.protobuf.Empty) returns (HealthStatusResponse); | |||
|
|
|||
| // TemplateLayerFilesUpload requests an upload URL for a file to be cached for the template build. | |||
| rpc TemplateLayerFilesUpload (TemplateLayerFilesUploadRequest) returns (TemplateLayerFilesUploadResponse); | |||
There was a problem hiding this comment.
This will basically run for every file? So if I have a folder with hundreds 10s or 100s of files, it will run this many times? Of course, it will be ideally just for the first build, but still.
Can we not dot it array at least? Maybe it will not be sent in batches now, but just make the interface prepared for that.
There was a problem hiding this comment.
This runs for every layer, every layer is one tar file
| # import { Sandbox } from 'e2b' | ||
| # const sandbox = await Sandbox.create('e2b-build-engine') | ||
|
|
||
| template_id = "p9kw2u9cc1zj1cov2zru" |
There was a problem hiding this comment.
How will this work on different clusters?
There was a problem hiding this comment.
the idea is that it's the same as the base image. Then it's references by the alias. Do you think there will be any issues with that?
Add basic ability to run template building on the infrastructure directly, instead of using local Docker.
This PR adds the following endpoints
The flow is described here: https://www.notion.so/e2bdev/API-Communication-Spec-20db8c29687380ab802be1a6f23eabdd?source=copy_link
Required changes
In order to use the new builder, you need to first run
make build-engine-templatein thepackages/sharedpackage. The build engine Docker image is available undere2bdev/build-engine.If the engine is not built or no steps are provided, you can still use the previous way where the already built Docker image is supplied.