Skip to main content
Modal

​
Cloud Inference

In addition to local device management and fan out, proto-tools also enables users to scale their tool use beyond their local machine through an integration with Modal. Modal is a third party serverless compute platform that allows users to execute models and tools in remote containers. After setting up an account and deploying the tools you would like to have access to, setting device="modal" in a tool config will dispatch the execution of your tool to a remote modal container, allowing you to scale up to a large number of GPUs on demand. This guide walks through how to deploy and call Protenix on remote compute using your Modal account. It assumes you have already followed the first-time setup in the Modal README: an authenticated account and an environment named proto-env. Open as a runnable notebook

​
Deploying a tool

This guide calls Protenix, so it needs the protenix app deployed into proto-env. If you have not done that yet, or you are not sure, the status command reports what the environment holds:
python
If protenix is not deployed, deploy it now using the following command:
python

​
Calling a deployed tool

Dispatch requires one configuration field. No Modal interface appears in the calling program, the input and configuration models are the models used locally, and the returned object is a validated ProtenixOutput. The physical device of the container is resolved during dispatch, so no GPU is named by the caller.
python
The input carries the biological entities and nothing about where the work runs. It is the same ProtenixInput a local call takes, so the cell above would be unchanged if you dropped device="modal" and folded on your own GPU. Now the call itself. The first one is a cold start: Modal has to schedule a container, start it, and load the model before inference begins, so expect it to take noticeably longer than the fold alone.
python
output is a validated ProtenixOutput, indistinguishable from one produced locally. The structures carry their confidence metrics, and can be written to disk or rendered inline.
python

​
Subsequent calls

A container does not shut down the moment it returns. It stays alive briefly with the model still resident, so a call arriving within that window skips both the container start and the model load and goes straight to inference. PROTO_MODAL_SCALEDOWN_WINDOW sets how long that lasts, in seconds, and defaults to 30. Run the next cell soon after the one above and the difference is the whole cold start.
python
Leave it longer than the window and the next call pays the cold start again, because the container has been scaled down and its GPU released. This is a trade rather than a free improvement. A longer window avoids cold starts and also bills for an idle GPU for the duration, so raise it while working interactively and leave it low for occasional calls. The value is read when the service is deployed, so it belongs in the environment you deploy from:

​
Go deeper

For the complete implementation reference, including the app manifest, image construction, standalone overrides, fingerprinting and drift detection, the transport envelope, and live progress streaming, consult the developer notes in the proto-tools repository: Modal Deployment ReferenceThe app manifest, image construction, standalone overrides, fingerprinting, the transport envelope, and device=“modal” dispatch.