> ## Documentation Index
> Fetch the complete documentation index at: https://proto.evodesign.org/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Cloud Inference
> Deploy tools to a Modal workspace you own, then call them with device='modal'
# Cloud Inference
In addition to local device management and fan out, `proto-tools` also enables
users to scale their tool use beyond their local machine through an integration
with [Modal](https://modal.com/). Modal is a third party serverless compute
platform that allows users to execute models and tools in remote containers.
After setting up an account and deploying the tools you would like to have access to,
setting `device="modal"` in a tool config will dispatch the execution of your tool
to a remote modal container, allowing you to scale up to a large number of GPUs
on demand.
This guide walks through how to deploy and call [Protenix](https://github.com/bytedance/Protenix) on remote
compute using your Modal account. It assumes you have already followed the first-time setup in the
[Modal README](/docs/tools/modal-integration): an authenticated
account and an environment named `proto-env`.
Open as a runnable notebook
## Deploying a tool
This guide calls Protenix, so it needs the `protenix` app deployed into `proto-env`. If you have
not done that yet, or you are not sure, the status command reports what the environment holds:
```python python icon="python" theme={null}
!proto-tools deploy --status
```
If `protenix` is not deployed, deploy it now using the following command:
```python python icon="python" theme={null}
# Uncomment and run this line if you have not deployed protenix
#!proto-tools deploy --apps protenix --env proto-env
```
## Calling a deployed tool
Dispatch requires one configuration field. No Modal interface appears in the calling program, the
input and configuration models are the models used locally, and the returned object is a validated
`ProtenixOutput`. The physical device of the container is resolved during dispatch, so no GPU is
named by the caller.
```python python icon="python" theme={null}
from proto_tools.tools.structure_prediction.protenix import (
run_protenix, ProtenixInput, ProtenixConfig,
)
# Let's create an input containing GFP
GFP = (
"MSKGEELFTGVVPILVELDGDVNGHKFSVSGEGEGDATYGKLTLKFICTTGKLPVPWPTLVTTFSYGVQCFSRYPDHMKQ"
"HDFFKSAMPEGYVQERTIFFKDDGNYKTRAEVKFEGDTLVNRIELKGIDFKEDGNILGHKLEYNYNSHNVYIMADKQKNG"
"IKVNFKIRHNIEDGSVQLADHYQQNTPIGDGPVLLPDNHYLSTQSALSKDPNEKRDHMVLLEFVTAAGITHGMDELYK"
)
input = ProtenixInput(complexes=[GFP])
```
The input carries the biological entities and nothing about where the work runs. It is the same
`ProtenixInput` a local call takes, so the cell above would be unchanged if you dropped
`device="modal"` and folded on your own GPU.
Now the call itself. The first one is a cold start: Modal has to schedule a container, start it,
and load the model before inference begins, so expect it to take noticeably longer than the fold
alone.
```python python icon="python" theme={null}
# Run the tool
output = run_protenix(input, ProtenixConfig(device="modal"))
```
`output` is a validated `ProtenixOutput`, indistinguishable from one produced locally. The
structures carry their confidence metrics, and can be written to disk or rendered inline.
```python python icon="python" theme={null}
output.structures[0].visualize()
```
## Subsequent calls
A container does not shut down the moment it returns. It stays alive briefly with the model still
resident, so a call arriving within that window skips both the container start and the model load
and goes straight to inference.
`PROTO_MODAL_SCALEDOWN_WINDOW` sets how long that lasts, in seconds, and defaults to `30`. Run the
next cell soon after the one above and the difference is the whole cold start.
```python python icon="python" theme={null}
# Calling the tool again
output_again = run_protenix(input, ProtenixConfig(device="modal", seed=1))
```
Leave it longer than the window and the next call pays the cold start again, because the container
has been scaled down and its GPU released.
This is a trade rather than a free improvement. A longer window avoids cold starts and also bills
for an idle GPU for the duration, so raise it while working interactively and leave it low for
occasional calls. The value is read when the service is deployed, so it belongs in the environment
you deploy from:
```bash theme={null}
export PROTO_MODAL_SCALEDOWN_WINDOW=300 # keep containers warm for five minutes
```
## Go deeper
For the complete implementation reference, including the app manifest, image construction, standalone
overrides, fingerprinting and drift detection, the transport envelope, and live progress streaming,
consult the developer notes in the proto-tools repository:
Modal Deployment ReferenceThe app manifest, image construction, standalone overrides, fingerprinting, the transport envelope, and device="modal" dispatch.
