> ## Documentation Index
> Fetch the complete documentation index at: https://proto.evodesign.org/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# ProGen3

> First released in 2025, ProGen3 is a family of autoregressive protein language models from Profluent that use a sparse mixture-of-experts architecture. It is trained on a large curated corpus of natural protein sequences.

<div class="page-hero"><img class="page-hero-banner" src="https://proto-bio.github.io/proto-assets/images/tool/progen3/hero.png" alt="ProGen3" /><div class="tool-org-badges page-hero-badges"><a href="/docs/tools/organizations/profluent" class="tool-org-badge" style={{background: "#2A9D8F"}} title="Profluent"><img src="https://mintcdn.com/bio-pro/_UGa2jUMKeVPCbLk/assets/images/cached/e4f184a1b145.png?fit=max&auto=format&n=_UGa2jUMKeVPCbLk&q=85&s=c04d16dc65a95665e453a13a8684438a" alt="" class="tool-org-badge-logo" width="200" height="200" data-path="assets/images/cached/e4f184a1b145.png" /> Profluent</a></div></div>

<Note>
  **License:** ProGen3 uses Apache-2.0 for code and CC-BY-NC-SA-4.0 for model weights and has restrictions around commercial use and may require explicit attribution when utilized. Please refer to the [code license](https://github.com/Profluent-AI/progen3/blob/main/LICENSE-CODE) and [model weights license](https://github.com/Profluent-AI/progen3#license) for full terms.
</Note>

<p class="entity-disclaimer">Proto is not affiliated with Profluent. This toolkit is open source and builds on the implementation produced by this organization. Product names, logos, and trademarks are the property of their respective owners.</p>

<hr class="entity-rule" />

<input type="radio" name="tab-progen3" id="none-progen3" class="tab-radio-input" />

<input type="radio" name="tab-progen3" id="github-progen3" class="tab-radio-input" defaultChecked />

<input type="radio" name="tab-progen3" id="hf-progen3" class="tab-radio-input" />

<input type="radio" name="tab-progen3" id="preprint-progen3" class="tab-radio-input" />

<input type="radio" name="tab-progen3" id="cite-progen3" class="tab-radio-input" />

<input type="radio" name="tab-progen3" id="source-progen3" class="tab-radio-input" />

<input type="radio" name="tab-progen3" id="notebook-progen3" class="tab-radio-input" />

<input type="radio" name="tab-progen3" id="proto-progen3" class="tab-radio-input" />

<div class="tool-tab-bar">
  <span class="tool-tab-wrap"><label for="github-progen3" class="tool-tab tab-open badge-github"><svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor"><path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" /></svg> GitHub</label><label for="none-progen3" class="tool-tab tab-close badge-github"><svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor"><path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" /></svg> GitHub</label></span> <span class="tool-tab-wrap"><label for="hf-progen3" class="tool-tab tab-open badge-hf"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> HuggingFace</label><label for="none-progen3" class="tool-tab tab-close badge-hf"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> HuggingFace</label></span> <span class="tool-tab-wrap"><label for="preprint-progen3" class="tool-tab tab-open badge-preprint"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z" /><polyline points="14 2 14 8 20 8" /><line x1="16" y1="13" x2="8" y2="13" /><line x1="16" y1="17" x2="8" y2="17" /><polyline points="10 9 9 9 8 9" /></svg> Preprint</label><label for="none-progen3" class="tool-tab tab-close badge-preprint"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z" /><polyline points="14 2 14 8 20 8" /><line x1="16" y1="13" x2="8" y2="13" /><line x1="16" y1="17" x2="8" y2="17" /><polyline points="10 9 9 9 8 9" /></svg> Preprint</label></span> <span class="tool-tab-wrap"><label for="cite-progen3" class="tool-tab tab-open badge-cite"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M3 21c3 0 7-1 7-8V5c0-1.25-.756-2.017-2-2H4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2 1 0 1 0 1 1v1c0 1-1 2-2 2s-1 .008-1 1.031V20c0 1 0 1 1 1z" /><path d="M15 21c3 0 7-1 7-8V5c0-1.25-.757-2.017-2-2h-4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2h.75c0 2.25.25 4-2.75 4v3c0 1 0 1 1 1z" /></svg> Cite</label><label for="none-progen3" class="tool-tab tab-close badge-cite"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M3 21c3 0 7-1 7-8V5c0-1.25-.756-2.017-2-2H4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2 1 0 1 0 1 1v1c0 1-1 2-2 2s-1 .008-1 1.031V20c0 1 0 1 1 1z" /><path d="M15 21c3 0 7-1 7-8V5c0-1.25-.757-2.017-2-2h-4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2h.75c0 2.25.25 4-2.75 4v3c0 1 0 1 1 1z" /></svg> Cite</label></span> <span class="tool-tab-wrap"><label for="source-progen3" class="tool-tab tab-open badge-source"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Tool Source</label><label for="none-progen3" class="tool-tab tab-close badge-source"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Tool Source</label></span> <span class="tool-tab-wrap"><label for="notebook-progen3" class="tool-tab tab-open badge-notebook"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2 3h6a4 4 0 0 1 4 4v14a3 3 0 0 0-3-3H2z" /><path d="M22 3h-6a4 4 0 0 0-4 4v14a3 3 0 0 1 3-3h7z" /></svg> Open as Notebook</label><label for="none-progen3" class="tool-tab tab-close badge-notebook"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2 3h6a4 4 0 0 1 4 4v14a3 3 0 0 0-3-3H2z" /><path d="M22 3h-6a4 4 0 0 0-4 4v14a3 3 0 0 1 3-3h7z" /></svg> Open as Notebook</label></span> <span class="tool-tab-wrap"><label for="proto-progen3" class="tool-tab tab-open badge-local"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="4 17 10 11 4 5" /><line x1="12" y1="19" x2="20" y2="19" /></svg> Run Locally</label><label for="none-progen3" class="tool-tab tab-close badge-local"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="4 17 10 11 4 5" /><line x1="12" y1="19" x2="20" y2="19" /></svg> Run Locally</label></span>
</div>

<a href="https://github.com/Profluent-AI/progen3" target="_blank" class="tab-panel github-panel" data-tab="github-progen3">
  <div class="gh-card-wrap">
    <img src="https://opengraph.githubassets.com/1/Profluent-AI/progen3" class="gh-card-img img-fallback" alt="Profluent-AI/progen3" />

    <div class="gh-card-fallback">
      <div class="gh-fallback-org"><svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor"><path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" /></svg> Profluent-AI/progen3</div>
    </div>
  </div>

  <span class="panel-goto-btn gh-goto-btn"><span><svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor"><path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" /></svg> View repo</span></span>
</a>

<div class="tab-panel hf-panel hf-multi-panel" data-tab="hf-progen3">
  <div class="hf-model-list">
    <a href="https://huggingface.co/Profluent-Bio/progen3-112m" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/Profluent-Bio/progen3-112m.png" class="hf-model-card-bg img-fallback" alt="progen3-112m" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen3-112m</span>
    </a>

    <a href="https://huggingface.co/Profluent-Bio/progen3-219m" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/Profluent-Bio/progen3-219m.png" class="hf-model-card-bg img-fallback" alt="progen3-219m" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen3-219m</span>
    </a>

    <a href="https://huggingface.co/Profluent-Bio/progen3-339m" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/Profluent-Bio/progen3-339m.png" class="hf-model-card-bg img-fallback" alt="progen3-339m" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen3-339m</span>
    </a>

    <a href="https://huggingface.co/Profluent-Bio/progen3-762m" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/Profluent-Bio/progen3-762m.png" class="hf-model-card-bg img-fallback" alt="progen3-762m" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen3-762m</span>
    </a>

    <a href="https://huggingface.co/Profluent-Bio/progen3-1b" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/Profluent-Bio/progen3-1b.png" class="hf-model-card-bg img-fallback" alt="progen3-1b" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen3-1b</span>
    </a>

    <a href="https://huggingface.co/Profluent-Bio/progen3-3b" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/Profluent-Bio/progen3-3b.png" class="hf-model-card-bg img-fallback" alt="progen3-3b" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen3-3b</span>
    </a>
  </div>
</div>

<a href="https://doi.org/10.1101/2025.04.15.649055" target="_blank" class="tab-panel preprint-panel" data-tab="preprint-progen3">
  <div class="paper-info">
    <div class="paper-title">Scaling unlocks broader generation and deeper functional understanding of proteins</div>
    <div class="paper-meta">Aadyot Bhatnagar, Sarthak Jain, ... Ali Madani</div>
    <div class="paper-meta paper-venue">bioRxiv (2025)</div>
  </div>

  <span class="panel-goto-btn preprint-goto-btn"><span><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z" /><polyline points="14 2 14 8 20 8" /><line x1="16" y1="13" x2="8" y2="13" /><line x1="16" y1="17" x2="8" y2="17" /><polyline points="10 9 9 9 8 9" /></svg> Read preprint</span></span>
</a>

<div class="tab-panel cite-panel" data-tab="cite-progen3">
  <div class="cite-code-wrap">
    ```bibtex theme={null}
    @article{bhatnagar2025progen3,
      title={Scaling unlocks broader generation and deeper functional understanding of proteins},
      author={Bhatnagar, Aadyot and Jain, Sarthak and Beazer, Joel and Curran, Samuel C and Hoffnagle, Alexander M and Ching, Kyle S and Martyn, Michael and Nayfach, Stephen and Ruffolo, Jeffrey A and Madani, Ali},
      journal={bioRxiv},
      year={2025},
      doi={10.1101/2025.04.15.649055},
      publisher={Cold Spring Harbor Laboratory}
    }
    ```
  </div>

  <span class="panel-goto-btn cite-copy-btn"><span><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M3 21c3 0 7-1 7-8V5c0-1.25-.756-2.017-2-2H4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2 1 0 1 0 1 1v1c0 1-1 2-2 2s-1 .008-1 1.031V20c0 1 0 1 1 1z" /><path d="M15 21c3 0 7-1 7-8V5c0-1.25-.757-2.017-2-2h-4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2h.75c0 2.25.25 4-2.75 4v3c0 1 0 1 1 1z" /></svg> Copy citation</span></span>
</div>

<a href="https://github.com/evo-design/proto-tools/tree/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen3" target="_blank" class="tab-panel source-panel" data-tab="source-progen3">
  <div class="source-info">
    <img src="https://github.com/evo-design.png?size=40" class="source-avatar" width="36" height="36" />

    <span class="source-path">evo-design/proto-tools<span class="source-subpath">/proto\_tools/tools/causal\_models/progen3</span></span>
  </div>

  <span class="panel-goto-btn source-goto-btn"><span><svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor"><path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" /></svg> View source</span></span>
</a>

<a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen3/examples/example.ipynb" target="_blank" class="tab-panel notebook-panel" data-tab="notebook-progen3">
  <div class="notebook-info">
    <span class="notebook-icon">
      <svg width="40" height="40" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round">
        <path d="M2 3h6a4 4 0 0 1 4 4v14a3 3 0 0 0-3-3H2z" />

        <path d="M22 3h-6a4 4 0 0 0-4 4v14a3 3 0 0 1 3-3h7z" />
      </svg>
    </span>

    <span class="notebook-label">Open Notebook</span>
  </div>

  <span class="panel-goto-btn notebook-goto-btn"><span><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2 3h6a4 4 0 0 1 4 4v14a3 3 0 0 0-3-3H2z" /><path d="M22 3h-6a4 4 0 0 0-4 4v14a3 3 0 0 1 3-3h7z" /></svg> Open notebook</span></span>
</a>

<div class="tab-panel proto-panel run-local-panel" data-tab="proto-progen3">
  <a href="https://github.com/evo-design/proto-tools" target="_blank" class="run-local-preview">
    <img noZoom src="https://opengraph.githubassets.com/1/evo-design/proto-tools" alt="proto-tools on GitHub" />
  </a>

  <div class="run-local-install">
    <span class="run-local-label">Run locally with proto-tools</span>

    <div class="run-local-code">
      ```bash theme={null}
      pip install git+https://github.com/evo-design/proto-tools.git
      ```
    </div>
  </div>
</div>

<div class="entity-contributors"><span class="entity-contributors-label">Toolkit contributors</span><span class="entity-contributors-people"><a class="entity-contributor" href="https://github.com/bviggiano" target="_blank" rel="noopener" title="bviggiano: 25 commits"><img noZoom class="entity-contributor-avatar" src="https://avatars.githubusercontent.com/u/21143637?v=4&s=64" alt="" loading="lazy" /><span class="entity-contributor-login">bviggiano</span></a><a class="entity-contributor" href="https://github.com/dguo8412" target="_blank" rel="noopener" title="dguo8412: 17 commits"><img noZoom class="entity-contributor-avatar" src="https://avatars.githubusercontent.com/u/46211285?v=4&s=64" alt="" loading="lazy" /><span class="entity-contributor-login">dguo8412</span></a><a class="entity-contributor" href="https://github.com/leba01" target="_blank" rel="noopener" title="leba01: 6 commits"><img noZoom class="entity-contributor-avatar" src="https://avatars.githubusercontent.com/u/124846286?v=4&s=64" alt="" loading="lazy" /><span class="entity-contributor-login">leba01</span></a></span></div>

| Function               | Description                                                 |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| ---------------------- | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `run_progen3_sample()` | Sample protein sequences using ProGen3 language model (GPU) | <a href="#api-run-progen3-sample" class="func-table-btn func-api-btn"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M4 19.5v-15A2.5 2.5 0 0 1 6.5 2H19a1 1 0 0 1 1 1v18a1 1 0 0 1-1 1H6.5a1 1 0 0 1 0-5H20" /></svg> Docs</a> <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen3/progen3_sample.py#L137" target="_blank" class="func-table-btn func-source-btn"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a> |
| `run_progen3_score()`  | Score protein sequences using ProGen3 language model (GPU)  | <a href="#api-run-progen3-score" class="func-table-btn func-api-btn"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M4 19.5v-15A2.5 2.5 0 0 1 6.5 2H19a1 1 0 0 1 1 1v18a1 1 0 0 1-1 1H6.5a1 1 0 0 1 0-5H20" /></svg> Docs</a> <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen3/progen3_score.py#L99" target="_blank" class="func-table-btn func-source-btn"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>    |

## Background

ProGen3 ([Bhatnagar et al., 2025](https://doi.org/10.1101/2025.04.15.649055)) is a family of generative protein language models from Profluent. ProGen3 models employ a sparse mixture-of-experts (MoE) architecture, which routes model activations in the transformer feed-forward layers to smaller specialized MLPs to make each forward pass more computationally tractable. The published family spans 112 million to 46 billion parameters; this toolkit exposes the `progen3-112m` through `progen3-3b` checkpoints. Pre-training used roughly 1.5 trillion amino-acid tokens sampled from the Profluent Protein Atlas, a curated collection of full-length natural proteins.

Unlike a strictly left-to-right model, ProGen3 is trained autoregressively in **both directions**: forward predicts each residue from the N-terminus toward the C-terminus, and reverse predicts from the C-terminus toward the N-terminus. Generation runs in a chosen direction, and scoring combines both directions into a single per-residue likelihood. Two capabilities follow from this objective. Sampling from the predicted next-residue distributions produces new candidate protein sequences, and the likelihood the model assigns to an existing sequence provides a zero-shot proxy-fitness score with no additional task-specific training.

### Learning Resources

* [ProGen3 showcase](https://www.profluent.bio/showcase/progen3) (Profluent) - an accessible overview of ProGen3, the Profluent Protein Atlas training data, and downstream applications such as antibody design and compact gene editors.

## Tools

<a name="api-run-progen3-sample" />

<div class="tool-section-card tool-section-card--sample">
  ### ProGen3 Sampling (`progen3-sample`)

  Generates protein sequences by autoregressive sampling. Given one or more prompt sequences, the model extends each prompt one amino acid at a time, drawing each residue from the model's predicted distribution under the configured `temperature` and `top_p` settings, in the chosen `direction`, until `max_new_tokens` residues have been generated (at least `min_new_tokens`).

  #### API Reference

  <div class="api-model-section api-input-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/shared_data_models.py#L226" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Input: CausalModelSampleInput">
      <ParamField path="prompts" type="List[string]" required>
        Prompt sequences to condition generation on. Can be provided as a single string or a list of strings.
      </ParamField>
    </Accordion>
  </div>

  <div class="api-model-section api-config-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen3/progen3_sample.py#L42" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Config: ProGen3SampleConfig">
      <ParamField path="model_checkpoint" type="enum" default="progen3-762m">
        ProGen3 weights variant. Sizes range from 112M (fastest) to 3B (highest quality).

        Available options: `progen3-112m`, `progen3-219m`, `progen3-339m`, `progen3-762m`, `progen3-1b`, `progen3-3b`
      </ParamField>

      <ParamField path="local_path" type="string">
        Override HuggingFace download with a local weights directory.
      </ParamField>

      <ParamField path="direction" type="enum" default="forward">
        `"forward"` generates N→C, `"reverse"` generates C→N.

        Available options: `forward`, `reverse`
      </ParamField>

      <ParamField path="max_new_tokens" type="integer" default="256">
        Maximum new tokens to generate per prompt (excludes prompt).
      </ParamField>

      <ParamField path="min_new_tokens" type="integer" default="1">
        Minimum new tokens to generate per prompt before stopping is allowed.
      </ParamField>

      <ParamField path="verbose" type="integer" default="0">
        Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). `True` is coerced to `1` and `False` to `0`.
      </ParamField>

      <ParamField path="device" type="string" default="cuda">
        Device to run the model on.
      </ParamField>

      <ParamField path="timeout" type="integer" default="3600">
        Maximum execution time in seconds. `None` waits indefinitely.
      </ParamField>

      <ParamField path="seed" type="integer">
        Random seed. When set, tools run reproducibly up to small GPU float noise (see `BaseToolOutput.approx_equal`), and the seed participates in cache keys. When None, cacheable seed-sensitive tools skip cache until seeded.
      </ParamField>

      <ParamField path="prepend_prompt" type="boolean" default="True">
        If `True`, returned sequences include the prompt and newly generated residues; if `False`, only the newly generated residues.
      </ParamField>

      <ParamField path="temperature" type="number" default="0.2">
        Softmax temperature; lower values are more deterministic, higher values increase diversity.
      </ParamField>

      <ParamField path="top_p" type="number" default="0.95">
        Nucleus sampling threshold over per-position token probabilities.
      </ParamField>

      <ParamField path="batch_size" type="integer" default="8">
        Maximum number of same-length prompts to process simultaneously on GPU.
      </ParamField>
    </Accordion>
  </div>

  <div class="api-model-section api-output-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/shared_data_models.py#L314" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Output: CausalModelSampleOutput">
      <ResponseField name="results" type="List[CausalModelSample]" required>
        One entry per generated sequence, in input order.

        <Expandable title="CausalModelSample">
          <ResponseField name="sequence" type="string" required>
            The generated sequence.
          </ResponseField>
        </Expandable>
      </ResponseField>
    </Accordion>
  </div>

  #### Applications

  This tool performs de novo protein design, generating novel sequences that resemble natural proteins, optionally conditioned on a prompt. Because generation can run in reverse (C-terminus toward N-terminus), a C-terminal fragment can be used as the prompt and the rest of the sequence grown toward the N-terminus, which a strictly left-to-right model cannot do.

  #### Usage Tips

  * **`direction` chooses which terminus is generated.** `"forward"` (the default) continues a prompt from the N-terminus toward the C-terminus; `"reverse"` treats the prompt as a C-terminal fragment and generates toward the N-terminus. Note that the reverse generation will append to the prompt to grow the sequence on the left. All starting sequences should still be provided in the left to right direction from N->C.
  * **Sampling defaults are conservative.** `temperature` defaults to `0.2` and `top_p` to `0.95`, which keep generations close to natural-looking sequences; raise `temperature` for more diverse but riskier designs. This tool exposes only nucleus (`top_p`) sampling for ProGen3; there is no top-k cutoff.
  * **`max_new_tokens` and `min_new_tokens` bound the generated length.** They count only newly generated residues (default `256` and `1`), separate from the prompt length.
  * **Output includes the prompt by default.** `prepend_prompt=True` (the toolkit default) returns the prompt joined to its continuation; set it `False` to receive only the newly generated residues.
  * **Generated sequences are candidates.** Validate them with downstream tools (for example structure prediction, function annotation, or homology search) before drawing biological conclusions.

  <a name="api-run-progen3-score" />
</div>

<div class="tool-section-card tool-section-card--score">
  ### ProGen3 Scoring (`progen3-score`)

  Scores existing protein sequences under ProGen3 using bidirectional likelihood. For each sequence it runs both a forward (N→C) and a reverse (C→N) pass, averages the per-position log-likelihoods into a single bidirectional value, and aggregates these into a log-likelihood, an average log-likelihood per residue, and a perplexity. It also exposes the forward, reverse, and bidirectional per-position values, and optionally the per-position logits.

  #### API Reference

  <div class="api-model-section api-input-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/shared_data_models.py#L24" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Input: CausalModelScoringInput">
      <ParamField path="sequences" type="List[string]" required>
        Sequences to score. Can be provided as a single string or a list of strings.
      </ParamField>
    </Accordion>
  </div>

  <div class="api-model-section api-config-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen3/progen3_score.py#L41" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Config: ProGen3ScoringConfig">
      <ParamField path="model_checkpoint" type="enum" default="progen3-762m">
        ProGen3 weights variant. Sizes range from 112M to 3B parameters.

        Available options: `progen3-112m`, `progen3-219m`, `progen3-339m`, `progen3-762m`, `progen3-1b`, `progen3-3b`
      </ParamField>

      <ParamField path="local_path" type="string">
        Override HuggingFace download with a local weights directory.
      </ParamField>

      <ParamField path="verbose" type="integer" default="0">
        Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). `True` is coerced to `1` and `False` to `0`.
      </ParamField>

      <ParamField path="device" type="string" default="cuda">
        Device to run the model on.
      </ParamField>

      <ParamField path="timeout" type="integer" default="3600">
        Maximum execution time in seconds. `None` waits indefinitely.
      </ParamField>

      <ParamField path="seed" type="integer">
        Random seed. When set, tools run reproducibly up to small GPU float noise (see `BaseToolOutput.approx_equal`), and the seed participates in cache keys. When None, cacheable seed-sensitive tools skip cache until seeded.
      </ParamField>

      <ParamField path="batch_size" type="integer" default="8">
        Number of sequences to process simultaneously on GPU.
      </ParamField>

      <ParamField path="return_logits" type="boolean" default="False">
        Whether to include forward-pass per-position logits in the output. Reverse-pass info is already exposed via `per_position_metrics` (forward/reverse/bidirectional log-likelihoods).
      </ParamField>
    </Accordion>
  </div>

  <div class="api-model-section api-output-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/shared_data_models.py#L148" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Output: CausalModelScoringOutput">
      <ResponseField name="scores" type="List[CausalModelScoringMetrics]" required>
        List of scoring outputs, one per input sequence. Each entry is a `Metrics` subclass with scalar metrics (`log_likelihood`, `avg_log_likelihood`, `perplexity`) and optional per-position `_pp`-suffixed list extras; `logits` and `vocab` are declared fields for raw model outputs.

        <Expandable title="CausalModelScoringMetrics">
          <ResponseField name="logits" type="array">
            Per-position logits array `(seq_len, vocab_size)`. `None` unless `return_logits=True`.
          </ResponseField>

          <ResponseField name="vocab" type="array">
            Token ordering for `logits`.
          </ResponseField>

          <ResponseField name="primary_metric" type="string">
            Name of the metric that best summarizes the result overall (e.g. `"avg_plddt"` for AlphaFold2). Used by downstream UI and reporting to pick a headline value.
          </ResponseField>

          <ResponseField name="metric_type" type="string">
            Concrete Metrics subclass tag; enables typed reconstruction after a serialization round-trip.
          </ResponseField>
        </Expandable>
      </ResponseField>

      **Metrics** (one set per `scores` item)

      | Metric               | Type  | Range | Availability |
      | -------------------- | ----- | ----- | ------------ |
      | `log_likelihood`     | float | ≤ 0.0 | always       |
      | `avg_log_likelihood` | float | ≤ 0.0 | always       |
      | `perplexity`         | float | ≥ 1.0 | always       |
    </Accordion>
  </div>

  #### Applications

  This tool gives a zero-shot measure of how consistent a protein sequence is with ProGen3's training distribution, usable as a fitness or plausibility signal without additional task-specific training. Because it uses both directions, every residue is scored with full surrounding context rather than left context only. Use it to rank or filter candidate sequences (including the output of `progen3-sample`), to compare variants of a sequence, or to flag sequences far from the model's training distribution.

  #### Usage Tips

  * **Scores are bidirectional, not a single-direction log-likelihood.** The reported `log_likelihood`, `avg_log_likelihood`, and `perplexity` are derived from the averaged forward and reverse per-position values, so they are not directly comparable to a one-directional model's scores.
  * **Compare length-normalized scores within one checkpoint.** Total `log_likelihood` scales with sequence length, so use `perplexity` or `avg_log_likelihood` when comparing sequences of different lengths. Different checkpoints learn different distributions that are not calibrated to a common scale, so scores from different `model_checkpoint` values are hard to compare directly; a lower perplexity means the sequence is more consistent with that checkpoint's training distribution.
  * **`return_logits` defaults to `False`.** Leave it off unless you need the per-position distributions, since the logits tensor is large (sequence length by the token vocabulary).
</div>

## Toolkit Notes

These apply to every ProGen3 tool in this toolkit (`progen3-sample`, `progen3-score`).

* **Requires a GPU; memory scales with checkpoint size.** This toolkit exposes the `progen3-112m` through `progen3-3b` checkpoints; larger checkpoints are more capable but need substantially more GPU memory. CPU execution is not practical.
* **`batch_size` trades memory for throughput across both tools.** It sets how many same-length prompts (`progen3-sample`) or sequences (`progen3-score`) are processed per GPU forward pass. Raise it for higher throughput on many short sequences; lower it (default `1`) if generation or scoring runs out of GPU memory.
* **`model_checkpoint` selects the model size.** The default is `progen3-762m`; smaller checkpoints (`progen3-112m`, `progen3-219m`, `progen3-339m`) are faster and lighter, while `progen3-1b` and `progen3-3b` are more capable at higher memory cost.

<Tip>
  **Example notebook:** See the [full working example](https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen3/examples/example.ipynb) for a copy-paste-ready walkthrough.
</Tip>

## Infrastructure Guides

The following guides cover how to run tools efficiently and at scale.

<CardGroup cols={2}>
  <Card title="Tool Persistence" icon="repeat" href="/docs/tools/guides/tool-persistence">Keep a tool's model warm across calls instead of reloading it every invocation.</Card>
  <Card title="Device Management" icon="cpu" href="/docs/tools/guides/device-management">How GPUs are allocated to tools and how to target specific devices.</Card>
  <Card title="Parallel Execution" icon="layers" href="/docs/tools/guides/parallel-execution">Fan a batch of inputs out across multiple GPUs.</Card>
</CardGroup>
