> ## Documentation Index
> Fetch the complete documentation index at: https://proto.evodesign.org/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# ProGen2

> ProGen2 is an autoregressive protein language model from Salesforce Research, first released in 2022 and published in 2023, trained on natural protein sequences from genomic, metagenomic, and immune-repertoire databases.

<div class="page-hero"><img class="page-hero-banner" src="https://proto-bio.github.io/proto-assets/images/tool/progen2/hero.png" alt="ProGen2" /><div class="tool-org-badges page-hero-badges"><a href="/docs/tools/organizations/salesforce-research" class="tool-org-badge" style={{background: "#00A1E0"}} title="Salesforce Research"><img src="https://mintcdn.com/bio-pro/_UGa2jUMKeVPCbLk/assets/images/cached/8968678dd27c.png?fit=max&auto=format&n=_UGa2jUMKeVPCbLk&q=85&s=90ccf0cdbec4208b08905f5e12d0ebd6" alt="" class="tool-org-badge-logo" width="187" height="187" data-path="assets/images/cached/8968678dd27c.png" /> Salesforce Research</a></div></div>

<Note>
  **License:** ProGen2 is open source and free for academic and commercial use under a BSD-3-Clause license. Please refer to [the license](https://github.com/enijkamp/progen2/blob/main/LICENSE.txt) for full terms.
</Note>

<p class="entity-disclaimer">Proto is not affiliated with Salesforce Research. This toolkit is open source and builds on the implementation produced by this organization. Product names, logos, and trademarks are the property of their respective owners.</p>

<hr class="entity-rule" />

<input type="radio" name="tab-progen2" id="none-progen2" class="tab-radio-input" />

<input type="radio" name="tab-progen2" id="github-progen2" class="tab-radio-input" defaultChecked />

<input type="radio" name="tab-progen2" id="hf-progen2" class="tab-radio-input" />

<input type="radio" name="tab-progen2" id="paper-progen2" class="tab-radio-input" />

<input type="radio" name="tab-progen2" id="cite-progen2" class="tab-radio-input" />

<input type="radio" name="tab-progen2" id="source-progen2" class="tab-radio-input" />

<input type="radio" name="tab-progen2" id="notebook-progen2" class="tab-radio-input" />

<input type="radio" name="tab-progen2" id="proto-progen2" class="tab-radio-input" />

<div class="tool-tab-bar">
  <span class="tool-tab-wrap"><label for="github-progen2" class="tool-tab tab-open badge-github"><svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor"><path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" /></svg> GitHub</label><label for="none-progen2" class="tool-tab tab-close badge-github"><svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor"><path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" /></svg> GitHub</label></span> <span class="tool-tab-wrap"><label for="hf-progen2" class="tool-tab tab-open badge-hf"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> HuggingFace</label><label for="none-progen2" class="tool-tab tab-close badge-hf"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> HuggingFace</label></span> <span class="tool-tab-wrap"><label for="paper-progen2" class="tool-tab tab-open badge-paper"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z" /><polyline points="14 2 14 8 20 8" /><line x1="16" y1="13" x2="8" y2="13" /><line x1="16" y1="17" x2="8" y2="17" /><polyline points="10 9 9 9 8 9" /></svg> Publication</label><label for="none-progen2" class="tool-tab tab-close badge-paper"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z" /><polyline points="14 2 14 8 20 8" /><line x1="16" y1="13" x2="8" y2="13" /><line x1="16" y1="17" x2="8" y2="17" /><polyline points="10 9 9 9 8 9" /></svg> Publication</label></span> <span class="tool-tab-wrap"><label for="cite-progen2" class="tool-tab tab-open badge-cite"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M3 21c3 0 7-1 7-8V5c0-1.25-.756-2.017-2-2H4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2 1 0 1 0 1 1v1c0 1-1 2-2 2s-1 .008-1 1.031V20c0 1 0 1 1 1z" /><path d="M15 21c3 0 7-1 7-8V5c0-1.25-.757-2.017-2-2h-4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2h.75c0 2.25.25 4-2.75 4v3c0 1 0 1 1 1z" /></svg> Cite</label><label for="none-progen2" class="tool-tab tab-close badge-cite"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M3 21c3 0 7-1 7-8V5c0-1.25-.756-2.017-2-2H4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2 1 0 1 0 1 1v1c0 1-1 2-2 2s-1 .008-1 1.031V20c0 1 0 1 1 1z" /><path d="M15 21c3 0 7-1 7-8V5c0-1.25-.757-2.017-2-2h-4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2h.75c0 2.25.25 4-2.75 4v3c0 1 0 1 1 1z" /></svg> Cite</label></span> <span class="tool-tab-wrap"><label for="source-progen2" class="tool-tab tab-open badge-source"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Tool Source</label><label for="none-progen2" class="tool-tab tab-close badge-source"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Tool Source</label></span> <span class="tool-tab-wrap"><label for="notebook-progen2" class="tool-tab tab-open badge-notebook"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2 3h6a4 4 0 0 1 4 4v14a3 3 0 0 0-3-3H2z" /><path d="M22 3h-6a4 4 0 0 0-4 4v14a3 3 0 0 1 3-3h7z" /></svg> Open as Notebook</label><label for="none-progen2" class="tool-tab tab-close badge-notebook"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2 3h6a4 4 0 0 1 4 4v14a3 3 0 0 0-3-3H2z" /><path d="M22 3h-6a4 4 0 0 0-4 4v14a3 3 0 0 1 3-3h7z" /></svg> Open as Notebook</label></span> <span class="tool-tab-wrap"><label for="proto-progen2" class="tool-tab tab-open badge-proto"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M13 2L3 14h9l-1 8 10-12h-9l1-8z" /></svg> Open on Proto</label><label for="none-progen2" class="tool-tab tab-close badge-proto"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M13 2L3 14h9l-1 8 10-12h-9l1-8z" /></svg> Open on Proto</label></span>
</div>

<a href="https://github.com/enijkamp/progen2" target="_blank" class="tab-panel github-panel" data-tab="github-progen2">
  <div class="gh-card-wrap">
    <img src="https://opengraph.githubassets.com/1/enijkamp/progen2" class="gh-card-img img-fallback" alt="enijkamp/progen2" />

    <div class="gh-card-fallback">
      <div class="gh-fallback-org"><svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor"><path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" /></svg> enijkamp/progen2</div>
    </div>
  </div>

  <span class="panel-goto-btn gh-goto-btn"><span><svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor"><path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" /></svg> View repo</span></span>
</a>

<div class="tab-panel hf-panel hf-multi-panel" data-tab="hf-progen2">
  <div class="hf-model-list">
    <a href="https://huggingface.co/hugohrban/progen2-small" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/hugohrban/progen2-small.png" class="hf-model-card-bg img-fallback" alt="progen2-small" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen2-small</span>
    </a>

    <a href="https://huggingface.co/hugohrban/progen2-medium" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/hugohrban/progen2-medium.png" class="hf-model-card-bg img-fallback" alt="progen2-medium" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen2-medium</span>
    </a>

    <a href="https://huggingface.co/hugohrban/progen2-large" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/hugohrban/progen2-large.png" class="hf-model-card-bg img-fallback" alt="progen2-large" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen2-large</span>
    </a>

    <a href="https://huggingface.co/hugohrban/progen2-xlarge" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/hugohrban/progen2-xlarge.png" class="hf-model-card-bg img-fallback" alt="progen2-xlarge" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen2-xlarge</span>
    </a>

    <a href="https://huggingface.co/hugohrban/progen2-oas" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/hugohrban/progen2-oas.png" class="hf-model-card-bg img-fallback" alt="progen2-oas" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen2-oas</span>
    </a>

    <a href="https://huggingface.co/hugohrban/progen2-BFD90" target="_blank" class="hf-model-card">
      <img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/models/hugohrban/progen2-BFD90.png" class="hf-model-card-bg img-fallback" alt="progen2-BFD90" />

      <span class="hf-model-card-label"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="16" height="16" class="hf-logo" /> progen2-BFD90</span>
    </a>
  </div>
</div>

<a href="https://doi.org/10.1016/j.cels.2023.10.002" target="_blank" class="tab-panel paper-panel" data-tab="paper-progen2">
  <div class="paper-info">
    <div class="paper-title">ProGen2: Exploring the boundaries of protein language models</div>
    <div class="paper-meta">Erik Nijkamp, Jeffrey A Ruffolo, ... Ali Madani</div>
    <div class="paper-meta paper-venue">Cell Systems (2023)</div>
  </div>

  <span class="panel-goto-btn pub-goto-btn"><span><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z" /><polyline points="14 2 14 8 20 8" /><line x1="16" y1="13" x2="8" y2="13" /><line x1="16" y1="17" x2="8" y2="17" /><polyline points="10 9 9 9 8 9" /></svg> Read paper</span></span>
</a>

<div class="tab-panel cite-panel" data-tab="cite-progen2">
  <div class="cite-code-wrap">
    ```bibtex theme={null}
    @article{nijkamp2023progen2,
      title={ProGen2: Exploring the boundaries of protein language models},
      author={Nijkamp, Erik and Ruffolo, Jeffrey A and Weinstein, Eli N and Naik, Nikhil and Madani, Ali},
      journal={Cell Systems},
      volume={14},
      number={11},
      pages={968--978},
      year={2023},
      publisher={Elsevier},
      doi={10.1016/j.cels.2023.10.002}
    }
    ```
  </div>

  <span class="panel-goto-btn cite-copy-btn"><span><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M3 21c3 0 7-1 7-8V5c0-1.25-.756-2.017-2-2H4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2 1 0 1 0 1 1v1c0 1-1 2-2 2s-1 .008-1 1.031V20c0 1 0 1 1 1z" /><path d="M15 21c3 0 7-1 7-8V5c0-1.25-.757-2.017-2-2h-4c-1.25 0-2 .75-2 1.972V11c0 1.25.75 2 2 2h.75c0 2.25.25 4-2.75 4v3c0 1 0 1 1 1z" /></svg> Copy citation</span></span>
</div>

<a href="https://github.com/evo-design/proto-tools/tree/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen2" target="_blank" class="tab-panel source-panel" data-tab="source-progen2">
  <div class="source-info">
    <img src="https://github.com/evo-design.png?size=40" class="source-avatar" width="36" height="36" />

    <span class="source-path">evo-design/proto-tools<span class="source-subpath">/proto\_tools/tools/causal\_models/progen2</span></span>
  </div>

  <span class="panel-goto-btn source-goto-btn"><span><svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor"><path d="M12 0C5.37 0 0 5.37 0 12c0 5.31 3.435 9.795 8.205 11.385.6.105.825-.255.825-.57 0-.285-.015-1.23-.015-2.235-3.015.555-3.795-.735-4.035-1.41-.135-.345-.72-1.41-1.23-1.695-.42-.225-1.02-.78-.015-.795.945-.015 1.62.87 1.845 1.23 1.08 1.815 2.805 1.305 3.495.99.105-.78.42-1.305.765-1.605-2.67-.3-5.46-1.335-5.46-5.925 0-1.305.465-2.385 1.23-3.225-.12-.3-.54-1.53.12-3.18 0 0 1.005-.315 3.3 1.23.96-.27 1.98-.405 3-.405s2.04.135 3 .405c2.295-1.56 3.3-1.23 3.3-1.23.66 1.65.24 2.88.12 3.18.765.84 1.23 1.905 1.23 3.225 0 4.605-2.805 5.625-5.475 5.925.435.375.81 1.095.81 2.22 0 1.605-.015 2.895-.015 3.3 0 .315.225.69.825.57A12.02 12.02 0 0024 12c0-6.63-5.37-12-12-12z" /></svg> View source</span></span>
</a>

<a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen2/examples/example.ipynb" target="_blank" class="tab-panel notebook-panel" data-tab="notebook-progen2">
  <div class="notebook-info">
    <span class="notebook-icon">
      <svg width="40" height="40" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round">
        <path d="M2 3h6a4 4 0 0 1 4 4v14a3 3 0 0 0-3-3H2z" />

        <path d="M22 3h-6a4 4 0 0 0-4 4v14a3 3 0 0 1 3-3h7z" />
      </svg>
    </span>

    <span class="notebook-label">Open Notebook</span>
  </div>

  <span class="panel-goto-btn notebook-goto-btn"><span><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2 3h6a4 4 0 0 1 4 4v14a3 3 0 0 0-3-3H2z" /><path d="M22 3h-6a4 4 0 0 0-4 4v14a3 3 0 0 1 3-3h7z" /></svg> Open notebook</span></span>
</a>

<div class="tab-panel proto-panel" data-tab="proto-progen2">
  <div class="proto-info">
    <div class="proto-cloud">
      <svg class="proto-cloud-bg" viewBox="0 0 640 512" xmlns="http://www.w3.org/2000/svg">
        <path d="M0 336c0 79.5 64.5 144 144 144H512c70.7 0 128-57.3 128-128c0-61.9-44-113.6-102.4-125.4c4.1-10.7 6.4-22.4 6.4-34.6c0-53-43-96-96-96c-19.7 0-38.1 6-53.3 16.2C367 64.2 315.3 32 256 32C167.6 32 96 103.6 96 192c0 2.7 .1 5.4 .2 8.1C40.2 219.8 0 273.2 0 336z" />
      </svg>

      <img noZoom src="https://mintcdn.com/bio-pro/KVh0EKV-IKblvXR8/assets/logo/evo-logo-light.svg?fit=max&auto=format&n=KVh0EKV-IKblvXR8&q=85&s=0cb66034ba45618505501aee6ea5f5c1" class="proto-panel-logo block dark:hidden" alt="Proto" width="198" height="151" data-path="assets/logo/evo-logo-light.svg" />

      <img noZoom src="https://mintcdn.com/bio-pro/KVh0EKV-IKblvXR8/assets/logo/evo-logo-dark.svg?fit=max&auto=format&n=KVh0EKV-IKblvXR8&q=85&s=2c9e23a14635e60384a434e220788f54" class="proto-panel-logo hidden dark:block" alt="Proto" width="198" height="151" data-path="assets/logo/evo-logo-dark.svg" />
    </div>
  </div>

  <div class="proto-actions">
    <a href="https://proto.evodesign.org/tools/progen2-sample" target="_blank" class="proto-action-btn"><span>ProGen2 Sampling</span><svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="7" y1="17" x2="17" y2="7" /><polyline points="7 7 17 7 17 17" /></svg></a>
    <a href="https://proto.evodesign.org/tools/progen2-score" target="_blank" class="proto-action-btn"><span>ProGen2 Scoring</span><svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="7" y1="17" x2="17" y2="7" /><polyline points="7 7 17 7 17 17" /></svg></a>
  </div>
</div>

<div class="entity-contributors"><span class="entity-contributors-label">Toolkit contributors</span><span class="entity-contributors-people"><a class="entity-contributor" href="https://github.com/bviggiano" target="_blank" rel="noopener" title="bviggiano: 26 commits"><img noZoom class="entity-contributor-avatar" src="https://avatars.githubusercontent.com/u/21143637?v=4&s=64" alt="" loading="lazy" /><span class="entity-contributor-login">bviggiano</span></a><a class="entity-contributor" href="https://github.com/dguo8412" target="_blank" rel="noopener" title="dguo8412: 24 commits"><img noZoom class="entity-contributor-avatar" src="https://avatars.githubusercontent.com/u/46211285?v=4&s=64" alt="" loading="lazy" /><span class="entity-contributor-login">dguo8412</span></a><a class="entity-contributor" href="https://github.com/leba01" target="_blank" rel="noopener" title="leba01: 6 commits"><img noZoom class="entity-contributor-avatar" src="https://avatars.githubusercontent.com/u/124846286?v=4&s=64" alt="" loading="lazy" /><span class="entity-contributor-login">leba01</span></a><a class="entity-contributor" href="https://github.com/adititm" target="_blank" rel="noopener" title="adititm: 1 commit"><img noZoom class="entity-contributor-avatar" src="https://avatars.githubusercontent.com/u/61667248?v=4&s=64" alt="" loading="lazy" /><span class="entity-contributor-login">adititm</span></a></span></div>

| Function               | Description                                                 |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| ---------------------- | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `run_progen2_sample()` | Sample protein sequences using ProGen2 language model (GPU) | <a href="#api-run-progen2-sample" class="func-table-btn func-api-btn"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M4 19.5v-15A2.5 2.5 0 0 1 6.5 2H19a1 1 0 0 1 1 1v18a1 1 0 0 1-1 1H6.5a1 1 0 0 1 0-5H20" /></svg> Docs</a> <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen2/progen2_sample.py#L188" target="_blank" class="func-table-btn func-source-btn"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a> |
| `run_progen2_score()`  | Score protein sequences using ProGen2 language model (GPU)  | <a href="#api-run-progen2-score" class="func-table-btn func-api-btn"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M4 19.5v-15A2.5 2.5 0 0 1 6.5 2H19a1 1 0 0 1 1 1v18a1 1 0 0 1-1 1H6.5a1 1 0 0 1 0-5H20" /></svg> Docs</a> <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen2/progen2_score.py#L106" target="_blank" class="func-table-btn func-source-btn"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>   |

## Background

ProGen2 ([Nijkamp et al., 2023](https://doi.org/10.1016/j.cels.2023.10.002)) is a family of autoregressive protein language models trained with a next-token prediction objective: during training the model learns to predict the next residue given all preceding residues. The family spans `progen2-small` (151 million parameters) up to `progen2-xlarge` (6.4 billion parameters). The checkpoints were trained on different protein collections as a result of the paper's finding that the training-data distribution has a large and sometimes counterintuitive effect on downstream performance. Most checkpoints are trained on natural proteins drawn from [UniRef90](https://www.uniprot.org/help/uniref) and the [BFD](https://bfd.mmseqs.com/) metagenomic set; `progen2-BFD90` uses the BFD90 collection, and `progen2-oas` is trained on antibody sequences from the [Observed Antibody Space](https://opig.stats.ox.ac.uk/webapps/oas/) database.

The autoregressive training objective instills two primary capabilities. First, new candidate protein sequences can be sampled from a starting prompt via the predicted next-residue distributions. Second, the model can be used to score existing protein sequences, as the likelihood the model assigns to a sequence is shown in the paper to provide a proxy zero-shot fitness score or measure of plausibility with no additional task-specific training.

## Tools

<a name="api-run-progen2-sample" />

<div class="tool-section-card tool-section-card--sample">
  ### ProGen2 Sampling (`progen2-sample`)

  Generates protein sequences by autoregressive sampling. Given one or more prompt sequences, the model extends each prompt one amino acid at a time, drawing each residue from the model's predicted distribution under the configured `temperature`, `top_p`, and `top_k` settings, until a stop token is produced or `max_new_tokens` new residues have been generated (default 256).

  #### API Reference

  <div class="api-model-section api-input-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/shared_data_models.py#L226" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Input: CausalModelSampleInput">
      <ParamField path="prompts" type="List[string]" required>
        Prompt sequences to condition generation on. Can be provided as a single string or a list of strings.
      </ParamField>
    </Accordion>
  </div>

  <div class="api-model-section api-config-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen2/progen2_sample.py#L73" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Config: ProGen2SampleConfig">
      <ParamField path="model_checkpoint" type="enum" default="progen2-large">
        ProGen2 weights variant. Sizes range from 151M (small) to 6B (xlarge).

        Available options: `progen2-small`, `progen2-medium`, `progen2-base`, `progen2-oas`, `progen2-large`, `progen2-BFD90`, `progen2-xlarge`
      </ParamField>

      <ParamField path="local_path" type="string">
        Override the default download with a local weights directory.
      </ParamField>

      <ParamField path="top_k" type="integer" default="0">
        Top-k truncation; `0` disables and uses top-p only.
      </ParamField>

      <ParamField path="max_new_tokens" type="integer" default="256">
        Maximum number of new tokens to generate per prompt (excludes prompt).
      </ParamField>

      <ParamField path="truncate_at_stop" type="boolean" default="True">
        Truncate generated sequences at the first stop token.
      </ParamField>

      <ParamField path="strip_special_tokens" type="boolean" default="True">
        Strip ProGen2 start/stop sentinel tokens (`1`/`2`) from output.
      </ParamField>

      <ParamField path="return_logits" type="boolean" default="False">
        Include per-position logits in the output.
      </ParamField>

      <ParamField path="verbose" type="integer" default="0">
        Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). `True` is coerced to `1` and `False` to `0`.
      </ParamField>

      <ParamField path="device" type="string" default="cuda">
        Device to run the model on.
      </ParamField>

      <ParamField path="timeout" type="integer" default="3600">
        Maximum execution time in seconds. `None` waits indefinitely.
      </ParamField>

      <ParamField path="seed" type="integer">
        Random seed. When set, tools run reproducibly up to small GPU float noise (see `BaseToolOutput.approx_equal`), and the seed participates in cache keys. When None, cacheable seed-sensitive tools skip cache until seeded.
      </ParamField>

      <ParamField path="prepend_prompt" type="boolean" default="True">
        Include the input prompt at the start of each generated sequence; when `False`, only newly generated tokens are returned.
      </ParamField>

      <ParamField path="temperature" type="number" default="0.2">
        Softmax temperature; lower values are more deterministic.
      </ParamField>

      <ParamField path="top_p" type="number" default="0.95">
        Nucleus sampling threshold over per-position token probabilities.
      </ParamField>

      <ParamField path="batch_size" type="integer" default="8">
        Number of prompts to process simultaneously on GPU.
      </ParamField>
    </Accordion>
  </div>

  <div class="api-model-section api-output-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen2/progen2_sample.py#L59" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Output: ProGen2SampleOutput">
      <ResponseField name="results" type="List[ProGen2Sample]" required>
        One generated protein sequence per prompt, with its logits.

        <Expandable title="ProGen2Sample">
          <ResponseField name="logits" type="array">
            Per-position logits for this sequence (shape: \[generated\_len, vocab\_size]).
          </ResponseField>

          <ResponseField name="sequence" type="string" required>
            The generated protein sequence.
          </ResponseField>
        </Expandable>
      </ResponseField>
    </Accordion>
  </div>

  #### Applications

  This tool performs de novo protein design, generating novel sequences that resemble natural proteins conditioned on a prompt such as a starting motif or partial domain. The antibody-trained `progen2-oas` checkpoint targets antibody and immune-repertoire generation specifically.

  #### Usage Tips

  * **Generated output is trimmed by default.** Generated sequences are cut at the first stop token with the start/stop sentinels removed (`truncate_at_stop` and `strip_special_tokens`, both `True`); set them `False` to keep the raw model output.
  * **Sampling defaults are conservative.** `temperature` defaults to `0.2` and `top_p` to `0.95`, which keep generations close to natural-looking sequences; raise `temperature` for more diverse but riskier designs. `top_k` defaults to `0`, which disables top-k truncation so only nucleus (`top_p`) sampling is applied.
  * **`max_new_tokens` bounds the generated length.** It caps newly generated residues (default `256`), separate from the prompt length.
  * **Output includes the prompt by default.** `prepend_prompt=True` (the toolkit default) returns the prompt joined to its continuation; set it `False` to receive only the newly generated residues.
  * **Generated sequences are candidates.** Validate them with downstream tools (for example structure prediction, function annotation, or homology search) before drawing biological conclusions.

  <a name="api-run-progen2-score" />
</div>

<div class="tool-section-card tool-section-card--score">
  ### ProGen2 Scoring (`progen2-score`)

  Scores existing protein sequences using ProGen2. For each sequence it computes the model's predicted probability of every residue given the preceding residues and aggregates these into a log-likelihood, an average log-likelihood per residue, and a perplexity (perplexity is fully determined by the average log-likelihood, computed as `exp(-avg_log_likelihood)`, but is the conventionally reported metric). Optionally returns the per-position logits and the token vocabulary.

  #### API Reference

  <div class="api-model-section api-input-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/shared_data_models.py#L24" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Input: CausalModelScoringInput">
      <ParamField path="sequences" type="List[string]" required>
        Sequences to score. Can be provided as a single string or a list of strings.
      </ParamField>
    </Accordion>
  </div>

  <div class="api-model-section api-config-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen2/progen2_score.py#L46" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Config: ProGen2ScoringConfig">
      <ParamField path="model_checkpoint" type="enum" default="progen2-large">
        ProGen2 weights variant.

        Available options: `progen2-small`, `progen2-medium`, `progen2-base`, `progen2-oas`, `progen2-large`, `progen2-BFD90`, `progen2-xlarge`
      </ParamField>

      <ParamField path="local_path" type="string">
        Override the default download with a local weights directory.
      </ParamField>

      <ParamField path="verbose" type="integer" default="0">
        Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). `True` is coerced to `1` and `False` to `0`.
      </ParamField>

      <ParamField path="device" type="string" default="cuda">
        Device to run the model on.
      </ParamField>

      <ParamField path="timeout" type="integer" default="3600">
        Maximum execution time in seconds. `None` waits indefinitely.
      </ParamField>

      <ParamField path="seed" type="integer">
        Random seed. When set, tools run reproducibly up to small GPU float noise (see `BaseToolOutput.approx_equal`), and the seed participates in cache keys. When None, cacheable seed-sensitive tools skip cache until seeded.
      </ParamField>

      <ParamField path="batch_size" type="integer" default="8">
        Number of sequences to process simultaneously on GPU.
      </ParamField>

      <ParamField path="return_logits" type="boolean" default="False">
        Include per-position logits in the output.
      </ParamField>
    </Accordion>
  </div>

  <div class="api-model-section api-output-section">
    <a href="https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/shared_data_models.py#L148" target="_blank" class="func-table-btn func-source-btn api-model-source"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6" /><polyline points="8 6 2 12 8 18" /></svg> Source</a>

    <Accordion title="Output: CausalModelScoringOutput">
      <ResponseField name="scores" type="List[CausalModelScoringMetrics]" required>
        List of scoring outputs, one per input sequence. Each entry is a `Metrics` subclass with scalar metrics (`log_likelihood`, `avg_log_likelihood`, `perplexity`) and optional per-position `_pp`-suffixed list extras; `logits` and `vocab` are declared fields for raw model outputs.

        <Expandable title="CausalModelScoringMetrics">
          <ResponseField name="logits" type="array">
            Per-position logits array `(seq_len, vocab_size)`. `None` unless `return_logits=True`.
          </ResponseField>

          <ResponseField name="vocab" type="array">
            Token ordering for `logits`.
          </ResponseField>

          <ResponseField name="primary_metric" type="string">
            Name of the metric that best summarizes the result overall (e.g. `"avg_plddt"` for AlphaFold2). Used by downstream UI and reporting to pick a headline value.
          </ResponseField>

          <ResponseField name="metric_type" type="string">
            Concrete Metrics subclass tag; enables typed reconstruction after a serialization round-trip.
          </ResponseField>
        </Expandable>
      </ResponseField>

      **Metrics** (one set per `scores` item)

      | Metric               | Type  | Range | Availability |
      | -------------------- | ----- | ----- | ------------ |
      | `log_likelihood`     | float | ≤ 0.0 | always       |
      | `avg_log_likelihood` | float | ≤ 0.0 | always       |
      | `perplexity`         | float | ≥ 1.0 | always       |
    </Accordion>
  </div>

  #### Applications

  This tool gives a zero-shot measure of how consistent a protein sequence is with ProGen2's training distribution, which is used in the paper as a proxy-fitness predictor without additional task-specific training. It can be used to rank or filter candidate sequences (including the output of `progen2-sample`), to compare variants of a sequence, or to flag sequences far from the model's training distribution.

  #### Usage Tips

  * **Compare length-normalized scores within one checkpoint.** Total `log_likelihood` scales with sequence length, so use `perplexity` or `avg_log_likelihood` when comparing sequences of different lengths. Different checkpoints learn different distributions that are not calibrated to a common scale, so scores from different `model_checkpoint` values are hard to compare directly. A lower perplexity means the sequence is more consistent with that checkpoint's training distribution.
  * **`return_logits` defaults to `False`.** Leave it off unless you need the per-position distributions, since the logits tensor is large (sequence length by the token vocabulary).
  * **A domain-matched checkpoint is not automatically better for scoring.** The ProGen2 paper found the antibody-specific `progen2-oas` checkpoint underperformed the universal checkpoints on antibody fitness prediction, so a universal checkpoint (such as the default `progen2-large`) is often the safer choice for scoring.
</div>

## Toolkit Notes

These apply to every ProGen2 tool in this toolkit (`progen2-sample`, `progen2-score`).

* **Requires a GPU; memory scales with checkpoint size.** The larger checkpoints, up to `progen2-xlarge` at 6.4 billion parameters, need substantially more GPU memory than `progen2-small`. CPU execution is not practical.
* **`batch_size` trades memory for throughput across both tools.** It sets how many prompts (`progen2-sample`) or sequences (`progen2-score`) are processed per GPU forward pass. Raise it for higher throughput on many short sequences; lower it (default `1`) if generation or scoring runs out of GPU memory.
* **`model_checkpoint` selects the training distribution.** The default `progen2-large` and the `small`, `medium`, `base`, and `xlarge` checkpoints are trained on broad natural-protein collections (UniRef90 and BFD); `progen2-BFD90` is trained on the BFD90 set and `progen2-oas` on antibody sequences from the Observed Antibody Space. The choice of model has performance implications for both sampling and scoring.

<Tip>
  **Example notebook:** See the [full working example](https://github.com/evo-design/proto-tools/blob/47e34afa5ea240a3b406e323dc38aa5dc85f223e/proto_tools/tools/causal_models/progen2/examples/example.ipynb) for a copy-paste-ready walkthrough.
</Tip>

## Infrastructure Guides

The following guides cover how to run tools efficiently and at scale.

<CardGroup cols={2}>
  <Card title="Tool Persistence" icon="repeat" href="/docs/tools/guides/tool-persistence">Keep a tool's model warm across calls instead of reloading it every invocation.</Card>
  <Card title="Device Management" icon="cpu" href="/docs/tools/guides/device-management">How GPUs are allocated to tools and how to target specific devices.</Card>
  <Card title="Parallel Execution" icon="layers" href="/docs/tools/guides/parallel-execution">Fan a batch of inputs out across multiple GPUs.</Card>
</CardGroup>
