> ## Documentation Index
> Fetch the complete documentation index at: https://proto.evodesign.org/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# CodonFM (Encodon)
> CodonFM (model name Encodon) is a codon-level masked language model of protein-coding sequences from NVIDIA. Unlike nucleotide- or amino-acid-level models, Encodon tokenizes a coding sequence in-frame at codon resolution (one token per 3 nt), so it directly captures synonymous-codon usage and codon-context signals that are invisible to an amino-acid model. This toolkit exposes four Encodon checkpoints (80M, 600M, 1B, and a codon-frequency-aware-masking 1B variant) for coding-sequence fitness scoring, mutation-effect prediction, embeddings, and differentiable sequence design.
**License:** CodonFM (Encodon) uses Apache-2.0 for code and Custom (NVIDIA Open Model License) for model weights. Please refer to the [code license](https://github.com/NVIDIA-BioNeMo/CodonFM/blob/main/LICENSE) and [model weights license](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/) for full terms.
Proto is not affiliated with NVIDIA, Arc Institute, and University of California, San Francisco. This toolkit is open source and builds on the implementations produced by these organizations. Product names, logos, and trademarks are the property of their respective owners.
```bibtex theme={null}
@article{codonfm2025,
title={Learning the language of codon translation with CodonFM},
author={Darabi, Sajad and Cao, Fan and Naghipourfar, Mohsen and Rabi, Sara and Sethia, Ankit and Gion, Kyle and Grewal, Jasleen and Cohen, Jonathan and Greenleaf, William and Goodarzi, Hani and Sundaram, Laksshman},
year={2025},
publisher={NVIDIA},
url={https://research.nvidia.com/labs/dbr/assets/data/manuscripts/nv-codonfm-preprint.pdf}
}
```
| Function | Description | |
| -------------------------- | ---------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `run_codonfm_embeddings()` | Extract final-layer CLS embeddings for coding sequences with the CodonFM/Encodon model (GPU) | Docs Source |
| `run_codonfm_fitness()` | Score coding-sequence fitness with the upstream CodonFM/Encodon visible-token objective (GPU) | Docs Source |
| `run_codonfm_gradient()` | Compute the CodonFM/Encodon masked pseudo-log-likelihood gradient for relaxed coding sequences (GPU) | Docs Source |
| `run_codonfm_sample()` | Resample a subset of codons in coding sequences with the CodonFM/Encodon model (GPU) | Docs Source |
| `run_codonfm_score()` | Score codon substitutions by ref-vs-alt log-likelihood ratio with the CodonFM/Encodon model (GPU) | Docs Source |
## Background
Encodon is a BERT-style Transformer encoder trained with a masked-language-modeling objective over codon tokens ([Darabi et al., 2025](https://research.nvidia.com/labs/dbr/assets/data/manuscripts/nv-codonfm-preprint.pdf)). Each coding sequence is split into in-frame codons, a fraction of codon tokens are masked, and the model predicts the original codon from its full bidirectional context. Because the vocabulary is codons rather than amino acids, the model learns the *language of codon translation* — which synonymous codon is expected in a given context — and not only the encoded protein. The released family spans four checkpoints (80M, 600M, 1B parameters, plus a codon-frequency-aware-masking 1B variant), all reading sequences up to 2048 tokens, i.e. coding sequences up to `(2048 - 2) × 3 = 6138 nt` after the CLS/SEP tokens.
## Tools
### CodonFM Fitness (`codonfm-fitness`)
Runs the upstream Encodon fitness routine: one unmasked forward pass, followed by the mean log-probability assigned to each visible non-padding input token. The mean includes the CLS and SEP special tokens as well as the codons. Higher is more model-typical, but this score is not masked pseudo-log-likelihood.
#### API Reference
Source
Coding sequence(s) at codon resolution. A single string is normalized to a one-item list; `U` maps to `T` and each length must be a multiple of 3 (codon-aligned) and at most 6138 nt.
Source
Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). `True` is coerced to `1` and `False` to `0`.
Device used for inference.
Maximum execution time in seconds. `None` waits indefinitely.
Random seed. When set, tools run reproducibly up to small GPU float noise (see `BaseToolOutput.approx_equal`), and the seed participates in cache keys. When None, cacheable seed-sensitive tools skip cache until seeded.
Encodon checkpoint to run.
Available options: `encodon_80m`, `encodon_600m`, `encodon_1b`, `encodon_1b_cdwt`
Sequences processed per GPU forward pass.
Source
Per-sequence fitness predictions.
Coding sequence that was scored (DNA alphabet).
Length of the scored sequence in nucleotides.
Mean per-token log-likelihood under Encodon; higher is more model-typical (a proxy for a well-formed, natural coding sequence).
#### Applications
Sequence fitness is a zero-shot naturalness score for ranking or filtering coding sequences: comparing codon-optimized designs against wild type, screening synthetic constructs, or scoring candidate CDS variants without any task-specific training.
#### Usage Tips
* **Fitness is a relative score against Encodon's training distribution, not an absolute quantity.** It is most meaningful when comparing related sequences of similar length; the per-token mean already normalizes for length, but very short sequences are noisy.
* **`batch_size` trades memory for throughput.** Lower it if you OOM on long CDS, raise it for short sequences.
### CodonFM Score (`codonfm-score`)
Scores individual **codon substitutions** by the reference-vs-alternate log-likelihood ratio at the mutated position. The reference codon is masked and the model's log-probability of the reference and alternate codons is compared; a positive `llr` (`ref − alt`) means the model favors the reference and the substitution is model-disfavored.
#### API Reference
Source
One or more codon substitutions, each against its own reference coding sequence.
The reference coding sequence (codon-aligned DNA/RNA).
1-based codon position of the substitution within `sequence`.
Reference codon at `codon_position` (must match `sequence`).
Alternate codon substituted at `codon_position`.
Source
Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). `True` is coerced to `1` and `False` to `0`.
Device used for inference.
Maximum execution time in seconds. `None` waits indefinitely.
Random seed. When set, tools run reproducibly up to small GPU float noise (see `BaseToolOutput.approx_equal`), and the seed participates in cache keys. When None, cacheable seed-sensitive tools skip cache until seeded.
Encodon checkpoint to run.
Available options: `encodon_80m`, `encodon_600m`, `encodon_1b`, `encodon_1b_cdwt`
Mutations processed per GPU forward pass.
Source
Per-mutation log-likelihood-ratio scores.
Reference coding sequence used as model context.
Length of the reference sequence in nucleotides.
1-based codon position of the substitution.
Reference codon.
Alternate codon.
Model log-likelihood of the reference codon at the site.
Model log-likelihood of the alternate codon at the site.
Log-likelihood ratio `ref - alt`; positive means the model favors the reference and the substitution is model-disfavored.
#### Applications
The masked log-likelihood ratio is a canonical zero-shot variant-effect estimator. Because Encodon is codon-level, it discriminates *synonymous* substitutions (same amino acid, different codon) that an amino-acid model cannot see — useful for studying codon-usage effects on expression and mRNA stability.
#### Usage Tips
* **Each mutation carries its own reference sequence, codon position (1-based), and ref/alt codons.** The reference codon is validated against the sequence at that position, so an off-by-one frame error is caught before dispatch rather than silently mis-scored.
* **Scores are position-conditional.** The same substitution scored in different sequence contexts will differ; that context-dependence is the point.
### CodonFM Embeddings (`codonfm-embedding`)
Returns the final-layer **CLS-token embedding** for each coding sequence — a fixed-length learned representation whose dimensionality follows the checkpoint.
#### API Reference
Source
Coding sequence(s) at codon resolution. A single string is normalized to a one-item list; `U` maps to `T` and each length must be a multiple of 3 (codon-aligned) and at most 6138 nt.
Source
Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). `True` is coerced to `1` and `False` to `0`.
Device used for inference.
Maximum execution time in seconds. `None` waits indefinitely.
Random seed. When set, tools run reproducibly up to small GPU float noise (see `BaseToolOutput.approx_equal`), and the seed participates in cache keys. When None, cacheable seed-sensitive tools skip cache until seeded.
Encodon checkpoint to run.
Available options: `encodon_80m`, `encodon_600m`, `encodon_1b`, `encodon_1b_cdwt`
Sequences processed per GPU forward pass.
Source
Per-sequence CLS embeddings.
Coding sequence that was embedded (DNA alphabet).
Length of the sequence in nucleotides.
The final-layer CLS-token embedding vector (hidden size depends on the checkpoint: 1024 for 80M, 2048 for 600M/1B).
#### Applications
The CLS embedding is a codon-aware sequence descriptor for downstream supervised tasks (classification, regression, clustering) and similarity search over coding sequences.
#### Usage Tips
* **Different checkpoints produce different embedding sizes.** Representations from one checkpoint do not transfer to another without re-fitting downstream models; pick one and keep it fixed for an analysis.
### CodonFM Gradient (`codonfm-gradient`)
Computes the gradient of the mean masked negative log-likelihood with respect to a relaxed `(L, 64)` distribution over all 64 DNA codons, including the three standard stop codons (lexicographic order `AAA, AAC, AAG, AAT, …`). Each row's current argmax codon is the masked-language-model target. The Encodon weights are frozen; the relaxed distribution passes through Encodon's embedding layer and normalization, each codon position is masked in turn, and a per-chunk backward pass accumulates the gradient. An optional Straight-Through Estimator runs the forward on hard one-hot codons while routing gradients through the soft probabilities.
#### API Reference
Source
Relaxed coding-sequence state with shape `(L, 64)` in lexicographic DNA codon order (`AAA, AAC, AAG, AAT, ...`; see :data:`CODONFM_CODON_VOCAB`). `L` must be ≤ 2046 (Encodon's positional cap minus the CLS/SEP tokens); over-length inputs raise `ValueError`. Each row's argmax codon is used as that position's masked-language-model target.
Optional softmax temperature. When set, applies `softmax(input / temperature)` before computing the gradient. When `None`, every input row must already be a probability distribution.
Source
Encodon weights variant.
Available options: `encodon_80m`, `encodon_600m`, `encodon_1b`, `encodon_1b_cdwt`
Straight-Through Estimator: hard one-hot codons in the forward pass with gradients flowing through soft probabilities. When `False`, uses soft blended codon embeddings directly.
Run backward pass and return gradient. Set `False` for forward-only masked-log-likelihood scoring.
Codon positions per forward pass for batched masked-PLL.
Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). `True` is coerced to `1` and `False` to `0`.
Device to run the model on.
Maximum execution time in seconds. `None` waits indefinitely.
Random seed. When set, tools run reproducibly up to small GPU float noise (see `BaseToolOutput.approx_equal`), and the seed participates in cache keys. When None, cacheable seed-sensitive tools skip cache until seeded.
Source
Gradient w\.r.t. the input codon logits, or `None` when `compute_gradient=False`.
Mean masked negative log-likelihood over codon positions.
Log-likelihood, perplexity, sequence length, and objective details.
Codon column ordering for the input logits and returned gradient.
#### Applications
This exposes Encodon as a differentiable, codon-level naturalness prior for continuous sequence design — usable inside gradient descent, MCMC, or any optimization loop over relaxed coding sequences (e.g. codon optimization with a learned constraint).
#### Usage Tips
* **`temperature` converts the raw input into a per-position distribution.** The default `1.0` applies `softmax(logits / T)`; set it to `None` only when every row is already a non-negative probability distribution summing to 1.
* **`use_ste` enables the Straight-Through Estimator** for stronger guidance toward discrete codons while keeping a usable gradient.
* **`compute_gradient=False` runs forward-only.** The `gradient` field is `None` but `loss` and `metrics` are still populated. This is a masked pseudo-log-likelihood objective; it is distinct from the visible-token objective returned by `codonfm-fitness`.
### CodonFM Sampling (`codonfm-sample`)
Resamples a subset of codons in a coding sequence. A number of codon positions (`num_mutations`, or `mask_fraction` of the codons) are chosen at random, masked, and refilled from Encodon's distribution over the 61 sense codons in a single forward pass. The sequence length is preserved and sampling cannot introduce a new stop codon. It can replace an existing stop if that position is selected, so keep a required terminal stop outside the editable region or restore it afterward.
#### API Reference
Source
Coding sequence(s) at codon resolution, optionally with whole codons masked as `___` to choose which positions are resampled.
Source
Encodon checkpoint to sample from.
Available options: `encodon_80m`, `encodon_600m`, `encodon_1b`, `encodon_1b_cdwt`
Which codons to resample, counted in codons rather than nucleotides. Ignored when the input already carries `___` masks, which name the positions outright. `fixed_positions` pins codons that must survive, which is how a start or stop codon is kept intact.
Position-selection method. Always `"random"` for this tier (uniform selection, no model).
Exact number of positions to mask per sequence.
Fraction of designable positions to mask (e.g. 0.15 for \~15%).
1-indexed positions that must NOT be masked. Applied uniformly to all sequences.
Softmax temperature for codon sampling; higher is more diverse.
Number of (same-length) sequences per GPU forward pass.
Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). `True` is coerced to `1` and `False` to `0`.
Device used for CodonFM inference.
Maximum execution time in seconds. `None` waits indefinitely.
Random seed. When set, tools run reproducibly up to small GPU float noise (see `BaseToolOutput.approx_equal`), and the seed participates in cache keys. When None, cacheable seed-sensitive tools skip cache until seeded.
Source
One resampled coding sequence per input.
Resampled coding sequence in the DNA alphabet.
#### Applications
Masked-codon resampling is the local-edit primitive behind coding-sequence design: it proposes model-plausible synonymous or missense codon changes for directed-evolution / MCMC / genetic-algorithm loops (e.g. as the mutation generator in a Proto Language optimizer, paired with the `codonfm-fitness` constraint).
#### Usage Tips
* **`num_mutations` overrides `mask_fraction`.** It sets an exact number of positions to resample, not a guaranteed Hamming distance: the model can draw the original codon again.
* **`temperature` controls diversity.** Below 1.0 sharpens toward the model's favorite codon; above 1.0 broadens exploration. Because sampling is stochastic, pass a `seed` for reproducible proposals.
## Toolkit Notes
These apply to every CodonFM tool in this toolkit (`codonfm-fitness`, `codonfm-score`, `codonfm-embedding`, `codonfm-gradient`, `codonfm-sample`).
* **Checkpoints download on demand.** The `nvidia/NV-CodonFM-Encodon-*-v1` repos are public; the standalone worker fetches the `.safetensors` weights and their `config.json` on first use and caches them. No HuggingFace token is required (one is used automatically if present).
* **A CUDA GPU is required.** The pinned upstream Encodon attention implementation uses xFormers kernels that do not provide a CPU execution path.
* **Inputs are codon-aligned nucleotide sequences.** Each length must be a multiple of 3; RNA `U` is mapped to `T`, while ambiguous bases such as `N` are rejected because upstream tokenization would shift codon positions. Inputs are not checked for a start codon, terminal stop, internal stops, or coding-strand orientation.
* **Max sequence length is 6138 nt (2046 codons).** Encodon's positional cap is 2048 tokens; longer inputs raise `ValueError` rather than truncating.
* **The default checkpoint is `encodon_80m`.** It is the fastest; the 600M/1B checkpoints trade throughput for fidelity. Pick a larger one for final scoring.
**Example notebook:** See the [full working example](https://github.com/evo-design/proto-tools/blob/ede09259d665f23bd5fb57c68b020dfbc539a080/proto_tools/tools/masked_models/codonfm/examples/example.ipynb) for a copy-paste-ready walkthrough.
## Infrastructure Guides
The following guides cover how to run tools efficiently and at scale.
Keep a tool's model warm across calls instead of reloading it every invocation.How GPUs are allocated to tools and how to target specific devices.Fan a batch of inputs out across multiple GPUs.
evo-design/proto-tools/proto\_tools/tools/masked\_models/codonfm
roknabadi
NVIDIA
Arc Institute
UCSF
NV-CodonFM-Encodon-80M-v1
