IgLM

GLJHU

License: IgLM is licensed under Custom (JHU Academic Software License Agreement) and has restrictions around commercial use and may require explicit attribution when utilized. Please refer to the license for full terms.

Proto is not affiliated with Gray Lab, Johns Hopkins University. This toolkit is open source and builds on the implementation produced by this organization. Product names, logos, and trademarks are the property of their respective owners.

GitHub GitHub Publication Publication Preprint Preprint Cite Cite Tool Source Tool Source Open as Notebook Open as Notebook Run on Modal Run on Modal

Graylab/IgLM

View repo

IgLM: Infilling language modeling for antibody sequence design

Richard W Shuai, Jeffrey A Ruffolo and Jeffrey J Gray

Cell Systems (2023)

Read paper Read preprint

@articleshuai2023iglm

@article{shuai2023iglm,
  title={IgLM: Infilling language modeling for antibody sequence design},
  author={Shuai, Richard W and Ruffolo, Jeffrey A and Gray, Jeffrey J},
  journal={Cell Systems},
  volume={14},
  number={11},
  pages={979--989.e4},
  year={2023},
  publisher={Elsevier},
  doi={10.1016/j.cels.2023.10.001}
}

Copy citation

evo-design/proto-tools

Toolkit contributors

Function	Description
`run_iglm_infill()`	Re-design a span of an antibody sequence with IgLM, conditioned on the rest of the chain (GPU)	Docs Source
`run_iglm_sample()`	Generate full-length antibody sequences with IgLM, conditioned on chain and species (GPU)	Docs Source
`run_iglm_score()`	Score antibody sequences with IgLM, conditioned on chain and species (GPU)	Docs Source

Background

IgLM (Shuai et al., 2023) is a transformer-based language model trained on antibody sequences from the Observed Antibody Space (OAS) database. Its architecture is based on GPT-2, with two released checkpoints: IgLM (13 million parameters) and IgLM-S (1.4 million parameters). The model’s principal innovation is its sequence-infilling objective. During training, a randomly selected span of 10-20 amino acids is masked and appended after the remaining sequence. This allows an autoregressive model to generate replacement residues using information from both sides of the masked region. This approach is particularly useful for antibody design, where the complementarity-determining regions (CDRs) can be modified while retaining the surrounding framework. The authors demonstrated this by generating CDR H3 libraries for 49 therapeutic antibodies. Compared with their parent sequences, the generated libraries showed improvements in computational estimates of solubility, aggregation propensity, and sequence similarity to human antibody repertoires. These results support IgLM’s use for antibody library design, although experimental characterization is still required to establish binding, stability, and developability. IgLM also assigns likelihoods to antibody sequences, providing a quantitative measure of how well they conform to the sequence distributions learned during training. The authors demonstrated that these scores can distinguish human from non-human antibodies, achieving a reported ROC AUC of 0.96 on their evaluation dataset.

Learning Resources

Observed Antibody Space - Database of antibody sequences used to train IgLM.
OAS95 dataset - Sequence dataset and training/test splits associated with the original publication.

Tools

IgLM Sequence Generation (`iglm-sample`)

Generates antibody variable-domain sequences conditioned on chain type and species of origin. An optional N-terminal sequence can be provided as a starting prompt. Each input produces num_sequences_per_input distinct sequences.

API Reference

Source

Input: IgLMSampleInput

List[string]

default:"['heavy']"

Chain type conditioning each prompt.

List[string]

default:"['human']"

Species-of-origin conditioning each prompt.

List[string]

default:"['']"

N-terminal prefix to continue for each item. An empty string generates from the conditioning tokens alone.

Source

Config: IgLMSampleConfig

boolean

default:"True"

Keep the prompt at the start of each generated sequence; when False, only the newly generated residues are returned.

integer

default:"0"

Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). True is coerced to 1 and False to 0.

string

default:"cuda"

Device to run the model on.

integer

default:"3600"

Maximum execution time in seconds. None waits indefinitely.

integer

Random seed. When set, tools run reproducibly up to small GPU float noise (see BaseToolOutput.approx_equal), and the seed participates in cache keys. When None, cacheable seed-sensitive tools skip cache until seeded.

enum

default:"IgLM"

IgLM weights variant.Available options: IgLM, IgLM-S

integer

default:"1"

Distinct sequences to generate per prompt.

number

default:"1.0"

Softmax temperature; lower is more conservative.

number

default:"1.0"

Nucleus sampling cutoff over per-position token probabilities.

integer

default:"150"

Token budget per generation, including conditioning tokens and the prompt.

integer

default:"32"

Candidate sequences per forward pass.

Source

Output: IgLMGenerationOutput

List[IgLMSamples]

required

One IgLMSamples per input item, in input order.

Show IgLMSamples

List[string]

required

Distinct generated sequences, in generation order.

Applications

Use iglm-sample to generate synthetic antibody sequence libraries or extend an existing N-terminal sequence into a complete variable domain. The original publication demonstrates the generation of human heavy chains beginning with EVQ and camelid heavy-chain sequences beginning with QVQ.Generated sequences can be evaluated with iglm-score or other methods for assessing structural properties and developability.

Usage Tips

Chain and species conditioning: chains accepts heavy or light, while species supports human, mouse, rat, rabbit, rhesus, and camel. A single value can be applied to an entire batch, or separate values can be provided for individual sequences.
Sequence diversity: temperature and top_p control sampling diversity. Lower values favor more probable residues, while higher values allow a broader range of sequences. The paper explores temperatures of 0.8-1.2.
Number of sequences: num_sequences_per_input specifies the number of distinct sequences requested for each input. If the implementation cannot generate enough unique, valid sequences within its sampling budget, it raises an error rather than returning an incomplete collection.
Maximum length: max_length sets the total token budget, including conditioning tokens and the input prompt. The default is 150, and both released models support a context length of up to 512 tokens. Sequences that fail to terminate within the available budget are discarded.
Prompt handling: By default, prepend_prompt=True includes the original prompt in the returned sequence. Set it to False to return only the generated continuation.

IgLM Span Redesign (`iglm-infill`)

Replaces a specified region of an antibody sequence with newly generated residues while preserving the surrounding sequence. The model uses both flanking regions, together with chain and species conditioning, to generate a replacement of variable length.Each result includes the redesigned full sequence, the newly generated residues, and their positions within the resulting sequence.

API Reference

Source

Input: IgLMInfillInput

List[string]

required

Parent antibody sequence(s) to re-design.

List[IgLMSpan]

required

Residue span replaced in each parent sequence (1-indexed, inclusive).

Show IgLMSpan

integer

required

First residue of the span (1-indexed, inclusive).

integer

required

Last residue of the span (1-indexed, inclusive).

List[string]

default:"['heavy']"

Chain type conditioning each sequence.

List[string]

default:"['human']"

Species-of-origin conditioning each sequence.

Source

Config: IgLMInfillConfig

integer

default:"0"

Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). True is coerced to 1 and False to 0.

string

default:"cuda"

Device to run the model on.

integer

default:"3600"

Maximum execution time in seconds. None waits indefinitely.

integer

enum

default:"IgLM"

IgLM weights variant.Available options: IgLM, IgLM-S

integer

default:"1"

Distinct sequences to generate per parent sequence.

number

default:"1.0"

Softmax temperature; lower is more conservative.

number

default:"1.0"

Nucleus sampling cutoff over per-position token probabilities.

integer

default:"150"

Token budget per generation, including conditioning tokens and the masked parent sequence.

integer

default:"32"

Candidate sequences per forward pass.

Source

Output: IgLMInfillOutput

List[IgLMInfilledSequences]

required

One bundle per parent sequence, in input order, holding the re-designed sequences, the residues IgLM wrote, and where each new span sits.

Show IgLMInfilledSequences

List[string]

required

Residues IgLM wrote into the span, aligned with sequences. Lengths vary; IgLM chooses how long the new span is.

List[IgLMSpan]

required

Where those residues sit in each returned sequence, ready to hand straight back to iglm-score.

List[string]

required

Full re-designed sequences, in generation order.

Applications

The primary application is diversification of antibody CDR loops while retaining the parent framework. This follows the approach used in the original study, where IgLM generated alternative CDR H3 sequences for therapeutic antibodies.The resulting libraries can be evaluated using IgLM likelihood scores, developability predictors, and downstream structural or experimental assays.

Usage Tips

Span coordinates: Residue positions are 1-indexed and inclusive. For example, IgLMSpan(start=99, end=106) selects eight residues. The implementation converts these coordinates to the 0-indexed, end-exclusive format used by upstream IgLM.
Variable-length generation: IgLM determines the length of each replacement span, which may differ from the original. The generated residues are available through infilled_residues, and their updated positions are recorded in spans.
Scoring generated regions: The returned spans can be passed directly to iglm-score to calculate the conditional likelihood of each redesigned region without manually adjusting coordinates.
Span length: The model was trained with masked regions of 10-20 residues. It can generate replacements of other lengths, but performance outside the training distribution is less well characterized.
Maximum length: max_length must accommodate the masked parent sequence, its conditioning tokens, and the generated replacement. Increase the default of 150 when working with longer sequences or expecting longer replacements.

IgLM Scoring (`iglm-score`)

Calculates the likelihood of antibody sequences under IgLM, conditioned on chain type and species of origin. The tool returns total log-likelihood, average log-likelihood, and perplexity, with optional per-position logits.When a span is provided, the model instead evaluates the conditional likelihood of that region given the remainder of the sequence.

API Reference

Source

Input: IgLMScoringInput

array

Optional span to score conditionally in each sequence (1-indexed, inclusive); None scores the whole sequence.

List[string]

default:"['heavy']"

Chain type conditioning each sequence.

List[string]

default:"['human']"

Species-of-origin conditioning each sequence.

List[string]

required

Antibody sequence(s) to score.

Source

Config: IgLMScoringConfig

enum

default:"IgLM"

IgLM weights variant.Available options: IgLM, IgLM-S

integer

default:"0"

Verbosity level (0=quiet, 1=info, 2=debug, 3=raw subprocess stderr). True is coerced to 1 and False to 0.

string

default:"cuda"

Device to run the model on.

integer

default:"3600"

Maximum execution time in seconds. None waits indefinitely.

integer

default:"8"

Sequences per forward pass.

boolean

default:"False"

Include per-position logits over IgLM’s 33-token vocabulary.

Source

Output: CausalModelScoringOutput

List[CausalModelScoringMetrics]

required

List of scoring outputs, one per input sequence. Each entry is a Metrics subclass with scalar metrics (log_likelihood, avg_log_likelihood, perplexity) and optional per-position _pp-suffixed list extras; logits and vocab are declared fields for raw model outputs.

Show CausalModelScoringMetrics

array

Per-position logits array (seq_len, vocab_size). None unless return_logits=True.

array

Token ordering for logits.

string

Name of the metric that best summarizes the result overall (e.g. "avg_plddt" for AlphaFold2). Used by downstream UI and reporting to pick a headline value.

string

Concrete Metrics subclass tag; enables typed reconstruction after a serialization round-trip.

Metrics (one set per scores item)

Metric	Type	Range	Availability
`log_likelihood`	float	≤ 0.0	always
`avg_log_likelihood`	float	≤ 0.0	always
`perplexity`	float	≥ 1.0	always

Applications

Whole-sequence likelihood can be used to assess how closely a sequence resembles the antibody repertoire represented in IgLM’s training data. This is useful for comparing candidate antibodies or studying differences between antibody sequence distributions.Conditional likelihood is particularly useful for evaluating regions generated by iglm-infill, allowing candidate CDR loops to be compared in the context of their surrounding framework.These scores reflect the model’s learned sequence distribution and should not be interpreted as direct predictions of binding affinity, stability, or experimental performance.

Usage Tips

Span coordinates: Spans use the same 1-indexed, inclusive convention as iglm-infill. Returned infilling spans can be supplied directly to the scoring tool.
Whole-sequence and conditional scoring: Omit spans to evaluate complete sequences. Providing a span scores that region using the remaining sequence as context.
Comparing scores: log_likelihood is summed over the scored tokens and depends on sequence length. Use avg_log_likelihood or perplexity when comparing sequences of different lengths. Lower perplexity indicates greater likelihood under the model.
Conditioning consistency: Compare scores obtained using the same checkpoint, chain type, species, and scoring procedure. Scores from different conditioning settings should not be treated as directly equivalent.
Scoring convention: Following the upstream implementation, likelihood calculations include the terminal token. avg_log_likelihood is the mean per-token log-likelihood, and perplexity is calculated as exp(-avg_log_likelihood).
Per-position logits: Set return_logits=True to obtain token logits for each scored position. These are returned with the vocabulary needed to identify the corresponding amino acid and special tokens.

Toolkit Notes

The Proto guides for tool persistence, device management, parallel execution, and cloud inference apply to all three IgLM tools.

Model checkpoints: Two pretrained checkpoints are available: IgLM (13M parameters) and IgLM-S (1.4M parameters). Both support the same conditioning vocabulary. The larger model is used for the principal evaluations in the original publication.
Model weights: Both checkpoints are distributed with the upstream Python package, so no separate model download or authentication is required.
Licensing: IgLM is distributed under the JHU Academic Software License Agreement. Use is restricted to eligible non-commercial, non-profit purposes unless a separate commercial license is obtained.
Supported sequences: IgLM was trained on antibody heavy- and light-chain variable domains. Constant regions, linked antibody constructs, and unrelated proteins fall outside its intended training distribution.
Conditioning: The model supports two chain types and six species of origin. These categories are fixed by the pretrained vocabulary and cannot be extended through inference settings alone.
Batch processing: batch_size controls the number of sequences processed together. Larger batches can improve generation throughput, subject to available device memory.
Sampling: Generation is stochastic, and multiple candidates can be produced for each input. Sampling temperature and nucleus probability provide control over sequence diversity.
Output organization: Generated sequences are grouped by input, preserving the association between each prompt or parent antibody and its candidate designs.
Evaluation: IgLM likelihood provides a measure of sequence plausibility under its training distribution, not a calibrated estimate of antibody function or developability. Additional computational and experimental evaluation is recommended.

Example notebook: See the full working example for a copy-paste-ready walkthrough.

Infrastructure Guides

The following guides cover how to run tools efficiently and at scale.

Tool Persistence

Keep a tool’s model warm across calls instead of reloading it every invocation.

Device Management

How GPUs are allocated to tools and how to target specific devices.

Parallel Execution

Fan a batch of inputs out across multiple GPUs.

Tools

Guides

Tool Catalog

Concepts

Implementing a Tool

Background

Learning Resources

Tools

IgLM Sequence Generation (`iglm-sample`)

API Reference

Applications

Usage Tips

IgLM Span Redesign (`iglm-infill`)

API Reference

Applications

Usage Tips

IgLM Scoring (`iglm-score`)

API Reference

Applications

Usage Tips

Toolkit Notes

Infrastructure Guides

Tool Persistence

Device Management

Parallel Execution

​Background

​Learning Resources

​Tools

​IgLM Sequence Generation (iglm-sample)

​API Reference

​Applications

​Usage Tips

​IgLM Span Redesign (iglm-infill)

​API Reference

​Applications

​Usage Tips

​IgLM Scoring (iglm-score)

​API Reference

​Applications

​Usage Tips

​Toolkit Notes

​Infrastructure Guides

Tool Persistence

Device Management

Parallel Execution

Background

Learning Resources

Tools

IgLM Sequence Generation (`iglm-sample`)

API Reference

Applications

Usage Tips

IgLM Span Redesign (`iglm-infill`)

API Reference

Applications

Usage Tips

IgLM Scoring (`iglm-score`)

API Reference

Applications

Usage Tips

Toolkit Notes

Infrastructure Guides