Data in msmu

In LC-MS/MS "shotgun" proteomics, data analysis typically follows a hierarchical path—starting from PSM-level data (PSM or precursor), progressing to peptides, and finally reaching proteins. Each stage introduces its own set of feature annotations, quantification matrices, and tool-specific metadata. As a result, shotgun proteomics data naturally form a multi-level and multi-dimensional structure: PSM/precursor, peptide, and protein; feature metadata; sample annotations; and QC metrics.

To manage these properties consistently, msmu adopts MuData from the scverse ecosystem as the fundamental data format. MuData, together with its constituent AnnData objects, is widely used in scRNA-seq to manage complex data matrices and their associated metadata. The same structure fits proteomics naturally: identification-level attributes, quantification values, and sample information can all be stored cleanly and explored in an integrated way.

msmu works with data formatted as a MuData object composed of multiple AnnData modalities. Therefore, understanding the usage of MuData and AnnData helps when working with msmu.

MuData modalities

A MuData object used in msmu is organized by modalities, each corresponding to a specific processing level such as psm, peptide, and protein:

mdata

mdata["psm"]

# or
mdata["protein"]

As a general AnnData object, each individual modality contains .X, .var, .varm, .obs, .obsm, .uns, etc.

• .X is a matrix holding the quantification data. It is dense for peptide- and protein-level modalities, and sparse where features belong to one run or one plex (see The quantification matrix).
• .var is a DataFrame containing metadata of features for each level. As an example, .var in psm modality (for PSMs or precursors) contains information describing scan number, filename, PEP, q-value, etc., with filename.scan as index.
• .varm is a dictionary-like structure to store additional per-feature matrices, such as boolean masks for filtering features.
• .obs is a DataFrame containing metadata of samples, such as sample name, condition, replicate number, etc., with filename or channel as index.
• .obsm is a dictionary-like structure to store additional per-sample matrices, such as PCA or UMAP coordinates.
• .uns is a dictionary-like structure to store unstructured annotations, such as decoy features pulled from search results.
• .layers is a dictionary-like structure to store additional per-feature quantification matrices, such as imputed values. Some functions in msmu provide options to read from or write to .layers.

Processing stages

The raw, normalized, and batch-corrected layers in this schematic are optional snapshots. normalise and correct_batch_effect update .X or a selected existing layer; they do not create those layers automatically.

Save the current .X before processing and restore it later from the named layer:

mdata = mm.dt.save_layer(mdata, modality="peptide", layer="raw")
mdata = mm.pp.log2_transform(mdata, modality="peptide")
mdata = mm.dt.load_layer(mdata, modality="peptide", layer="raw")

save_layer copies .X into the layer without changing .X. An existing layer raises an error unless overwrite=True is passed. load_layer copies the layer into .X without changing the layer or reverting other annotations. Both work in place, preserve dense or sparse storage, and record provenance. Backed MuData is unsupported; reopen it in memory with mm.read_h5mu(path) first.

Data ingestion from DB search tools

Although different search tools return result files with heterogenous formats, their contents can typically be organized into two main conceptual parts to construct peptide- and protein-level data.

• Identification data - Identified features with associated annotations
• Quantification data - Quantitative values for features across samples

read_* functions in msmu extract the essential columns required for QC and downstream processing and migrate them into the .var of the psm modality. read_* functions are implemented in msmu/_read_write/_reader_registry

Reader	Identification input	Required options / quantification	Created modalities
read_sage	results.sage.tsv	label="tmt" requires quantification_file="tmt.tsv"; label="label_free" uses lfq.tsv when available	TMT: psm; LFQ with quantification: psm, peptide
read_diann	report.tsv or report.parquet	Quantification is in the report; default level="precursor"	psm (including DIA precursor data)
read_maxquant	evidence.txt	label="tmt" or "label_free", acquisition="dda"; quantification is in the file	TMT: psm; LFQ: psm, peptide
read_fragpipe	psm.tsv	label="tmt" or "label_free", acquisition="dda"; LFQ quantification uses combined_modified_peptide.tsv	TMT: psm; LFQ with quantification: psm, peptide
read_delpi	DELPI output table	Quantification is in the table	psm
read_h5mu	Saved .h5mu file	None	Preserves saved modalities

Supply files, not output-directory placeholders. MaxQuant/FragPipe DIA and DIA-NN level="protein_group" are not implemented. Identification-only LFQ imports do not supply peptide intensities: add quantification with io.add_quant as shown in the FlashLFQ section of the DDA-LFQ tutorial before quantitative analysis.

• Inputs
  • identification_file: A file path to identification data
  • quantification_file: A file path to quantification data (if applicable) (for tools outputting separate quantification files like Sage)
  • label: used label (tmt or label_free)
  • acquisition: required as "dda" for the implemented MaxQuant and FragPipe readers
  • drop_search_result (Sage, DIA-NN, MaxQuant, DELPI; default False): skip keeping the raw search output in .varm["search_result"]. It is a full copy of the search table, so dropping it noticeably lowers memory and file size when you do not need to trace values back to the original columns.
• Output
  • mudata: Data ingested MuData object
• Columns migrated into mdata["psm"].var
  • filename, peptide(modified), stripped_peptide, scan_num, proteins, missed_cleavages, peptide_length, charge, PEP, q_value, contaminant
• proteins holds parsed UniProt accessions in one canonical form across every reader, [rev_][Cont_]<accession> (e.g., P07339, Cont_P02769), and contaminant flags whether any member of the group carried a contaminant marker. See Filter.
• Decoy features are isolated from .var and stored in .uns["decoy"] for later use in FDR calculation.
• Quantification data for LFQ (DDA) is stored in peptide modality.
• Raw information from a search tool is stored in mdata["psm"].varm["search_result"]

mdata = mm.read_sage(
    identification_file="path/to/results.sage.tsv",
    quantification_file="path/to/tmt.tsv",
    label="tmt",  # or "label_free"
)

mdata = mm.read_diann(
    identification_file="path/to/report.tsv",
)

mdata = mm.read_maxquant(
    identification_file="path/to/evidence.txt",
    label="tmt",  # or "label_free"
    acquisition="dda",  # DIA is not implemented for this reader
)

mdata = mm.read_fragpipe(
    identification_file="path/to/output_file/psm.tsv",
    quantification_file="path/to/quantification_file/combined_modified_peptide.tsv", # for LFQ
    label="label_free",
    acquisition="dda",
)

The quantification matrix

Two different things leave a cell of .X empty. A missing value could have been measured but was not; a structurally absent cell could never have carried a number, because the sample and the feature belong to different acquisition units — a TMT reporter channel exists only for the PSMs of its own plex. Which cells are measurable is fixed by the design before any data exists, so it fixes both the shape of the matrix and how msmu stores it.

Experiment	Quantification in	.obs (samples)	.var (features)	Design	.X
DDA, label-free	peptide	run	peptide	complete	dense
DDA, TMT — single plex	psm	channel	filename.scan	complete	dense
DDA, TMT — multi-plex	psm	channel_set	filename.scan	block diagonal	sparse
DIA — precursor level	psm	run	run.precursor	one cell per feature	sparse

The matrices are drawn at one scale, and color marks the group a feature belongs to. The bar under each one is the same grid seen as storage: the outline is every cell, the fill is what .X really keeps — a dense array allocates every slot, including the struck-through empty ones, while a sparse one keeps only the colored cells and stops. For label-free DDA the quantification arrives at peptide level, so the peptide modality carries it and psm keeps identifications only.

Multi-plex TMT. Before splitting, the observations are the C reporter channels, so row 126 stands for a different sample in every plex. split_tmt() relabels them into C x n_set distinct channel_set samples; a cell is then measurable only when the sample and the PSM share a plex, which makes the matrix block diagonal with C x m_k blocks and 1/n_set of it measurable.

DIA. The feature id carries the run (.var.index is run.precursor), so a feature has exactly one measurable cell — the degenerate case where every block is a single row, with 1/n_run measurable. Each report row maps to one cell, so read_diann() writes them straight into the sparse store and no wide table is built.

Both grouped forms keep only the observed cells, in a scipy sparse .X.

The sparsity is an implementation detail of the storage, not of the analysis. Every function that reads .X goes through a sparse-aware path, so absent cells stay NaN and are never read as a measured zero — preprocessing (log2_transform, normalise, scale_data, correct_batch_effect, collapse_obs), summarization (to_peptide, to_protein, to_ptm), statistics (run_de, corr, pca, umap), plotting, and the exporters all behave the same as on a dense matrix. Normalization, scaling, and batch correction also return sparse output for sparse input, so the saving is not spent at the first preprocessing step.

Summarization removes the grouping — a peptide is one feature across every run and plex, so its cells are all measurable — and peptide- and protein-level modalities are dense again:

mdata = mm.read_diann(identification_file="path/to/report.parquet")
mdata["psm"].X          # <6x5038 sparse matrix ...>, 5,037 stored cells (16.7%)

mdata = mm.pp.to_peptide(mdata)
mdata["peptide"].X      # dense ndarray

To read the values out yourself, use mm.io.to_readable(), which restores absent cells as NaN. AnnData's own mdata["psm"].to_df() densifies a sparse .X with zeros, so absent measurements silently become 0 intensities:

mm.io.to_readable(mdata, modality="psm")   # absent cells -> NaN
mdata["psm"].to_df()                       # absent cells -> 0  (do not use on a sparse .X)

to_readable() returns .var and the quantification side by side; narrow it with the optional include / exclude (feature columns to keep or drop) and quantification=False (annotations only).

Returned objects and in-place changes

Keep the returned object when chaining processing steps:

mdata = mm.pp.log2_transform(mdata, modality="peptide")
mdata = mm.pp.normalise(mdata, modality="peptide", method="median")

These functions copy the input, as do pp.add_filter, pp.apply_filter, pp.attach_sdrf, pp.apply_sdrf_to_obs, pp.infer_protein, pp.to_peptide, pp.to_protein, tl.pca, and tl.umap. Ignoring their return value discards the processed copy. Consult the API reference for other operations rather than assuming all functions have the same mutation behavior. In particular, pp.to_ptm adds its new PTM modality to the supplied MuData in place; copy first to preserve an unmodified input.

dt.assign, dt.map, dt.replace, dt.drop, dt.save_layer, and dt.load_layer modify the supplied MuData in place and return that same object. Copy first when you need an independent branch:

branch = mdata.copy()
branch = mm.dt.assign(branch, "reviewed", True, modality="protein")

dt.concat returns a new combined MuData. tl.run_de returns a result object and records its execution history on the input MuData. Plotting returns a plotly figure; export functions return tables or write files. Direct pandas/numpy assignments are not automatically recorded; see Provenance for the supported boundaries.