To improve reproducibility and usability, we suggest the Gene Expression Omnibus (GEO) improve its submission guidelines for processed data stemming from single-cell RNA sequencing.
Current GEO guidelines require that a count matrix is included with a submission. But because the current GEO requirements surrounding supplemental files are vague, many studies do not include downstream analysis results (e.g., cluster assignments, UMAP/t-SNE projection coordinates, and cell type classification metadata), which vary due to algorithm parameter settings and non-deterministic steps in the processing pipeline.
For single-cell RNA-seq, cell type assignments are not labeled by sample names, as would be the case in bulk RNA-seq data. Hence, a count matrix is simply an intermediate file format, similar to other intermediate formats for bulk RNA-seq experiments (BAM/SAM/BED), which also do not enable reproduction of conclusions. This is counter to GEO’s own stance on processed data, “defined as the data on which the conclusions in the related manuscript are based”. The lack of a clear GEO guideline leads to reproducibility issues, as studies that only provide count matrices require significant domain-specific expertise to associate per-cell mRNA counts with a cell type or phenotype.
We suggest that in addition to count matrices, a metadata table containing relevant information (e.g., sample, clustering, cell-type, pseudo-time information) should be a required component of single-cell RNA-seq submissions. These metadata tables are already generated by commonly used single-cell RNA-seq analysis workflows (Seurat, SingleCellExperiment, scanpy) and could easily be included with other submission components.
For example, we compare the usability and reproducibility of two single-cell mRNA-seq submissions in GEO:
GEO accession GSE137710 contains a metadata file for each sample (e.g., “GSE137710_human_melanoma_cell_metadata_9315x14.tsv.gz”), enabling rapid downstream analyses to find expression patterns and markers for newly described cell types.
In contrast, GEO accession GSE124494 only includes count matrix information, requiring a non-expert to recreate (i.e., guess) the published associations between cell-type and gene expression. This is also a particularly challenging example as the associated publication used now-outdated versions of Seurat and its integration algorithm, requiring even more work to reproduce. This metadata file was certainly created and could have easily been attached to the submission.
Moving forward, we suggest two remedies to this issue:
Updating the GEO submission guidelines to require metadata file submissions. This would be done on https://www.ncbi.nlm.nih.gov/geo/info/seq.html by updating the “Processed data files” with a section specifically outlining required data types for single-cell mRNA-seq submissions.
For single-cell mRNA-seq data, in addition to standard count matrices (genes-by-cells), we expect users to deposit metadata annotations generated during the course of analysis.
Encouraging previous depositors of single-cell mRNA-seq data to update their GEO records with metadata, if it was not included in the original submission.