About this page
Tutorial- What this page solves
- You have found a pancreatic cancer microarray dataset on GEO and want to compare gene expression between tumor and normal tissue. You do not know which file to download, whether the values need log2, or how to assign each sample to its group.
- Who it is for
- Medical graduate students and researchers who are doing a differential analysis on GEO data for the first time. They know some R but have not analysed microarray data before.
- What you will have
- You will end up with an expression matrix annotated with gene symbols, a full limma result table for all genes, a volcano plot and a heatmap, and checks for whether the groups and annotation are correct.
- Covers
- Download the Series Matrix of a microarray dataset from GEO and decide whether the values need log2.
- Convert probes to gene symbols, run a two-group limma analysis, and draw a volcano plot and a heatmap.
- Common problems such as failed downloads, group mismatches and annotation conflicts (see the troubleshooting section).
- Approaches for RNA-seq data, merging several datasets and using TCGA as a control (see the choices section).
- Does not cover
- Other analyses such as single-cell analysis and WGCNA; see the related guides.
- Enrichment analysis (GO, KEGG, GSEA); see the differential expression and enrichment tutorial.
- Wet-lab validation such as qPCR and immunohistochemistry.
- Survival analysis and prognostic models built on GEO data.
Main steps
Step 1 · Choose the file
Why this stepOne GEO study (accession starting with GSE) usually comes with several files. Choosing the wrong one means redoing every later step, so first decide which file this example uses.
This example is the pancreatic cancer microarray dataset GSE16515, with 52 samples on platform GPL570. The platform is the chip model; it determines which genes each probe corresponds to.
For microarray data, download the Series Matrix file, whose name ends in _series_matrix.txt.gz. It is the expression matrix processed by the submitter, together with the sample information. For other file types, see the choice “What to do when the data is not a Series Matrix”.
Step 2 · Download
Why this stepEvery later step reads this file. Download it to a fixed folder so that you do not have to download it again after closing R.
The first download of the GSE16515 Series Matrix (6.4 MB) took 8.3 seconds. getGPL = FALSE skips the 85.6 MB GPL570 annotation file, because step 4 uses an annotation package instead.
If the download fails or returns something other than data, look up the exact error in the download entries of the troubleshooting section. On a poor connection, use the manual download method in the choices section.
library(GEOquery)
options(timeout = 600) # default is 60 s; large files get cut off
Sys.setenv(VROOM_CONNECTION_SIZE = 500000L) # avoids the connection buffer error
dir.create("geo_cache", showWarnings = FALSE)
# Fixed cache directory: the default is tempdir(), which is wiped when R exits
gse <- getGEO("GSE16515", destdir = "geo_cache",
GSEMatrix = TRUE, getGPL = FALSE) # skip the 85.6 MB GPL570 annotation
length(gse) # multi-platform Series return several ExpressionSets
eset <- gse[[1]]
dim(eset) # 54613 probes x 52 samples
annotation(eset) # "GPL570"Step 3 · Preprocessing
Why this steplimma needs values on a log scale. A Series Matrix may already be log2-transformed or may contain linear values, so judge from the numbers themselves rather than from the description.
The GSE16515 description says GC-RMA, a method whose output is usually log2. However, the largest value in the matrix is 65045, so the values are actually linear.
The code below applies the GEO2R rule. When the result is TRUE, take log2 first, then check the distribution of each sample with a boxplot. The reason for this rule is in the explanation section.
library(limma)
ex <- exprs(eset)
# The rule GEO2R uses
qx <- as.numeric(quantile(ex, c(0, .25, .5, .75, .99, 1), na.rm = TRUE))
LogC <- (qx[5] > 100) || (qx[6] - qx[1] > 50 && qx[2] > 0)
qx; LogC
# GSE16515: 2.28 7.11 18.17 86.87 3283.25 65045 -> TRUE
if (LogC) { ex[ex <= 0] <- NaN; ex <- log2(ex) }
# Check distributions: sample medians in the boxplot should roughly line up
boxplot(ex, las = 2, outline = FALSE)
# Only if they clearly do not: quantile normalization (what GEO2R's Force normalization does)
# ex <- normalizeBetweenArrays(ex, method = "quantile")Step 4 · Annotation
Why this stepMatrix rows are probe IDs, but plots and comparison with other datasets need gene symbols. One probe can match several genes and one gene can have several probes, so you need one consistent rule.
This example uses the annotation package hgu133plus2.db for GPL570. After removing probes without annotation and probes that match several genes, keep the probe with the highest mean expression for each gene.
For choosing an annotation package or GPL table, see the choice “Choose the annotation package by platform”. For why the handling of multiple probes changes the result, see the explanation section.
library(hgu133plus2.db) # annotation package for GPL570; see the table for other platforms
pk <- AnnotationDbi::select(hgu133plus2.db, keys = rownames(ex),
columns = "SYMBOL", keytype = "PROBEID")
multi <- names(which(table(pk$PROBEID) > 1)) # probes mapping to several genes: 1528
pk <- pk[!is.na(pk$SYMBOL) & !pk$PROBEID %in% multi, ] # drop unannotated and multi-gene probes
ex <- ex[pk$PROBEID, ]; sym <- pk$SYMBOL # 43112 probes, 20826 genes left
# Several probes per gene: keep the probe with the highest mean (11041 genes have >1 probe)
o <- order(sym, -rowMeans(ex))
keep <- o[!duplicated(sym[o])]
ex_gene <- ex[keep, ]; rownames(ex_gene) <- sym[keep]
# Alternative, averaging: ex_gene <- avereps(ex, ID = sym)
# Platforms without a package: use the GPL table (getGEO("GPLxxx") or .annot.gz on FTP)
# gpl <- getGEO("GPL570", destdir = "geo_cache")
# anno <- Table(gpl)[, c("ID", "Gene Symbol")]
# anno <- anno[anno$`Gene Symbol` != "" & !grepl("///", anno$`Gene Symbol`), ]Step 5 · Groups
Why this steplimma matches groups to columns by position. If the group vector is off by one, every result is wrong and no error is shown, so check each item after extraction.
Group information is usually stored in a column named like xxx:ch1, so use grep to find the column name first. In this example, the tissue:ch1 column has 16 normal and 36 tumor tissues.
The first level of the factor is the reference group, and the comparison is written as tumor relative to normal. If the sample count does not match the paper or the source of the groups is unclear, see the group entries in the troubleshooting section.
pd <- pData(eset)
grep(":ch1$", colnames(pd), value = TRUE) # group labels usually live in xxx:ch1 columns
table(pd[["tissue:ch1"]])
# Normal Tissue in Pancreatic Cancer Sample 16 / Tumor Tissue ... 36
group <- factor(ifelse(grepl("Tumor", pd[["tissue:ch1"]]), "Tumor", "Normal"),
levels = c("Normal", "Tumor")) # first level is the reference
# Three checks: sample names match, no mismatches in the cross-table, group sizes match the paper
stopifnot(identical(rownames(pd), colnames(ex_gene)))
table(group, pd[["tissue:ch1"]])
patient <- sub("^Pancreatic Sample ?([0-9]*)-.*$", "\\1", pd$title) # pairing, used laterStep 6 · Differential analysis
Why this steplimma fits a linear model for each gene across the two groups, estimating the fold change and giving an adjusted P value. The design and the contrast set the direction of the comparison, so write them down first.
design is the design matrix of the groups, and makeContrasts sets the direction of the comparison. This example is tumor minus normal, so logFC > 0 means higher expression in tumor.
The filter is adj.P < 0.05 and |logFC| > 1. For the reason for these thresholds, other choices, and an equivalent way to write a two-group comparison, see the choices section.
design <- model.matrix(~ 0 + group)
colnames(design) <- levels(group) # Normal, Tumor
contr <- makeContrasts(Tumor - Normal, levels = design) # logFC > 0 means up in tumor
fit <- lmFit(ex_gene, design)
fit2 <- eBayes(contrasts.fit(fit, contr), trend = TRUE)
tt <- topTable(fit2, number = Inf) # BH adjustment by default
deg <- subset(tt, adj.P.Val < 0.05 & abs(logFC) > 1)
table(sign(deg$logFC)) # measured here: -1 399, 1 1136Step 7 · Plots
Why this stepUse the plots to check whether the result makes sense. The volcano plot shows fold change against significance, and the heatmap shows whether the differential genes separate the two groups.
The volcano plot uses the same P value on its y axis as the filter (adj.P throughout this page), and labels the 10 genes with the smallest adj.P. The heatmap takes 25 up-regulated and 25 down-regulated genes, with samples ordered by group.
If the heatmap shows genes in only one direction, see the heatmap entry in the troubleshooting section. For plotting details, see the reference section.
library(ggplot2); library(ggrepel); library(pheatmap)
v <- tt; v$gene <- rownames(v)
v$change <- with(v, ifelse(adj.P.Val < 0.05 & logFC > 1, "Up",
ifelse(adj.P.Val < 0.05 & logFC < -1, "Down", "NS")))
lab <- head(v[v$change != "NS", ][order(v$adj.P.Val[v$change != "NS"]), ], 10)
p <- ggplot(v, aes(logFC, -log10(adj.P.Val), colour = change)) +
geom_point(size = 0.8, alpha = 0.6) +
scale_colour_manual(values = c(Down = "#2b6cb0", NS = "grey75", Up = "#c53030")) +
geom_vline(xintercept = c(-1, 1), linetype = 2) +
geom_hline(yintercept = -log10(0.05), linetype = 2) + # same P on the axis as in the filter
geom_text_repel(data = lab, aes(label = gene), size = 3, colour = "black") +
labs(x = "log2 fold change (Tumor vs Normal)", y = "-log10 adjusted P") + theme_bw()
ggsave("volcano.png", p, width = 6, height = 5, dpi = 300)
# Heatmap: the 25 up- and 25 down-regulated genes with the smallest adj.P
up <- head(rownames(v)[v$change == "Up"][order(v$adj.P.Val[v$change == "Up"])], 25)
dn <- head(rownames(v)[v$change == "Down"][order(v$adj.P.Val[v$change == "Down"])], 25)
o <- order(group)
ann <- data.frame(group = group[o], row.names = colnames(ex_gene)[o])
pheatmap(ex_gene[c(up, dn), o], scale = "row", cluster_cols = FALSE,
annotation_col = ann, show_colnames = FALSE, fontsize_row = 6,
breaks = seq(-2, 2, length.out = 101), # cap z-scores so outliers do not flatten the scale
filename = "heatmap.png", width = 7, height = 8)Choose by your situation
Hand it to an agent
The download, annotation, differential analysis and plotting steps are fixed, so they suit a scientific agent. Choosing the dataset, defining the groups and interpreting the results remain your responsibility.
Example instruction: “Run a differential analysis of tumor versus normal pancreatic tissue using GSE16515 and GSE15471. For each dataset, decide whether log2 is needed, annotate with hgu133plus2.db and keep the probe with the highest mean expression, run limma separately, then merge the matrices and put the batch in the design matrix. Use adj.P < 0.05 and |logFC| > 1. Output the intersection of the differential genes from both datasets, PCA before and after batch correction, a volcano plot, and a heatmap with 25 up and 25 down genes.”
- 01
Download and check
Download the Series Matrix with GEOquery and cache it; if the interface is blocked, switch to FTP. Record the sample count, the platform and the data_processing description.
- 02
Preprocessing
Decide on log2 with the GEO2R rule, draw a boxplot, complete the probe annotation, and report probe and gene counts before and after annotation.
- 03
Differential analysis and plots
Extract the groups and check the sample order, run limma, and output the full gene result table, the volcano plot and the heatmap.
- 04
Merge and validate
Merge the datasets with the batch in the design matrix; output the PCA and an overlap table of differential genes from the two datasets.
- 05
Adversarial review
Check that the groups match the paper, that the logFC direction is correct, that log2 was not applied twice, and that the heatmap includes both up- and down-regulated genes.
- The project keeps the cached raw files, R scripts, sessionInfo, result tables and all figures.
- You still need to check that the dataset fits your research question, that the groups and sample exclusions match the paper, and that the thresholds suit your paper.
- Experimental validation such as qPCR and immunohistochemistry is up to you.
Choices
GEO also provides other file types. The table below shows when each one fits; first check which kind of data you have.
The NCBI-generated counts are produced as follows: HISAT2 aligns reads to GRCh38 (GCA_000001405.15), featureCounts counts them, and gene annotation is Annotation Release 109.20190905. Runs with alignment rates below 50% and single-cell data are not counted, and counts are summed when one GSM has several runs. GEO2R uses this same set of counts with DESeq2 for RNA-seq.
Species coverage: the official page (revised 2026-07-08) still says mouse data is “expected in 2025”. On 2026-10-11 this page searched E-utilities for "rnaseq counts"[Filter]: there were 0 Series with only mouse samples, and about 27,700 records with this label, all human.
GDS status: counting by year in E-utilities, 279 were added in 2014, 64 in 2015, 2 in 2016, and none since 2018. Old tutorials that use getGEO("GDSxxxx") only return old data.
| File | Contents | When to use | Watch out |
|---|---|---|---|
| Series Matrix (GSE…_series_matrix.txt.gz) | Submitter-processed expression matrix and sample metadata | Default for microarrays; GEO2R uses it too | The scale is up to the submitter, so check for log2; multi-platform GSEs have several files |
| Raw CEL files (GSE…_RAW.tar) | Raw Affymetrix scan signals | When Series Matrix values are odd or missing, or you want uniform RMA across datasets | Run RMA with oligo or affy; files are large and slow to download |
| NCBI-generated RNA-seq raw counts | Gene × sample integer matrix from NCBI's uniform alignment and counting | Human bulk RNA-seq where the submitter gave no counts or only FPKM | Human only for now; may not match the paper; the bundled FPKM and TPM are not for differential testing |
| Submitter supplementary files (suppl) | Counts, FPKM, TPM or other formats | When you need to match the paper exactly | Formats vary; check column names and whether values are integers before reading |
| GDS | Curated dataset with predefined groups | Reproducing pre-2015 datasets | Essentially no new GDS since 2016; recent data have none |
Choices
If the getGEO web interface is blocked or the download times out, download the file manually from FTP and read the local file.
A mirror commonly used in China is geoChina() from the AnnoProbe package by Biotrainee, which downloads the submitter-processed ExpressionSet (.Rdata) from a Tencent Cloud server. This page tested the server on 2026-10-11 and it was still reachable. Its GSE list is stored in the package, and the repository was last updated in November 2022, so datasets published later are not in the mirror.
GEOquery version difference: this page was measured with GEOquery 2.74.0 (Bioconductor 3.20). The development version on GitHub (2.81.x) plans to change the return type of getGEO from ExpressionSet to SummarizedExperiment; then exprs() and pData() must be replaced with assay() and colData().
# FTP path rule: replace the last three digits of the GSE number with nnn
# (GSE16515 -> GSE16nnn; accessions below 1000 use GSEnnn)
# -c resumes partial downloads; rerun the same command after an interruption
wget -c https://ftp.ncbi.nlm.nih.gov/geo/series/GSE16nnn/GSE16515/matrix/GSE16515_series_matrix.txt.gz
# Platform annotation (GPL570 is below 1000, so the folder is GPLnnn)
wget -c https://ftp.ncbi.nlm.nih.gov/geo/platforms/GPLnnn/GPL570/annot/GPL570.annot.gz
# Raw CEL files
wget -c https://ftp.ncbi.nlm.nih.gov/geo/series/GSE16nnn/GSE16515/suppl/GSE16515_RAW.tar
# Read the local file in R without going online
Rscript -e 'library(GEOquery); e <- getGEO(filename = "GSE16515_series_matrix.txt.gz", getGPL = FALSE); print(dim(e))'
Choices
The annotation package is determined by the chip platform. If no matching package exists, use the GPL table.
Annotation packages are newer than GPL tables. On GPL570, the two disagree on 3874 of the 41039 single-gene probes, mostly because genes have been renamed, for example FAM122C is now PABIR3 and CCDC11 is now CFAP53. State the annotation source and version in the methods.
| Platform | Annotation package | Notes |
|---|---|---|
| GPL570 HG-U133 Plus 2.0 | hgu133plus2.db | 54,675 probes; the most common |
| GPL96 HG-U133A | hgu133a.db | 22,283 probes |
| GPL571 HG-U133A_2 | hgu133a2.db | |
| GPL6244 HuGene 1.0 ST | hugene10sttranscriptcluster.db | Annotated by transcript cluster |
| GPL6947 / GPL10558 Illumina HumanHT-12 | illuminaHumanv3.db / illuminaHumanv4.db | Matrix row names are ILMN_ probe IDs |
| Agilent and other platforms without a package | GPL table or .annot.gz on FTP | The Gene Symbol column may contain " /// ", meaning one probe maps to several genes |
Choices
The threshold decides how many differential genes you get. Paired samples and fold-change thresholds each have their own code, so choose according to your design.
Reason for the thresholds: adj.P uses the BH method to control the false discovery rate, and 0.05 is the default in GEO2R. |logFC| > 1 is a convention with no statistical basis. The limma topTable help states that when P value and fold change are weakly correlated, filtering by P first and then by fold change lets the false discovery rate exceed the nominal level, and treat is recommended when a fold-change threshold is needed. If you use the conventional thresholds in a paper, state them and give the full gene table in the supplement.
The number of differential genes depends strongly on sample size: for the same tissue type, GSE15471 (78 samples) gives 2306 up-regulated genes and GSE16515 (52 samples) gives 1136. When up- and down-regulated counts are unbalanced, handle up and down genes separately in the heatmap and enrichment analysis.
A two-group comparison can also be written as ~group and then take coef = 2. This page checked on GSE15471 that the largest difference in logFC between the two forms is 8.8 × 10⁻¹⁵, so use whichever you can read easily. For more than two groups, use ~0 + group and state each contrast clearly.
# Build the fold-change threshold into the test (the approach limma recommends)
tr <- topTreat(treat(contrasts.fit(fit, contr), lfc = 1, trend = TRUE), number = Inf)
sum(tr$adj.P.Val < 0.05) # measured here: 166
# Paired samples: if every sample is paired, put patient in the design
# design <- model.matrix(~ patient + group)
# Partly paired (GSE16515 has 16 pairs plus 20 unpaired tumors):
corfit <- duplicateCorrelation(ex_gene, design, block = patient)
fitb <- lmFit(ex_gene, design, block = patient, correlation = corfit$consensus)
ttb <- topTable(eBayes(contrasts.fit(fitb, contr), trend = TRUE), number = Inf)| Cutoff or setting (GSE16515) | Up | Down | Notes |
|---|---|---|---|
| adj.P < 0.05 and |logFC| > 1 | 1136 | 399 | Most common in the literature |
| adj.P < 0.05 and |logFC| > 0.585 (1.5-fold) | 2565 | 1001 | Common with few samples or small effects |
| adj.P < 0.05 only | 4339 | 5992 | With large samples nearly every gene is significant |
| treat(lfc = 1), adj.P < 0.05 | 166 total | Tests whether the fold change is significantly above 2 | |
| eBayes(trend = FALSE), cutoff as row 1 | 1134 | 397 | trend has little effect on arrays |
| Partly paired, duplicateCorrelation, cutoff as row 1 | 1134 | 403 | Within-patient correlation 0.136, 16 pairs |
Choices
Differential analysis of RNA-seq needs raw integer counts. If GEO only has FPKM or TPM, first check whether NCBI generated raw counts; if not, you must download the fastq files from SRA and quantify them again.
The example measured on 2026-10-11 is GSE164073, with 18 samples (3 ocular surface tissues × infection/control × 3 replicates) and the design matrix ~ tissue + infection. After filterByExpr, 16933 genes remain; voom gives 54 up-regulated and 118 down-regulated genes (adj.P < 0.05, |logFC| > 1), taking 2.4 seconds.
library(GEOquery); library(edgeR); library(limma); library(org.Hs.eg.db)
acc <- "GSE164073"
hasRNASeqQuantifications(acc) # TRUE means NCBI-generated counts exist
# Download raw counts directly (this URL form is exempt from reCAPTCHA since 2026-09-02)
url <- paste0("https://www.ncbi.nlm.nih.gov/geo/download/?type=rnaseq_counts&acc=", acc,
"&format=file&file=", acc, "_raw_counts_GRCh38.p13_NCBI.tsv.gz")
f <- file.path("geo_cache", paste0(acc, "_raw_counts.tsv.gz"))
if (!file.exists(f)) download.file(url, f, mode = "wb")
cnt <- as.matrix(read.delim(f, row.names = 1, check.names = FALSE)) # row names are Entrez Gene IDs
sym <- mapIds(org.Hs.eg.db, rownames(cnt), "SYMBOL", "ENTREZID") # replaces the blocked annotation file
pd <- pData(getGEO(acc, destdir = "geo_cache", getGPL = FALSE)[[1]])[colnames(cnt), ]
inf <- factor(ifelse(grepl("SARS", pd[["infection:ch1"]]), "CoV2", "mock"), levels = c("mock", "CoV2"))
tis <- factor(pd[["tissue:ch1"]])
design <- model.matrix(~ tis + inf)
y <- DGEList(cnt)
keep <- filterByExpr(y, design = design) # 39376 -> 16933 genes
y <- normLibSizes(y[keep, , keep.lib.sizes = FALSE])
v <- voom(y, design)
tt <- topTable(eBayes(lmFit(v, design)), coef = "infCoV2", number = Inf)
tt$SYMBOL <- sym[rownames(tt)]| Data | Method | Notes |
|---|---|---|
| Microarray (log2 intensities) | limma | lmFit + eBayes |
| RNA-seq counts, 3–10 samples per group | DESeq2, edgeR or limma-voom | Results overlap heavily; GEO2R uses DESeq2 |
| RNA-seq counts with very different depths or a need for sample weights | limma-voom (voomWithQualityWeights) | Fast; design matrices written as for arrays |
| RNA-seq counts, dozens or more population samples per group | Wilcoxon rank-sum test (after edgeR normalization) | Li et al. 2022 reported excess false positives from DESeq2 and edgeR on such data |
| Only FPKM or TPM | No count-model differential testing | limma-trend on log2(TPM + 1) is exploratory at best; NCBI also notes FPKM and TPM are unsuitable for quantitative cross-sample comparison |
Choices
Before merging, confirm that the platforms are the same and that each dataset is already on a log2 scale. Batch effects are often larger than biological differences: after merging GSE16515 and GSE15471 (both GPL570), the first principal component explains 48.9% of the variance and mainly separates the two datasets.
The correct approach is to put the batch in the design matrix of the differential analysis and keep analysis and plotting separate. A corrected matrix is used only for PCA and heatmaps, not for limma. If you must use ComBat, include mod so that the group effect is protected.
library(sva)
# m: merged gene matrix of two same-platform GSEs (each confirmed log2); batch and group are factors
table(batch, group) # check the design first: does every batch contain both groups?
# Differential expression: put batch in the design
design <- model.matrix(~ group + batch)
fit <- eBayes(lmFit(m, design), trend = TRUE)
tt <- topTable(fit, coef = "groupTumor", number = Inf)
# Corrected matrix for PCA and heatmaps only (not fed back into limma)
m_plot <- removeBatchEffect(m, batch = batch, design = model.matrix(~ group))
# If you must use ComBat, pass mod to protect the group effect
m_cb <- ComBat(m, batch = batch, mod = model.matrix(~ group))| Approach | Balanced: both datasets have normal and tumor | Unbalanced: GSE15471 tumors only |
|---|---|---|
| No correction | 1174 / 621 | 3441 / 296 |
| Batch in the design matrix (recommended) | 1149 / 235 | 1117 / 404 |
| ComBat(mod = ~group), then limma | 1144 / 232 | 1108 / 417 |
| removeBatchEffect, then limma | 1149 / 236 | Not tested |
| ComBat without mod, then limma | 1063 / 220 | Not tested |
Choices
GEO holds scattered studies submitted by different labs, with different platforms and processing. TCGA is a cancer cohort processed with one pipeline, with full clinical and survival data, but it has few normal controls.
TCGA has few normal samples, so a common approach is to merge GTEx normal tissues. The two use different sequencing and processing pipelines, so after merging, batch and group are completely confounded, as explained in the batch section. Conclusions from such a comparison need validation with a GEO dataset or an experiment.
library(TCGAbiolinks)
q <- GDCquery(project = "TCGA-PAAD",
data.category = "Transcriptome Profiling",
data.type = "Gene Expression Quantification", # without this line you get twice the files
workflow.type = "STAR - Counts") # HTSeq - Counts no longer exists
GDCdownload(q, files.per.chunk = 20) # chunked; rerunning skips files already downloaded
se <- GDCprepare(q)
cnt <- SummarizedExperiment::assay(se, "unstranded") # use this assay for DESeq2/edgeR
table(se$sample_type) # TCGA-PAAD: Primary Tumor 178, Solid Tissue Normal 4| Item | GEO | TCGA (GDC) |
|---|---|---|
| Source | Thousands of independent studies, arrays and sequencing | 33 cancer types, uniform sequencing and processing |
| Normal controls | Many datasets include matched normal tissue | Few: TCGA-PAAD has 178 primary tumors and 4 normal tissues |
| Clinical data | Up to the submitter, often only the group | Survival, stage, treatment and more |
| Expression data | Series Matrix, CEL, counts | STAR - Counts (unstranded, TPM, FPKM and other columns) |
| Typical use | Finding DEGs, external validation | Prognostic models, survival analysis, cross-validation with GEO |
Concepts and judgment
Explanation
The basis for deciding on log2 is the distribution of the values. A description can be wrong; the values are not.
GEO2R decides automatically using only the grouped samples, and applies or skips log2 to the whole matrix. The code in step 3 uses the same rule, which relies on quantiles rather than on the description.
The number of differential genes changes a lot with the approach used on the same matrix. For measured results, see “Measured comparison of three log2 approaches” in the reference section.
Explanation
When a gene has several probes, the choice of probe changes the number of differential genes directly.
The way multiple probes are handled changes the number of differential genes
On GSE16515 with the same threshold, keeping the probe with the highest mean expression gives 1136 up and 399 down; averaging with avereps gives 834 up and 294 down; keeping the probe with the largest interquartile range gives 1237 up and 431 down. 450 genes are significant only under the maximum-mean method and 43 only under the averaging method; the difference mainly comes from low-expression probes pulling down the mean for the same gene. State which method you used in the methods.
Why keeping the probe with the highest mean expression is recommended
Keep the probe with the highest mean expression (AnnoProbe’s filterEM uses the highest median, with similar results). Averaging mixes in probes that are not expressed or not specific, which shrinks the differences. Running limma at the probe level and then merging to genes gives 1771 genes, more than analysing genes directly, because a gene is counted if any of its probes is significant.
Explanation
A heatmap of differential genes almost always separates the two groups, so it cannot be used as evidence that the groups are reliable.
With cluster_cols = TRUE, a heatmap of differential genes almost always separates the two groups, because the differential genes were selected for exactly that difference. To show the difference between groups, use a PCA based on all genes.
Reference and appendix
Reference
The table gives measured results for GSE16515 and GSE15471, using the same thresholds as the main text (|logFC| > 1, adj.P < 0.05).
| Approach | Measured on GSE16515 / GSE15471 (|logFC| > 1, adj.P < 0.05) |
|---|---|
| Log2-transform linear values, then limma (correct) | GSE16515: 1136 up, 399 down |
| Run limma on linear values | GSE16515: 2804 up, 3868 down, median |logFC| 4.51 |
| Apply log2(x+1) to data already in log2 | GSE15471: 0 (correct handling gives 2306 up, 348 down) |
Reference
Truncating z values
After scale = "row", some z values reach 4 or more, which compresses the colour scale so most cells look alike. Setting breaks to -2 to 2 shows values beyond that in the darkest colour.
Labelling genes on the volcano plot
Only the 10 genes with the smallest adj.P are labelled, using ggrepel to avoid overlap. The top genes here are LAMB3, SLC2A1, TMPRSS4, MET and S100P, which match genes known to be up-regulated in pancreatic cancer, a first check that the result is plausible.
Reference
GEO data-mining papers are often returned for a few recurring reasons, listed below. Fix as many as you can before submission.
| Objection | Response |
|---|---|
| Small sample size | Report samples per group; use limma's empirical Bayes method (more stable than t-tests with few samples); merge same-platform datasets with batch in the design matrix; frame conclusions as candidate genes |
| No validation set | Analyse an independent GSE separately and report DEG overlap and direction agreement. Here GSE16515 and GSE15471 gave 1507 and 1534 DEGs, with 787 shared, all in the same direction |
| Platforms differ and cannot be merged | Run differential expression per dataset and intersect, or integrate rankings with RobustRankAggreg; merge matrices only within a platform |
| Unclear batch-effect handling | Show PCA colored by batch before and after correction; state that batch is in the design matrix and removeBatchEffect is used only for plots |
| Arbitrary cutoffs | State the adj.P method and the |logFC| cutoff; add treat results or a sensitivity analysis |
| Bioinformatics only | Validate key genes by qPCR, immunohistochemistry or TCGA and HPA data; validate prognostic models in an external cohort |
| Not reproducible | Provide GSE accessions, platform, R and package versions, the full script and results for all genes |
Reference
The following points come from community posts on CSDN, Jianshu and Biotrainee. Each is supported by the original error text, source code or reproduction on this page.
The connection buffer error
A CSDN post shows the error “The size of the connection buffer (131072) was not large enough” when getGEO reads GSE94994. After setting Sys.setenv("VROOM_CONNECTION_SIZE" = 99999999) the file reads correctly. The path “Using locally cached version” in the error text points to the Rtmp temp folder, which means the author did not set destdir.
The probe handling rule in AnnoProbe
For a probe that matches several genes, filterEM keeps only one annotation, and the source code comments admit this is a problem. For a gene with several probes, it keeps the probe with the highest median. When using idmap + filterEM, the methods can say “keeping the probe with the highest median expression”.
Reading a GPL table
A Biotrainee post uses GPL570 as an example: skip the description lines beginning with # before reading the table; when Gene Symbol contains " /// " the post takes the first gene. Some platform pages only allow online viewing without a download button, so download the file from the annot folder on FTP.
log2(dat + 1) in templates
A widely shared template writes dat <- log2(dat + 1) right after the boxplot. The original author used linear data; copying this onto log2 data gives 0 differential genes in this page’s test. When copying, replace this line with the rule above.
Troubleshooting
Find your error message or symptom below. Entries follow the order of the steps and roughly how often each problem comes up. Each entry gives the cause and the fix, with code attached where it can be copied directly.
Error: The downloaded content looks like an HTML page, not GEO data
Cause: Since late August 2026 NCBI returns a reCAPTCHA verification page (HTTP 200) for scripted requests under /geo/, and GEOquery parses it as data.
Fix: Since 2026-09-02 NCBI has opened SOFT text and file download links that include acc. If the error persists, use getGPL = FALSE, or download from FTP and read with filename. FTP is not affected.
Error: The size of the connection buffer (131072) was not large enough
Cause: vroom uses a default buffer that is too small for very long lines.
Fix: Set a larger buffer before calling getGEO, and increase it again if needed. The CSDN post on this error is in the reference section.
Sys.setenv(VROOM_CONNECTION_SIZE = 500000L)Error: Timeout of 60 seconds was reached, or the downloaded file is incomplete
Cause: R’s default download timeout is 60 seconds, so large files are cut off partway.
Fix: Set options(timeout = 600). If it still fails, resume with wget -c; see the manual download method in the choices section.
Every restart of R downloads the file again
Cause: The default cache folder is tempdir(), which is deleted when R closes.
Fix: Add destdir to getGEO to point to a fixed folder. When you see Using locally cached version, the cached copy is being read.
Cannot connect to the server
Cause: The default download method does not work with the current network.
Fix: Set options('download.file.method.GEOquery' = 'libcurl') and try again. If it still fails, download the file manually.
A gene is missing from the annotation package, or its name differs from the GPL table
Cause: Annotation packages are newer than GPL tables, and genes have been renamed; for example, FAM122C is now PABIR3. Mixing two sources makes the gene names not match.
Fix: Choose one annotation source (hgu133plus2.db in this example), state the package name and version in the methods, and do not mix sources.
The probe or gene count after annotation is far off from what the platform should give
Cause: The wrong platform was chosen, or the matrix row names are not probe IDs (for example ILMN_ IDs on Illumina platforms).
Fix: Check the platform ID with annotation(eset), and use head(rownames(ex)) to confirm the row names match the probe IDs in the annotation package.
The Gene Symbol column contains " /// ", meaning one probe maps to several genes
Cause: The GPL table has 2796 such probes and hgu133plus2.db has 1528. Community posts often keep the first gene name, and AnnoProbe’s source code admits this handling is a problem.
Fix: Remove these probes before the differential analysis, as in the step 4 code.
Thousands of differential genes, with a very large median |logFC|
Cause: Linear values were analysed by limma without log2. For GSE16515 this gives 2804 up and 3868 down genes, with median |logFC| of 4.51.
Fix: Apply the step 3 rule; when the result is TRUE, take log2 before running limma.
ex[ex <= 0] <- NaN
ex <- log2(ex)Data already on log2 scale gives almost no differential genes
Cause: A template runs log2(dat + 1) unconditionally, which compresses fold changes to almost zero on data that is already log2. GSE15471 (already log2) gave 0 differential genes when processed this way; correct processing gives 2306 up and 348 down.
Fix: Delete the unconditional log2 line and use the step 3 rule instead. The source of the template is in the reference section.
A NaN warning appears when taking log2
Cause: The data contain negative values or many zeros, which is common for MAS5 background-subtracted data or for Illumina data that were not log-transformed. Taking log2 directly produces NaN.
Fix: First count the negative values and zeros. If the data cannot be log2-transformed, use the raw CEL files and reprocess with RMA; see the choices section.
Thousands of differential genes drop to 0 after grouping
Cause: The group vector was taken from a re-sorted pData, so it no longer matches the matrix columns. In GSE16515, 18 of the 52 labels were misaligned, and the differential genes fell from 1535 to 0.
Fix: Do not sort pData, and do not build groups after merging with an external table. Check the sample order with stopifnot(identical(rownames(pd), colnames(ex_gene))).
All samples are assigned to one group, for example all as tumor
Cause: Groups were judged from source_name or title. In GSE15471 every source_name is pancreas, so all 78 samples would be called tumor; the groups are actually in the sample:ch1 column.
Fix: Find the group column with grep(":ch1$", colnames(pd)), then check the sample count of each group with table().
The sign of logFC is opposite to what you expected
Cause: Factor levels are sorted alphabetically by default. With control and case, case comes first and the logFC direction is reversed. Also, GEO2R has treated the group defined first as the test group since November 2020, the opposite of before, so logFC signs in old tutorial screenshots may be reversed.
Fix: Set the reference level explicitly. When comparing with old tutorial screenshots, follow the current GEO2R rule.
group <- factor(group, levels = c("Normal", "Tumor"))The sample count does not match the paper, for example 78 instead of 36
Cause: The GSE may contain several platforms, or cell lines and technical replicates. The 78 samples in GSE15471 come from 36 patients, and the 6 samples with _rep in the title are repeat arrays from 3 patients.
Fix: First check whether length(gse) is greater than 1. Do not treat repeat arrays as independent samples; merge them by patient and tissue with avereps, or use duplicateCorrelation with patient as the block. See the choices section for the code.
All 50 genes in the heatmap are up-regulated
Cause: Taking the top 50 by adj.P picks up-regulated genes, which are more numerous and more significant in this dataset, so all 50 fall on the up side.
Fix: Take 25 up-regulated and 25 down-regulated genes so both directions are shown. The code is in step 7.
After merging, there are unusually many differential genes and P values are much smaller than expected
Cause: Correcting the matrix with ComBat or removeBatchEffect before limma overstates the residual degrees of freedom, so P values are too small. The removeBatchEffect help page says it is for plotting and exploration only.
Fix: Put the batch in the limma design matrix and run the analysis on the original matrix; use the corrected matrix only for PCA and heatmaps.
design <- model.matrix(~ group + batch)After ComBat, the differences between groups shrink noticeably
Cause: Without mod, ComBat also removes group differences that overlap with the batch. In this page’s balanced design, 86 fewer up-regulated genes were found. When batch and group are unbalanced, ComBat with mod can still make downstream tests overconfident (Nygaard et al., 2016); in this page’s unbalanced case, 5 permutations of group labels gave 0 to 1 false positive genes after ComBat.
Fix: If you must use ComBat, include mod = model.matrix(~ group). For differential analysis, prefer putting the batch in the design matrix.
m_cb <- ComBat(m, batch = batch, mod = model.matrix(~ group))table(batch, group) shows that one dataset contains only one group
Cause: Batch and group are completely confounded, so no method can separate them. All group differences then come from differences between datasets, so this kind of merged analysis is not valid. Reviewers can spot this from table(batch, group).
Fix: Do not merge these datasets for comparison; use datasets that each contain both groups.
getRNASeqData fails with invalid 'row.names' length
Cause: The download link for the human annotation file Human.GRCh38.p13.annot.tsv.gz has no acc parameter and is still blocked by reCAPTCHA, so GEOquery reads a verification HTML page. Measured on 2026-10-11 with GEOquery 2.74.0.
Fix: The raw counts matrix itself can be downloaded. Annotate it with org.Hs.eg.db using Entrez IDs; in this example 37692 of 39376 GeneIDs map to gene symbols.
sym <- mapIds(org.Hs.eg.db, rownames(cnt), "SYMBOL", "ENTREZID")Only FPKM or TPM files are available, and you want to run a differential analysis directly
Cause: DESeq2, edgeR and limma-voom all need integer counts. FPKM and TPM are not suitable for comparing across samples.
Fix: First check with hasRNASeqQuantifications() whether NCBI generated raw counts; if not, download the fastq files from SRA and quantify again. With only TPM, log2(TPM + 1) and limma-trend can be used only for exploration.
GDCdownload fetches twice the expected number of files
Cause: GDCquery was run without data.type, so the results include two data types.
Fix: Add data.type = "Gene Expression Quantification".
GDCquery finds no HTSeq data, or the legacy argument fails
Cause: Since Data Release 32 (2022-03-29), GDC has removed HTSeq counts and kept only STAR - Counts. The Legacy Archive was retired on 2023-05-04, and old hg19 code no longer works.
Fix: Set workflow.type to "STAR - Counts" and do not use legacy = TRUE.
GDCprepare fails with: Column `disease_response` doesn't exist
Cause: A known problem in TCGAbiolinks 2.37.x and 2.38.0 for some projects. Users on GitHub issues report that version 2.34 works.
Fix: Go back to version 2.34, or download the organised GDC TCGA expression and clinical matrices from UCSC Xena.
GDCquery_clinic no longer works
Cause: A change to the GDC interface in early 2025 broke this function, and the related issue is still open.
Fix: The maintainers suggest installing the devel branch of TCGAbiolinks from GitHub.
GDCdownload fails with Error in if (ret == 1) break
Cause: A known TCGAbiolinks problem; issue #656 is still unresolved.
Fix: Use method = "client" to download through the GDC Data Transfer Tool. Download in chunks with files.per.chunk = 20; after a failure, rerunning skips files that are already downloaded.
geoChina prints: Your GSE may not be expression by array
Cause: The GSE list in the AnnoProbe mirror has not been updated since November 2022, so newer GSEs are not in it.
Fix: Download with getGEO directly; see step 2.
References
- NCBI GEO: About GEO2R — log2 auto-detection, the November 2020 group-order change, circular contrasts, 10-minute timeout, DESeq2 for RNA-seq
- NCBI GEO: NCBI-generated RNA-seq count data — Count pipeline, species scope, file types and limitations
- GEOquery GitHub issue #230: NCBI reCAPTCHA blocks programmatic access — The block from August 2026, affected endpoints, NCBI's 2026-09-02 exemptions and the getGPL = FALSE workaround
- GEOquery vignette: RNA-seq quantifications from GEO — Using hasRNASeqQuantifications and getRNASeqData
- limma help pages: topTable, treat, removeBatchEffect — Fold-change filtering not recommended, treat instead; removeBatchEffect not for preparing data for lmFit
- Biostars: GEO2R script for analysing microarray data — The log2 check in the GEO2R script
- Nygaard V, Rødland EA, Hovig E. Methods that remove batch effects while retaining group differences may lead to exaggerated confidence in downstream analyses. Biostatistics, 2016 — ComBat in unbalanced designs
- Li Y, et al. Exaggerated false positives by popular differential expression methods when analyzing human population samples. Genome Biol, 2022 — False positives from DESeq2 and edgeR in large RNA-seq studies; the Wilcoxon test
- Law CW, et al. voom: precision weights unlock linear model analysis tools for RNA-seq read counts. Genome Biol, 2014 — The limma-voom method
- GDC: Why did GDC remove HTSeq gene expression quantification — HTSeq removal and STAR - Counts
- GDC: Legacy Archive retires — Legacy Archive retired on 2023-05-04
- TCGAbiolinks GitHub issues #639, #656, #657 — Current problems and workarounds for GDCquery_clinic, GDCdownload and GDCprepare
- AnnoProbe source (geoChina.R, filterEM.R) — Community source: mirror address and probe de-duplication rules
- CSDN: fixing the getGEO connection buffer error — Community post: original error and VROOM_CONNECTION_SIZE setting, with the author's run output
- CSDN (Shengxin Baidu): probe-to-gene annotation for GEO array platforms — Community post: reading GPL files and handling " /// "
- CSDN repost of a Jianshu post: multi-group differential analysis with limma — Community post: equivalence of two design-matrix forms; counter-example of unconditional log2 in a template
- GSE16515 (pancreatic cancer, GPL570) — Dataset run for this page
- GSE15471 (pancreatic cancer, GPL570) — Dataset used for merging and validation on this page