Consider two population distributions that follow a normal distribution. Both have the

Samples are called significantly different, if their p value is below a certain significance threshold ($\alpha$ level). While "the lower the better", a common threshold is a p value of 0.05 or 0.01. In the presented case in average $10,000 * 0.05 = 500$ tests are

Given the conditions described in the first chapter, the FDR of this experiment with a p value threshold of 0.05 is 0.173. Out of the 2019 reported significant comparisons, in average 350 are expected to be false positives, which gives an straight forward interpretation of the data confidence. In real-world experiments the proportion of null tests and tests deriving from an actual difference is of course unknown. **The proportion of null tests however tends to be distributed equally in the p value histogram.** By identification of the average null frequency, a proportion of FP and TP can be determined and the FDR can be defined. This frequency estimate is called $\pi_0$, which leads to an FDR definition of:

###q value Consequently for each presented p value a corresponding FDR can be calculated. The minimum local FDR at each p value is called q value. $$\hat q(p_i) = min_{t \geq p_i} \hat{FDR}(p_i)$$ Since the q value is not monotonically increasing, it is smoothed by assigning the lowest FDR of all p values, that are equal or higher the current one. **By defining $\pi_0$, all other parameters can be calculated from the given p value distribution to determine the all q values.** The most prominent FDR-controlling method is known as Benjamini-Hochberg correction. It sets $\pi_0$ as 1, assuming that all features are null. In studies with an expected high proportion of true positives, a $\pi_0$ of 1 is too conservative, since there definitely are true positives in the data. A better estimation of $\pi_0$ is given in the following:

By performing t tests for all comparisons 3743 (38 %) of the genes lead to a pvalue lower than 0.05. By eye, you would estimate $\pi_0$ as 0.4, indicating, only a small fraction of the genes are unaltered (null). After q value calculations, you would filter for a specific FDR (e.g. 0.05) and end up with an p value threshold of 0.04613, indicating a FDR of max. 0.05 in the final reported 3642 genes. ```no-highlight pi0 = 0.4 m = 9856 D(p) = number of sig. tests at given p FP(p) = p*0.4*9856 FDR(p) = FP(p) / D(p) ``` FDR(0.04613) = 0.4995

###The automatic detection of $\pi_0$ is facilitated as follows For a range of $\lambda$ in e.g. $\{0.0 .. 0.05 .. 0.95\}$, calculate $\hat \pi_0 (\lambda) = \frac {\#[p_j > \lambda]}{m(1 - \lambda)}$ *) let pi0Est = [|0. .. 0.05 .. 0.95|] |> Array.map (fun lambda -> let num = examplePVals |> Array.sumBy (fun x -> if x > lambda then 1. else 0.) let den = float examplePVals.Length * (1. - lambda) lambda, num/den ) (**

The resulting diagram shows, that with increasing $\lambda$ its function value $\hat \pi_0(\lambda)$ tends to $\pi_0$. The calculation

For an $\lambda$, range of $\{0.0 .. 0.05 .. 0.95\}$ the bootstrapping method determines either 0.8 or 0.85 as optimal $\lambda$ and therefore $optimal \hat \pi_0$ is either $0.3703$ or $0.3686$. The