The sampling distribution of means is normal, provided that:

1. the sample size is large enough
2. \(\mu\) and \(\sigma\) are defined for the probability density or mass function that generated the data

It represents the range of plausible value of the \(\mu\) parameter.

If you take samples repeatedly and compute the CI each time, 95% of those CIs will contain the true population mean \(\mu\).

The probability that the null hypothesis is true, or the probability that the alternative hypothesis is false.

The probability of obtaining the observed sample statistic, or some value more extreme than that, conditional on the assumption that the null hypothesis is true.

• Type I, II, M, S errors
• Power (1-Type II error) is a function of:
  • the effect size
  • the standard deviation
  • the sample size.

Null hypothesis significance testing (NHST) is only meaningful when power is high.

Examples:

• Acceptability ratings on a Likert scale
• Binary grammaticality judgements

Probability distribution \(p(Y)\):

• Probability Mass Function (PMF)
• PMF maps each possible outcome of Y to a value between [0, 1].

Examples:

• Reading times or reaction times (in ms)
• EEG signals (in microvolts)
• f0, F1, F2…

Probability distribution \(p(Y)\):

• Probability Density Function (PDF)
• PDF maps a range of values of possible outcomes of Y to a value between [0, 1].

Expectation: \(E\left[Y\right]=\sum_y y \cdot f(y)=n\theta\)

Variance: \(Var(X)=n\theta(1-\theta)\)

• Estimated \(\hat{\theta}=\dfrac{k}{n}\) (Maximum likelihood estimate of the true parameter \(\theta\))

Expectation: \(E[X]=\int xf(x)dx=\mu\)

Variance: \(Var(X)=\sigma^2\)

• Estimated \(\hat{\mu}=\bar{\mu}=\dfrac{\sum y}{n}\)
• Estimated \(\hat{\sigma}^2=\dfrac{\sum (y-\bar{y})^2}{n-1}\) (MLE estimate)

We have a joint PMF \(p_{X,Y} (x, y)\) for each possible pair of values of X and Y.

The marginal distributions: \[\begin{equation} p_{X}(x)=\sum_{y\in S_{Y}}p_{X,Y}(x,y) \end{equation}\]

The conditional distributions: \[\begin{equation} p_{X\mid Y}(x\mid y) = \frac{p_{X,Y}(x,y)}{p_Y(y)} \end{equation}\]

The joint distribution of X and Y: \[\begin{equation} \begin{pmatrix} X \\ Y \end{pmatrix}\sim \mathcal N_{2}\left(\begin{pmatrix} \mu_{X} \\ \mu_{Y} \end{pmatrix}, \begin{pmatrix} \sigma _{X}^2 & \rho_{XY}\sigma _{X}\sigma _{Y}\\ \rho_{XY}\sigma _{X}\sigma _{Y} & \sigma _{Y}^2\\ \end{pmatrix} \right) \end{equation}\]

In the variance-covariance matrix, \(\rho_{XY}\) is the correlation between X and Y; their covariance is \(Cov(X,Y) = \rho_{XY}\sigma _{X}\sigma _{Y}\).

The area under the curve sums to 1. \[\begin{equation} \iint_{S_{X,Y}} f_{X,Y}(x,y)\, dx dy = 1 \end{equation}\]

The marginal distributions:

If \(y\) is a vector of data points, we have:

Here \(f(\cdot)\) is a PDF (continuous case) or a PMF (discrete case).

We can rewrite it as follows:

Prior \(f(\theta)\): the initial probability distribution of the parameter(s), before seeing the data.

Posterior \(f(\theta \mid y)\): the probability distribution of the parameters conditional on the data.

Likelihood \(f(y \mid \theta)\): the PMF (discrete case) or the PDF (continuous case) expressed as a function of parameters \(\theta\).

Marginal Likelihood \(f(y)\): It standardizes the posterior distribution to ensure that the area under the curve of the distribution sums to 1.

If \(y\) is a vector of data points, we have:

Here \(f(\cdot)\) is a PDF (continuous case) or a PMF (discrete case).

We can rewrite it as follows:

Marginal Likelihood \(f(y)\): It functions as the “normalisating constant”.

Without it, we have: \[\begin{equation} f(\theta\mid y) \propto f(y\mid \theta) f(\theta) \end{equation}\]

• No prior knowledge
• Quantifies long-run probability of finding a false positive
• Hard cut-off decisions

• Incorporates prior knowledge
• Quantifies uncertainty around possible parameter values
• Gradual assessment of evidence

We need a distribution that can represent our uncertainty about the probability \(\theta\) of success.

The beta distribution is commonly used as prior for proportions.

Where \(B(a,b)=\int_0^1 \theta^{a-1}(1-\theta)^{b-1}\, d\theta\), a normalizing constant that ensures that the area under the curve sums to 1.

The expectation and variance of Beta PDF:

MCMC (Markov Chain Monte Carlo) sampling

• Gibbs sampling
• Random walk Metropolis
• Hamiltonian Monte Carlo (HMC, by Stan)

Bayes’ rule: \[\begin{equation} \begin{aligned} p(\Theta|Y) &= \cfrac{ p(Y|\Theta) \cdot p(\Theta) }{p(Y)}\\ p(\Theta|Y) &= \cfrac{ p(Y|\Theta) \cdot p(\Theta) }{\int_{\Theta} p(Y|\Theta) \cdot p(\Theta) d\Theta } \end{aligned} \end{equation}\]

It works when we know the closed form of the PDF we want to simulate from and can derive the inverse of that function.

1. Sample one number \(u\) from \(Unif(0,1)\). Let \(u=F(z)=\int_L^zf(x)dx\).
2. \(z=F^{-1}(u)\) is a draw from \(f(x)\).

• High-dimensional (hierarchical) models
• Only works with continuous parameters
• Based on Hamiltonian dynamics \(Energy(\theta,k)=U(\theta)+KE(k)\)

In trace plot, we see a characteristic “fat hairy caterpillar” shape, and we say that the sampler has converged.

The first few samples (warm-ups) were dropped, because the initial samples may not yet be sampling from the posterior.

We usually run four parallel chains with different initial values chosen randomly. They all end up sampling from the same distribution and overlap visually (mixing).

Suppose we have data from a single subject repeatedly pressing the space bar as fast as possible, without paying attention to any stimuli. The data are response times in milliseconds in each trial.

RQ: How long it takes to press a key for this subject?

Assumptions: \[\begin{equation} rt_n \sim \mathit{Normal}(\mu, \sigma) \end{equation}\]

chains:the number of independent runs for sampling (default: 4)

iter: the number of iterations that the sampler makes to sample from the posterior distribution of each parameter (default: 2000)

warmup: the number of iterations from the start of sampling that are eventually discarded (default: half of iter)

1. Define the model (likelihood and prior)
2. Check if priors are reasonable

• Prior predictive distribution
• Sensitivity analysis

3. Fit model(s)
4. Check if posterior prediction matches data

• Posterior predictive distribution
• \(p(D_{pred} \mid y) = \int_\Theta p(D_{pred} \mid \Theta)p(\Theta \mid y) d\Theta\)