Basic stats

Here we review some very very basic properties of statistics, in preparation for interviews. Let \(X\) be a random variable, that is, a measurable function from some probability space \(\Omega\) to \(\mathbb{R}\). Typically we don’t care precisely about what \(\Omega\) is. Note that the probability measure \(\mu\) on \(\Omega\) pushforwards to a probability measure \(\mu_{X}\) on \(\mathbb{R}\), which is called the distribution of \(X\). Note that the distribution itself doesn’t determine the \(X\).

The expected value \(\mbb{E}[X]\) of \(X\) is \(\int_{\mathbb{R}}x d\mu_X\). This is also called the mean. More generally, given a measureable function \(f \colon \mathbb{R} \to \mathbb{R}\), such as any continuous function, the expected value of \(\mbb{E}[f(X)]\) is \(\int_{\mathbb{R}}x d\mu_{f\circ X} = \int_{\mathbb{R}}f(x)d\mu_X\) by \(u\)-substitution.

Independence

Given two random variables \(X, Y\), \(X \times Y\) is a measureable function \(\Omega \to \mathbb{R}^2\). We say that \(X\) and \(Y\) are independent if this the probability distribution \(\mu_{X \times Y}\) is the product \(\mu_X \times \mu_Y\) on \(\mathbb{R}^2\). Intuitively, this is saying that the events of \(X\) and \(Y\) are independent of each other. This can be generealized to \(n\) variables, which is called mutually independent. We say taht \(n\) variables are pairwise independent if each pair is. Note that mutually independent implies pairwise independent, but not vice versa. There is an easy example with using just two coin tosses.

Covariance

Given two random variables \(X,Y\), the covariance \(\Cov(X,Y)\) between \(X\) and \(Y\) is \(\mbb{E}[XY]- \mbb{E}[X] \mbb{E}[Y]\). Note that when \(X\) and \(Y\) are independent the covariance is \(0\). The variance \(\Var(X)\) of a random variable \(X\) is \(\Cov(X,X) = \mbb{E}[X^2] - \mbb{E}[X]^2 = \mbb{E}[(X-\mbb{E}[X])^2]\). Its square root is the standard deviation of \(X\), typically written as \(\sigma\). We can try to approximate \(Y\) using \(X\), when \(X,Y\) are mean zero, the simplest approximation is linear (with no scalar). Let \(\beta\) be the slope that minimize \(\mbb{E}[(Y- \beta X)^2]\). Working this out, we see that \(\beta\) is \(\frac{\Cov(X,Y)}{\Cov(X,X)}\). This is mostly used in stocks, where \(X\) is the overall market prize and \(Y\) is the specific price, and \(\beta\) measures the violatility of the stock, relative to the overall market. In particular, let’s assume that \(\beta >0\), then \(\beta < 1\) implies that its less violatile than the broader market, while \(\beta > 1\) means more violatile. This also makes sense when \(X\) and \(Y\) are not mean zero, but I currently don’t have a minimalizing understanding.

Another measure is the correlation between \(X\) and \(Y\) is the correlation coefficient \(\frac{\Cov(X,Y)}{\sqrt{\Var(X)}\sqrt{\Var{Y}}}\). It takes values between \(-1\) and \(1\), and \(-1\) means perfectly inversely correlated while \(1\) means perfectly correlated. It is also called the \(R\) value.