# Probability Theory 

##### Definition

Probabilities denote the chance that an event occurs, it is not any kind of guarantee that this event will happen after a certain amount of trials.

## Terms

Name Definition
Sample Space $\Omega$ nonempty set of outputs of experiment
Sigma Algebra $\mathbb F \subseteq 2^\Omega$ set of subsets of outputs (events)
Probability $\mathop{P}: \mathbb F \mapsto [0,1]$
Random Variable $\textit{X}: \Omega \mapsto \mathbb X$ mapped subsets of $\Omega$
Observations: $x_1, \ldots, x_N$ single values of $\textit{X}$
Observation Space $\mathbb X$ possible observations of $\textit{X}$
Unknown parameter $\theta \in \Theta$ parameter of propability function
Estimator $\textit{T}: \mathbb X \mapsto \Theta$ $\textit{T}(\textit{X}) = \hat{\theta}$, finds $\hat{\theta}$ from $\textit{X}$

## Conditional Probabilty

The probability for an event $A$ given that the event $B$ already occured is

$\mathop{P}_B(A) = \mathop{P}(A|B) = \frac{\mathop{P}(A \cap B)}{\mathop{P}(B)}$

with events $A,B$.

Multiplication: $\mathop{P}(A \cap B) = \mathop{P}(A|B)\mathop{P}(B) = \mathop{P}(B|A)\mathop{P}(A)$

Bayes’ law: $\mathop{P}(B_k | A) = \frac{\mathop{P}(A | B_k)\mathop{P}(B_k)}{\sum\limits_{i \in I} \mathop{P}(A|B_i) \mathop{P}(B_i)}$

Total probabilty: $\mathop{P}(A) = \sum\limits_{i \in I} \mathop{P}(A|B_i)\mathop{P}(B_i)$

## Random Variables

$\textit{X}: \Omega \mapsto \Omega'$ is a random variable if for all events $A' \in \mathbb F'$ there exists an event $A \in \mathbb F$ such that $\left\{\omega \in \Omega|\textit{X}(\omega) \in A'\right\} \in \mathbb F$.

## Distribution

PDF: $f_{\textit{X}}(x) = \frac{\,\text{d}F_{\textit{X}}(x)}{\,\text{d}x}$ CDF: $F_{\textit{X}}(x) = \int\limits_{-\infty}^{x}{f_{\textit{X}}(\xi)\,\text{d}\xi}$