Statistics
494 ideas in this subject. The map below lists the main ideas of the subject by area, in teaching order. 244 of its 245 ideas have a page so far.
Start here
- ConceptMathematical statistics
Mathematical statistics is the part of mathematics that studies how to learn about an unknown probability distribution from data. It models data as the values of random variables whose law belongs to a statistical model. It then studies procedures such as estimators, tests and confidence sets through their behaviour under each law of the model. It gives the rigorous basis for statistical inference.
- ConceptStatistical model
A statistical model is a set of probability distributions that are candidates for the law of the observed data. It fixes what is assumed about the data before any inference is made. Every method of statistical inference starts from a model, and its guarantees hold only under the laws in that model.
- ConceptParametric model
A parametric model is a statistical model whose laws are described by finitely many real numbers, the parameters. Inference in such a model reduces to learning a point of a subset of . Most classical theory, such as maximum likelihood and the Cramér–Rao bound, is developed for parametric models.
- ConceptIdentifiability
A statistical model is identifiable when different parameter values give different laws of the data. Then the law of the data determines the parameter, and it makes sense to estimate the parameter. Without identifiability no amount of data can separate parameters that give the same law, and consistent estimation of them is impossible.
- ConceptStatistical inference
Statistical inference is the process of drawing conclusions about an unknown probability distribution from data generated by it. It works inside a statistical model and produces estimates, confidence sets, tests or predictions. The two main schools are frequentist inference and Bayesian inference.
- ConceptStatistic
A statistic is a quantity computed from a sample, by a fixed rule that does not depend on unknown parameters. Examples are the sample mean and the sample maximum. The node statistic gives the precise definition.
Map of the subject
Foundations of statistics 13 of 13
- Mathematical statistics concept
- Statistical model definition
- Parametric model definition
- Identifiability definition
- Statistical inference concept
- Statistic definition
- Sampling distribution definition
- Sample mean and sample covariance definition
- Order statistic definition
- Empirical distribution function definition
- Likelihood function definition
- Likelihood principle statement
- Frequentist inference concept
Bayesian inference 11 of 11
- Bayesian inference concept
- Prior probability definition
- Posterior probability definition
- Conjugate prior definition
- Jeffreys prior definition
- Credible interval definition
- Bayes factor definition
- Lindley's paradox counterexample
- Bernstein–von Mises theorem statement
- Empirical Bayes method concept
- Bayesian hierarchical model definition
Sufficiency and information 15 of 15
- Sufficient statistic definition
- Fisher–Neyman factorization theorem statement
- Minimal sufficient statistic definition
- Complete statistic definition
- Ancillary statistic definition
- Basu's theorem statement
- Rao–Blackwell theorem statement
- Lehmann–Scheffé theorem statement
- Exponential family definition
- Pitman–Koopman–Darmois theorem statement
- Fisher information definition
- Observed information definition
- Cramér–Rao bound statement
- Hellinger distance definition
- Total variation distance between probability measures definition
Statistical decision theory 14 of 14
- Statistical decision theory concept
- Loss function definition
- Risk function definition
- Bayes estimator definition
- Bayes risk definition
- Minimax estimator definition
- Bayes risk is at most minimax risk statement
- Admissible decision rule definition
- Complete class theorem statement
- Stein's example counterexample
- James–Stein estimator definition
- Shrinkage estimator concept
- Stein's lemma statement
- Stein's unbiased risk estimate statement
Sampling theory 4 of 4
- Simple random sample definition
- Stratified sampling concept
- Horvitz–Thompson estimator definition
- Missing data mechanisms concept
Distribution theory 9 of 9
- Student's t-distribution definition
- F-distribution definition
- Noncentral chi-squared distribution definition
- Wishart distribution definition
- Hotelling's T-squared distribution definition
- Cochran's theorem statement
- Independence of sample mean and sample variance for normal samples statement
- Delta method statement
- Edgeworth expansion statement
Point estimation 21 of 21
- Estimator definition
- Point estimation concept
- Bias of an estimator definition
- Mean squared error definition
- Bias–variance decomposition statement
- Consistent estimator definition
- Efficiency of an estimator definition
- Minimum-variance unbiased estimator definition
- Method of moments concept
- Maximum likelihood estimation concept
- Consistency of the maximum likelihood estimator statement
- Asymptotic normality of the maximum likelihood estimator statement
- Score definition
- M-estimator definition
- Estimating equations concept
- Generalized method of moments concept
- Expectation–maximization algorithm concept
- Hodges' superefficient estimator counterexample
- Local asymptotic normality definition
- Hájek–Le Cam convolution theorem statement
- Contiguity of measures definition
Interval estimation 5 of 5
- Confidence interval definition
- Pivotal quantity definition
- Neyman construction concept
- Binomial proportion confidence interval definition
- Behrens–Fisher problem problem
Hypothesis testing 20 of 21
- Statistical hypothesis test concept
- Statistical hypothesis test definition
- Type I and type II errors definition
- Power of a test definition
- P-value definition
- Neyman–Pearson lemma statement
- Uniformly most powerful test definition
- Monotone likelihood ratio definition
- Karlin–Rubin theorem statement
- Likelihood-ratio test concept
- Wilks' theorem statement
- Score test concept
- Wald test concept
- Student's t-test concept
- F-test concept
- Chi-squared test concept
- Fisher's exact test concept
- multiple comparisons problem problem
- Family-wise error rate definition
- Bonferroni correction concept
- False discovery rate and the Benjamini–Hochberg procedure concept
Model selection 4 of 4
- Akaike information criterion definition
- Bayesian information criterion definition
- Minimum description length concept
- Cross-validation concept
Nonparametric inference 24 of 24
- Nonparametric statistics concept
- Kernel density estimation concept
- Kernel regression concept
- Local regression concept
- Smoothing spline definition
- Kolmogorov–Smirnov test concept
- Dvoretzky–Kiefer–Wolfowitz inequality statement
- Cramér–von Mises criterion concept
- Anderson–Darling test concept
- Sign test concept
- Wilcoxon signed-rank test concept
- Mann–Whitney U test concept
- Kruskal–Wallis test concept
- Spearman's rank correlation coefficient definition
- Kendall rank correlation coefficient definition
- Permutation test concept
- Bootstrap concept
- Jackknife concept
- U-statistic definition
- L-estimator definition
- Empirical likelihood concept
- Isotonic regression concept
- Quantile regression concept
- Pickands–Balkema–De Haan theorem statement
Robust statistics 5 of 5
- Robust statistics concept
- Breakdown point definition
- Influence function definition
- Huber loss definition
- Hodges–Lehmann estimator definition
Multivariate analysis 24 of 24
- Correlation coefficient definition
- Partial correlation definition
- Mahalanobis distance definition
- Principal component analysis concept
- Factor analysis concept
- Canonical correlation concept
- Linear discriminant analysis concept
- Multivariate analysis of variance concept
- Independent component analysis concept
- Multidimensional scaling concept
- Copula definition
- Sklar's theorem statement
- Contingency table definition
- Simpson's paradox counterexample
- Mixture model definition
- Latent variable model definition
- Cluster analysis concept
- k-means clustering concept
- Graphical model concept
- Bayesian network definition
- Markov random field definition
- Hammersley–Clifford theorem statement
- Directional statistics concept
- von Mises–Fisher distribution definition
Linear inference and regression 21 of 21
- Linear regression concept
- Ordinary least squares concept
- Gauss–Markov theorem statement
- Coefficient of determination definition
- Frisch–Waugh–Lovell theorem statement
- Generalized least squares concept
- Weighted least squares concept
- Ridge regression concept
- Lasso concept
- Elastic net concept
- Analysis of variance concept
- General linear model definition
- Mixed model definition
- Generalized linear model definition
- Logistic regression definition
- Poisson regression definition
- Probit model definition
- Nonlinear regression concept
- Errors-in-variables model definition
- Instrumental variables estimation concept
- Heteroscedasticity definition
Design of experiments 8 of 8
- Design of experiments concept
- Randomization concept
- Blocking concept
- Factorial experiment concept
- Fractional factorial design concept
- Optimal experimental design concept
- Kiefer–Wolfowitz equivalence theorem statement
- Response surface methodology concept
Sequential statistics 8 of 8
- Sequential analysis concept
- Sequential probability ratio test concept
- Wald–Wolfowitz optimality of the sequential probability ratio test statement
- Wald's equation statement
- Stochastic approximation concept
- Robbins–Monro algorithm concept
- CUSUM concept
- Change detection problem
Inference for stochastic processes and time series 20 of 20
- Time series concept
- Autocorrelation function definition
- Partial autocorrelation function definition
- Autoregressive model definition
- Yule–Walker equations statement
- Moving-average model definition
- Autoregressive–moving-average model definition
- Autoregressive integrated moving average definition
- Box–Jenkins method concept
- Wold's decomposition theorem statement
- Periodogram definition
- Spectral density estimation concept
- Whittle likelihood definition
- Dickey–Fuller test concept
- Cointegration definition
- Autoregressive conditional heteroskedasticity definition
- Baum–Welch algorithm concept
- Particle filter concept
- Kriging concept
- Variogram definition
Survival analysis and censored data 9 of 9
- Survival analysis concept
- Censoring definition
- Survival function definition
- Hazard function definition
- Kaplan–Meier estimator definition
- Nelson–Aalen estimator definition
- Proportional hazards model definition
- Log-rank test concept
- Accelerated failure time model definition
Causal inference 2 of 2
- Causal inference concept
- Rubin causal model definition
Statistical learning 3 of 3
- Statistical learning theory concept
- Empirical risk minimization concept
- Representer theorem statement
Statistics on topological and algebraic structures 4 of 4
- Algebraic statistics concept
- Topological data analysis concept
- Fréchet mean definition
- Functional data analysis concept
More ideas in this subject
Concepts 21
- Kullback–Leibler divergence
- Cumulant
- Persistent homology
- Hypergeometric function of a matrix argument
- Propagation of uncertainty
- Numerical methods for linear least squares
- Standard score
- Dirichlet process
- Joint probability distribution
- Empirical process
- Marginal distribution
- Stochastic difference equation
- Covariance matrix
- Multivariate normal distribution
- Information geometry
- Hidden Markov model
- Skewness
- Kurtosis
- Sample mean
- Sample variance
- Chi-squared distribution
Theorems and results 9
Examples 5
Problems 1
Definitions 215
- Design of experiments
- Factorial experiment
- Loss function
- Blocking
- Causal inference
- Censoring
- Cluster analysis
- Coefficient of determination
- Contingency table
- Directional statistics
- Nonparametric statistics
- Accelerated failure time model
- Admissible decision rule
- Autoregressive integrated moving average
- Elastic net
- Factor analysis
- Fractional factorial design
- Huber loss
- Independent component analysis
- k-means clustering
- Kullback–Leibler divergence
- Linear discriminant analysis
- Mahalanobis distance
- Minimax estimator
- Nelson–Aalen estimator
- Periodogram
- Rubin causal model
- Simple random sample
- Statistical decision theory
- Stratified sampling
- Survival function
- Topological data analysis
- Weighted least squares
- Algebraic statistics
- Autocorrelation function
- Autoregressive conditional heteroskedasticity
- Autoregressive model
- Autoregressive–moving-average model
- Baum–Welch algorithm
- Bayesian network
- Canonical correlation
- Copula
- Cumulant
- Exponential family
- Fréchet mean
- General linear model
- Graphical model
- Hazard function
- Heteroscedasticity
- James–Stein estimator
- Kaplan–Meier estimator
- Latent variable model
- Markov random field
- Mixed model
- Mixture model
- Moving-average model
- Partial correlation
- Persistent homology
- Proportional hazards model
- Spectral density estimation
- Survival analysis
- von Mises–Fisher distribution
- Analysis of variance
- Cointegration
- CUSUM
- Errors-in-variables model
- Fisher information
- Hellinger distance
- Hypergeometric function of a matrix argument
- Linear regression
- Multidimensional scaling
- Optimal experimental design
- Partial autocorrelation function
- Probit model
- Risk function
- Statistical learning theory
- Total variation distance between probability measures
- Instrumental variables estimation
- Generalized least squares
- Propagation of uncertainty
- Local regression
- Empirical risk minimization
- Lasso
- Ridge regression
- Ordinary least squares
- Principal component analysis
- Nonlinear regression
- Response surface methodology
- Smoothing spline
- Statistical model
- Parametric model
- Minimum description length
- Randomization
- Identifiability
- Numerical methods for linear least squares
- Kendall rank correlation coefficient
- Standard score
- Dirichlet process
- Isotonic regression
- Pivotal quantity
- Joint probability distribution
- Time series
- Empirical process
- Marginal distribution
- Robbins–Monro algorithm
- Stochastic difference equation
- Kriging
- Particle filter
- Bayes risk
- Bayesian hierarchical model
- Functional data analysis
- Variogram
- Shrinkage estimator
- Covariance matrix
- Quantile regression
- Multivariate normal distribution
- Stochastic approximation
- Bayes estimator
- Statistic
- Estimator
- Order statistic
- Ancillary statistic
- Empirical distribution function
- Information geometry
- Kernel regression
- Sampling distribution
- Type I and type II errors
- Consistent estimator
- Cross-validation
- F-test
- Family-wise error rate
- Kernel density estimation
- L-estimator
- Mann–Whitney U test
- Power of a test
- Sequential analysis
- Statistical inference
- Uniformly most powerful test
- Correlation coefficient
- Hodges–Lehmann estimator
- Jeffreys prior
- Kolmogorov–Smirnov test
- Anderson–Darling test
- Cramér–von Mises criterion
- Mathematical statistics
- Contiguity of measures
- Hidden Markov model
- Log-rank test
- P-value
- U-statistic
- Bias of an estimator
- Complete statistic
- Dickey–Fuller test
- Likelihood function
- Multivariate analysis of variance
- Spearman's rank correlation coefficient
- False discovery rate and the Benjamini–Hochberg procedure
- Horvitz–Thompson estimator
- Influence function
- Missing data mechanisms
- Sequential probability ratio test
- Sufficient statistic
- Wilcoxon signed-rank test
- Bonferroni correction
- Empirical Bayes method
- Mean squared error
- Minimal sufficient statistic
- Monotone likelihood ratio
- Permutation test
- Robust statistics
- Sign test
- Skewness
- Minimum-variance unbiased estimator
- Local asymptotic normality
- Bootstrap
- Kurtosis
- Score
- Sample mean
- Bayes factor
- Fisher's exact test
- Sample variance
- Breakdown point
- Jackknife
- Credible interval
- Sample mean and sample covariance
- Conjugate prior
- Confidence interval
- Akaike information criterion
- Bayesian information criterion
- M-estimator
- Likelihood-ratio test
- Neyman construction
- Point estimation
- Efficiency of an estimator
- Logistic regression
- Poisson regression
- Chi-squared distribution
- Estimating equations
- Expectation–maximization algorithm
- Generalized linear model
- Binomial proportion confidence interval
- Frequentist inference
- Whittle likelihood
- Student's t-distribution
- F-distribution
- Kruskal–Wallis test
- Noncentral chi-squared distribution
- Score test
- Empirical likelihood
- Observed information
- Generalized method of moments
- Wald test
- Hotelling's T-squared distribution
- Wishart distribution
- Box–Jenkins method