Chapter 12 sections from Deep Learning with PyTorch.
7 items
Hyperparameter optimization begins by deciding what may vary.
Grid search is one of the simplest methods for hyperparameter optimization.
Random search is a hyperparameter optimization method that samples configurations at random from a search space.
Bayesian optimization is a hyperparameter optimization method for expensive black-box functions. It is useful when each training run costs enough that random search wastes too much compute.
Population-based training, or PBT, is a hyperparameter optimization method that trains many models at the same time.
Neural architecture search, or NAS, is the process of automatically searching for model architectures.
Automated machine learning, or AutoML, refers to systems that automate parts of the model development process.