Machine Learning Topics

The following topics will be covered in this module (not necessarily in the given order).

Topic 0: Data Exploration

The most important aspect to ensure good behaving predictive models, is the data used to develop that model. This topic will explore the different types of data, requirements placed on data, data quality issues, data pre-processing, data transformation, feature selection, normalization, and sampling.

Topic 1: Introduction to Machine Learning

This is a short introduction to machine learning, with discussions on the machine learning pipeline, machine learning approaches, and different machine learning algorithms.

Topic 2: Information-based Learning

This topic covers covers machine learning approaches based on information theory. The focus is on decision trees, including classification trees, regression trees, model trees, oblique trees, and evolutionary trees.

Topic 3: Similarity-based Learning

This topic covers machine learning algorithms where predictions are made based on similarity measures. Nearest-neighbour algorithms and artificial immune systems are included in this paradigm. However, we will only explore nearest-neighbour algorithms.

Topic 4: Reinforcement Learning

This topic considers learning by reward and punishment.

Topic 5: Kernel-based Learning

Here the focus is on kernel-based approaches such as support vector machines, support vector regression, and radial basis function neural networks.

Topic 6: Probability-based Learning

This topic will be done if time permits.

The focus here is on the expectation-maximization algorithm, Gaussian micture models, naive Bayes, and Bayesian networks.

Topic 7: Error-based Learning

The focus of this topic is on parameterized predictive models where models are constructed as the result of an optimization process. These models define some error function, and values are assigned to the parameters of the model via an optimization algorithm such that the error function is minimized. We consider linear regression, logistic regression, multinomial regrees, and then shallow neural networks.

Topic 8: Unsupervised Learning

The focus of this topic is on neural network approaches where the model is constructed using unsupervised learning approaches. We will explore Hebbian learning, learning vector quantization, and the self-organizing feature map.

Topic 9: Ensemble Learning

The focus of this topic is on machine learning approaches where multiple learners are applied in a collaborative approach to solve a problem. Included in this paradigm are neural network ensembles, random forests, and boosting and bagging approaches.

Topic 10: Online Learning

This topic considers learning from data that becomes incrementally available over time.