Course Outline
DAY 1 - ARTIFICIAL NEURAL NETWORKS
Introduction and ANN Structure.
- Biological neurons versus artificial neurons.
- The structure of an Artificial Neural Network.
- Activation functions commonly used in ANNs.
- Typical classes of network architectures.
Mathematical Foundations and Learning mechanisms.
- Reviewing vector and matrix algebra.
- Understanding state-space concepts.
- Core concepts of optimization.
- Error-correction learning.
- Memory-based learning approaches.
- Hebbian learning.
- Competitive learning.
Single layer perceptrons.
- The structure and learning process of perceptrons.
- Introduction to pattern classifiers and Bayes' classifiers.
- Utilizing perceptrons as pattern classifiers.
- The convergence of perceptrons.
- Limitations inherent to perceptrons.
Feedforward Artificial Neural Networks.
- Structures of multi-layer feedforward networks.
- The backpropagation algorithm.
- Training and convergence in backpropagation.
- Functional approximation using backpropagation.
- Practical considerations and design issues in backpropagation learning.
Radial Basis Function Networks.
- Pattern separability and interpolation techniques.
- Regularization Theory.
- The relationship between regularization and RBF networks.
- Design and training of RBF networks.
- Approximation properties of RBF networks.
Competitive Learning and Self-organizing Artificial Neural Networks.
- General clustering procedures.
- Learning Vector Quantization (LVQ).
- Competitive learning algorithms and architectures.
- Self-organizing feature maps.
- Key properties of feature maps.
Fuzzy Neural Networks.
- Neuro-fuzzy systems.
- Background on fuzzy sets and logic.
- Designing fuzzy systems.
- Designing fuzzy Artificial Neural Networks.
Applications
- Discussion of various Neural Network application examples, highlighting their advantages and challenges.
DAY 2 - MACHINE LEARNING
- The PAC Learning Framework
- Guarantees for finite hypothesis sets – consistent cases
- Guarantees for finite hypothesis sets – inconsistent cases
- Generalities
- Deterministic versus stochastic scenarios
- Bayes error noise
- Estimation and approximation errors
- Model selection
- Rademacher Complexity and VC Dimension
- The bias-variance tradeoff
- Regularization
- Overfitting
- Validation techniques
- Support Vector Machines
- Kriging (Gaussian Process regression)
- Principal Component Analysis (PCA) and Kernel PCA
- Self-Organizing Maps (SOM)
- Kernel-induced vector spaces
- Mercer Kernels and kernel-induced similarity metrics
- Reinforcement Learning
DAY 3 - DEEP LEARNING
This session builds upon the topics covered on Day 1 and Day 2.
- Logistic and Softmax Regression
- Sparse Autoencoders
- Vectorization, PCA, and Whitening
- Self-Taught Learning
- Deep Networks
- Linear Decoders
- Convolution and Pooling
- Sparse Coding
- Independent Component Analysis
- Canonical Correlation Analysis
- Demonstrations and Applications
Requirements
A solid grasp of mathematics is required.
A strong understanding of basic statistics is essential.
While not mandatory, basic programming skills are recommended.
Testimonials (2)
Working from first principles in a focused way, and moving to applying case studies within the same day
Maggie Webb - Department of Jobs, Regions, and Precincts
Course - Artificial Neural Networks, Machine Learning, Deep Thinking
It was very interactive and more relaxed and informal than expected. We covered lots of topics in the time and the trainer was always receptive to talking more in detail or more generally about the topics and how they were related. I feel the training has given me the tools to continue learning as opposed to it being a one off session where learning stops once you've finished which is very important given the scale and complexity of the topic.
