Applied machine learning and neural networks

Topic outline

• General
  

  General

  • Course assessment plan File
  • Kaggle.com URL
• Topic 1. Introduction to Machine learning. History
  

  Topic 1. Introduction to Machine learning. History

  Cybernetics and the first neural networks
  The first digital computers
  The emergence of expert systems
  Probabilistic approach
  Random Forest
  Recent achievements

  • Slides File
  • Lab 1: Introduction to Data Analysis in Python Assignment
• Topic 2. General concepts
  

  Topic 2. General concepts

  Machine learning problem
  Classification and regression
  Features and their types
  Model and training method. Loss functional
  Probabilistic statement of the training problem. The maximum likelihood principle and its relation to empirical risk minimization.  Decision function
  Overfitting and the generalizing ability of an algorithm
  

  • Slides File
  • Lab2. General concepts Assignment
• Topic 3. Distance-based algorithms
  

  Topic 3. Distance-based algorithms

  Generalized algorithm
  Examples: Nearest Neighbor, Parsen Window method
  Margin
  The Curse of Dimensionality
  Choosing a distance
  Distance-based regression

  • Slides File
  • Lab3. Distance-based algorighms Assignment
• Topic 4. Bayesian approach
  

  Topic 4. Bayesian approach

  Bayesian approach

  Probability density estimation

       - nonparametric
       - parametric

  • Slides File
  • Lab4. Probability Density Estimation Assignment
• Topic 5. Linear classification algorithms
  

  Topic 5. Linear classification algorithms

  Linear regression
  Logistic regression
  Margins for a linear classifier
  One hot encoding
  ROC and AUC
  SVM

  • Slides File
  • Lab5. Logistic regression. ROC. AUC. One-hot encoding Assignment
• Topic 6. Neural Networks
  

  Topic 6. Neural Networks

  Linear perceptron
  Stochastic gradient descent method
  Neural nets structure
  Backpropagation
  Objective functions for regression, binary/multiclass/multilabel classification
  Activation functions
  Neural nets design

  • Slides File
  • Lab 6. Neural Networks Basics Assignment
  • Lab 7. Neural Nets Advanced Assignment
  • Lab 8. Pretrained Neural Nets Assignment
• Topic 7. Rule-based learning
  

  Topic 7. Rule-based learning

  The concept of rule

  The criterion for the rule quality

  Finding rules

  Classification algorithms based on rules

  • Slides File
• Topic 8
  

  Topic 8
• Topic 9
  

  Topic 9
• Topic 10
  

  Topic 10