Neural Networks and Deep Learning

Description

This course will give an introduction to the concept of Neural Networks (NN) and Deep Learning. Topics covered will include: NN building blocks, including concepts such as neurons, activation functions, loss functions, gradient descent and back-propagation; Convolutional Neural Networks; Recursive Neural Networks; Autoencoders; and best practices when designing NNs,

Learning outcomes

Students that complete this course will be able to:

ML ‘philosophy’/variants

• Distinguish between the concepts of “Artificial Intelligence”, “Machine Learning”, “Neural Networks”, “Deep Learning”
• Distinguish between different types of learning (e.g. supervised, unsupervised, reinforcement) and recognise which applies to their own problem
• Distinguish between linear and non-linear approaches and recognise which is best suited for application to their own problem

NN building blocks

• Describe what a feed-forward neural network (FFNN) is, along with its components (neurons, layers, weights, bias, activation functions, cost functions)
• Explain how training of a FFNN works from a mathematical point of view (gradient descent, learning rate, backpropagation)
• Execute with pen and paper a few steps of training of a very simple FFNN model
• Tell the difference between a shallow and a deep network

Network architectures

• Explain broadly how different NN architectures (feed-forward, convolutional, recurrent, autoencoders, etc) are wired and how they work
• Apply the most appropriate architecture to a given problem/dataset
• Implement different architectures in python with Keras and train them on given datasets
• Analyze training curves and prediction outputs to evaluate if the training has been successful
• Debug possible issues with the training and suggest changes to fix them

Good practices in project design

• Explain the difference between training, validation and testing
• Explain what overfitting is from a mathematical point of view, and what issues it causes
• Apply the right tools to curb/fix overfitting issues
• Identify what constitutes good practices of dataset design and how to avoid introducing information leakage or other biases when building your own datasets

Pre-requisites

Required prior knowledge

Programming with Python

• The course will be taught using Python. So you will need to have experience with python programming e.g. having attended the NBIS workshop Introduction to Python or equivalent

Statistics and Machine Learning

• You need to have previous experience in the fields of Statistics and/or Machine Learning e.g. having attended the NBIS Biostatistics and Machine Learning Workshop or equivalent

Command line on UNIX/Linux/MacOSX/Windows

• Regardless of your operative system, you need to:
  • Have a version of Anaconda/Miniconda installed on your system
  • Be able to use Anaconda prompt (Windows users only)
  • Be able to launch scripts and software from command line
  • Unix, Linux and MacOSX already have a terminal app installed (on MacOSX, you can find it by searching for "Terminal" in the launch pad).
    • On Windows, you may use, e.g., PowerShell... TBA

Recommended prior knowledge

Jupyter Notebooks (recommended)

• While we will give a brief introduction to Jupyter Notebooks, you will benefit more if you get acquainted with using these notebooks in advance.
  • Check this tutorial https://www.dataquest.io/blog/jupyter-notebook-tutorial if you don't know where to begin

Anaconda/Miniconda (recommended)

• Necessary packages and softwares needed to run the course practical exercises will be installed through conda. We recommend you know how to install/uninstall packages and environments under conda before the course.
  • If you don't know where to begin, check this tutorial https://conda.io/projects/conda/en/latest/user-guide/getting-started.html

Level

beginner

Course leaders

Claudio Mirabello (course leader)

Christophe Avenel (course leader)

Bengt Sennblad, Marcin Kierczak, Per Unneberg

Please use course email for contacts: edu.neural-nets-deep-learning@nbis.se