Aug 22, 2020
fast.ai 2020 — Lesson 1
High school math is enough to understand deep learning.
Lots of data is not needed but some record-breaking results have been made with <50 items.
You don’t need an expensive computer but start of the art results can be made for free.
Deep learning is the same as deep neural networks.
The idea of this course is to approach deep learning from top to bottom. It means that first we get to see and test what DL can be used and then we start learning more details.
The biggest regret a lot of people are saying is that they wish to have spent more time coding and building models instead of learning the theory.
fastai is built top of PyTorch and these are written in Python and it’s the language we will use during this course. Many people think that fastai is just for beginners and teachers but it’s actually using layered API which makes it infinite customizable and practical for every purpose.
GPU (NVIDIA) machine is needed to support the libraries. It’s not recommended buy this kind of device but there are free and paid cloud machines available.
Forums are helpful to find information about setting up these tools.
Jupyter Notebook is coding environment often used by DL people. It’s easier to experiment things using Jupyter Notebooks than running Python code in terminal. Linux highly recommended.
When using variables in the notebook it’s taking the value from notebook’s state. This means that it doesn’t matter what the variable was defined above cell if it’s not run.
shift + enter = run cell and go to the next
By pressing h it’s possible to see all of the shortcuts.
It’s recommended to take duplication of the notebooks as you want to change them so updating them don’t overwrite your modifications.
After every section there are questionnaire which makes sure that you learned the most important things. It doesn’t matter how many you get right but it just confirms that you haven’t missed anything important. If you don’t understand something after some time just continue and come back after a few chapters.
Above code classifies cat and dog images. This is a deep learning model. Notice that this happened in less than 20 seconds. It might be different if you are running this in different device but if it’s taking more than 5 minutes there is something wrong.
The idea of machines learning was that we could give inputs and weights to a model that then uses those to give some results. The key is that there need to be some way to calculate how good the weights are (for example how often model wins in checkers) and then there need to be some way to improve them.
It’s important to have good function as a model because it might limit how good it can be. Neural network is great for this because it’s proven that it can solve any problem to any level of accuracy. That makes weights the bottleneck. With the best weights the accuracy should be perfect but it’s not simple to get the best weights. The idea is to modify them with using some metric and get the performance. Then performance is used to modify the weights using stochastic gradient descent (SGD).
Neural networks and SGD are nearly entirely relying on addition and multiplication. Not complex math!
Nowadays some terms have changed.
Architecture = model
Parameters = weights
Performance is now in three different parts. Architecture gives predictions and the performance of the model is loss. It’s calculated by using the real labels and predictions.
This learning approach only creates predictions not actions. For example it can say what someone might say about some product but it’s not going to recommend any actions.
from fastai2.vision.all import *Normally people import only parts they need but this method imports everything. fastai library is designed in a way that this only gives you the parts you need.
- NOTICE!
fastai2will befastaiby the time the course is publicly available. Right now it's using unpublished version of the library.
path = untar_data(URLs.PETS)/'images'This downloads the dataset to computer if not exists.
def is_cat(x): return x[0].isupper()This checks if the first character is uppercase because this is how cats are labeled.
dls = ImageDataLoaders.from_name_func(
path, get_image_files(path), valid_pct=0.2, seed=42,
label_func=is_cat, item_tfms=Resize(224))This creates “data loader” getting some information.
learn = cnn_learner(dls, resnet34, metrics=error_rate)Learner is the thing that learns. It gets the data loader, architecture (in this case resnet34), and the metric human uses to evaluate it.
Metric is calculated using the data that’s not used in training. This makes sure learner can’t overfit to the data.
The idea of machine learning is to get proper fit and the only way to see that is to use data that’s not used in training.
learn.fine_tune(1)This trains the model.
doc(ImageDataLoaders.from_name_func)Opens document for this function.
fastai uses a lot of practices that are not traditional in Python programming.
Before the next lesson:
- Setup all tools
- Get comfortable with the notebook, docs, and style of writing Python
Originally posted: https://www.notion.so/lankinen/Lesson-1-c1863a27813441c6b8e9302836c6addd
