MegEngine Deep Learning Getting Started Tutorial

Applicable people & prerequisite knowledge requirements

This set of tutorials is suitable for beginners who are not familiar with the basic concepts and processes of deep learning, and emphasizes deepening the understanding of theory through practice. With a series of code combat, you will gradually become familiar with this field and master the basic use of the MegEngine framework. Before reading this set of tutorials, you need to have a certain Python foundation and understand the most basic university engineering mathematics knowledge.

Content Arrangement and Organization

This set of：deep learning introductory tutorial consists of multiple chapters0

core theme	Knowledge point	programming exercise
:ref:`frame base use <megengine-basics>	Tensor / functional / autodiff / optimizer / Computational graph / Loss function…	Fitting a line based on random data
:ref:`Linear regression <linear-regression>	Machine Learning Concepts and Processes / MiniBatch Gradient Descent / Prepare DataLoader / Evaluation Metrics…	Forecast California house prices
:ref:`Linear classification <linear-classification>	Computer Vision Image Coding/ data.dataset/Feature Processing/Softmax/Cross Entropy/…	MNIST Handwritten Digit Classification
:ref:`Neural Network <neural-network>	Nonlinear activation function / fully connected neural network / random initialization / use module / deep learning parameters…	Fashion-MNIST Apparel Category
:ref:`Convolutional Neural Network <convolutional-neural-network>	RGB three-channel color image / convolution operation…	CIFAR-10 Color Image Classification
:ref:`Model development tips <model-development-basic>	Model saving and loading/Using pre-trained models/ResNet/Model development common tricks…	Read and run Models code

Each chapter will focus on explaining certain concepts to you, demonstrating the corresponding code, and at the end of the chapter will provide a comprehensive programming practice task to help you string together the concepts covered in the current chapter. Some chapters have extended material at the end. These are not mandatory, but challenging ones, so give it a try and you’ll gain a deeper understanding when you’re done (you can also choose to skip these sections).

The content between different chapters is arranged step by step. The tasks of the current chapter will not depend on the knowledge points in the subsequent chapters. However, in order to consolidate understanding, it may be deliberately compared with the content in the previous chapters, which can be triggered while recalling. More thinking, so readers are required to read sequentially and take some time to digest. Otherwise, you might be overwhelmed when we mention concepts that have come up before.

你还会看到一些折叠性质的内容

When explaining some concepts, some supplementary materials are provided in the tutorial in a collapsible style, which is not expanded by default. These materials are like sidenotes or footnotes that appear in books, which may be biographical, historical background, etc., and are intended to be extended explanations, but they are not details that must be fully grasped (after all, I occasionally want to insert some cold words. joke).

MegEngine 深度学习入门教程不会涉及到的内容

From an MLOps perspective, this set of tutorials only shows you how to use：for model development in the most basic scenarios.

• The collection and labeling of data is also full of knowledge, and the datasets used in this tutorial are all publicly available, so the above process will not be mentioned.

• After practicing all the tutorials, you will have a basic understanding of how to get a pre-trained ResNet model. So far, you can be called an “artificial intelligence model trainer (alchemy master)” who is just getting started. But for our model to work, we also need to understand how* to deploy the model to an actual production environment for efficient inference. * At this time, you need to focus more on performance and business requirements.

• In addition, this set of tutorials will only show you how to do the “training model” thing right (running), and doing things well is the goal of the next stage. You will be exposed to some advanced functional concepts of MegEngine, such as distributed training, quantization, mixed precision, etc…

MegEngine is an integrated training and push, dynamic and static, flexible and efficient framework, then you will find its true charm.

This set of tutorials uses common computer vision tasks as practice. In fact, MegEngine can also be used to solve tasks in the fields of natural language processing, audio, reinforcement learning, etc., but the data of these tasks is not so intuitive, so these domain knowledge will not be introduced in the introductory tutorial. Emphasize that the core of this set of tutorials is to help users learn MegEngine. In the process, it will make readers realize as much as possible：”Why do we need to use the deep learning framework to complete these tasks? What steps does it complete on behalf of users?” Understanding the original intention of the design can help you become a better MegEnginer and write more standardized and efficient code. In addition to explaining What is… and How to…, it is also important to understand Why…. This set of tutorials will also cultivate your good habit of querying and reading official documents, and authority is very important. Some questions may be answered in unofficial Q&A forums or blogs, but their explanations of principles and mechanisms may be wrong.

Compared with other materials

The content organization of this set of courses is largely based on the lecture materials of Stanford University’s CS229, CS230, CS231n and other courses, but it is very different from these materials. For example, CS229 is a machine learning course. This set of courses uses the generalized linear model explanation idea, but does not pay attention to the details of mathematical derivation and probabilistic explanation. This is because MegEngine, as a deep learning framework, its automatic derivation mechanism can help users focus on network structure design, so it is difficult to encounter the situation where the backpropagation process needs to be pushed by hand (at least beginners will not be discouraged by this). ). This is not to say that mathematics is not important, it is just that at the beginning stage, it is more critical to build an intuitive understanding with hands-on practice. It may be more difficult to learn and absorb knowledge directly from highly abstract and concise passages; but if we do it ourselves To code, to debug, you can grasp these concepts faster.

We will use mathematical notation that is not exactly consistent with CS229 (there is a reference table at the end of the text) to make it easier to write code that is similar in form in practice.

Other machine learning models and algorithms in CS 229 can also be implemented using MegEngine, but we will only focus on the linear model part. After Logistic regression, we began to transition to neural network models. The narrative in this part was inspired by CS230. But we will mention the concepts related to Mini-batch gradient descent and vectorization very early in the tutorial, and provide related practices; Therefore, after demonstrating the calculation logic of a single sample, the subsequent code will directly show the Mini-batch writing method. In this set of tutorials, multiple computer vision tasks are chosen as the main line, and each coding is like fighting monsters and upgrading to the next level. Classic datasets such as MNIST, Fashion-MNIST and CIFAR10 are selected as exercises, and the difficulty is increasing. We briefly introduced some computer vision knowledge in the tutorial, and there will be more detailed explanations in CS231n and some traditional computer vision textbooks.

I hope this set of MegEngine tutorials can help you develop an intuitive understanding of deep learning knowledge while learning the use of：. The most important thing is to build confidence. This will allow you to attach these concepts to a tangible foundation in time as you read more theoretical material in the future.

See also

If you are interested in machine learning theory, a good reading material is “`Probabilistic Machine Learning: An Introduction <https://probml.github.io/pml-book/book1.html>” by `Kevin Patrick Murphy <https://www.cs.ubc.ca/~murphyk/>. After completing this tutorial, the reader can A natural link to the introduction to the first part of the book. The book has more introductions to the mainstream content of modern machine learning, and provides code examples.

Please remain skeptical and tolerant

The code that appears in the tutorial does not necessarily represent best practice

When we organize MegEngine tutorials, we try to provide relevant content with the primary purpose of teaching as much as possible. For the purpose of easy understanding, some demo codes are written not close to the best practices in actual engineering production, and may even be deprecated. After going through the beginner stage, look for best practices in User’s Guide or megengine-reference.

The open source community is also a good place for us to learn the usage of MegEngine. We recommend several official：maintenance projects0

• Models: Various mainstream deep learning models implemented by MegEngine;

• Awesome MegEngine: Officially included MegEngine related projects.

Note

If you find something ambiguous in the documentation, please report it to the MegEngine team as soon as possible.

After we go to college, we will find that some theoretical knowledge we have learned in high school may be “not so correct”, and then construct new cognition. The same is true for the tutorials in MegEngine. It only represents that the content is the cognitive form that the author believes is the most easily accepted by readers at the current stage. The development of things is an upward spiral, and you may see different accounts of the same thing elsewhere, sometimes similar, sometimes contradictory. Therefore, readers need to realize that the significance of this set of tutorials is only to help beginners get started, and does not represent the eternal objective truth. You’ll even see in some tutorials that we deliberately give the “wrong” usage first and then the right way - this doesn’t mean that for learners at the previous tutorial stage, the “wrong” stuff shouldn’t be Learn. These are all good scenery on the journey. After reaching the end, in addition to looking for the next target, it is also best to look back.

As we learn to use more advanced tools to express and communicate theoretical knowledge, don’t feel that the introductory material is too superficial. At this point they are no longer for you, you should look for materials more suitable for your stage of research and study. People often make the mistake of thinking that something advanced and abstract is better because of the “curse of knowledge”, when in fact it often varies from person to person. We hope that each MegEngine only needs to learn the content of the tutorial once. In the subsequent daily use, you will no longer need to read the long explanation, just check User’s Guide and megengine-reference. .

This set of tutorials may take a lot of time to polish and verify. If you think you have better ideas, you are welcome to submit them.

Mathematical Symbol Reference Table

Symbol example	meaning	Mathematical index form	programming index form
\(x \in \mathbb{R}\)	scalar	/	/
\(\boldsymbol{x} \in \mathbb{R}^{n}\)	n-dimensional vector	\({x}_{i}\)	x[i]
\(X \in \mathbb{R}^{m\times n}\)	m-row n-column matrix	\({x}_{i,j}\)	x[i][j]
\(\mathsf{X} \in \mathbb{R}^{r \times g \times b}\)	3-dimensional tensor	\({x}_{i,j,k}\)	x[i][j][k]

Note that：starts at 1 when using the math description; starts at 0 when using the programming description.