第一章-学习目标与预备知识.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "777f6bb2",
   "metadata": {},
   "source": [
    "# 第一章：学习目标与预备知识\n",
    "\n",
    "**前言：**\n",
    "1. 本项目参考Aston Zhang等人编著的`动手深度从学习`一书，由于在安装`MXNET`会出现问题，为了追求方便(主要是懒)，同时加深对`MXNET`以及`Pytorch`的理解，在接下来的代码实现中，将会使用`Pytorch`替代`MXNET`来完成代码的编写。\n",
    "2. 为了最求简介明了，本项目将尽可能的减少无用的话术，由于本人才疏学浅，难免会有所遗漏，如有此问题，还请谅解\n",
    "\n",
    "## 1.1 环境准备\n",
    "\n",
    "在本项目中，使用`Python`版本为`3.10.5`，通过现有的`Python`安装`Jupyter Notebook`，命令如下：\n",
    "```cmd\n",
    "pip install -i https://pypi.douban.com/simple/ jupyter\n",
    "```\n",
    "\n",
    "紧接着，你需要安装一些必备的库文件，但是在后面使用时，我也会按照顺序，提示如何安装，但就目前而言，我们只需要安装`Pytorch`，这里推荐前往官网查找安装命令，选择与自己相适应的组合，然后官网会主动给出对应的安装命令，地址如下。\n",
    ">https://pytorch.org/get-started/locally/\n",
    "\n",
    "\n",
    "## 1.2 数据操作\n",
    "\n",
    "在深度学习中，打交道最多的就是数据，在个人看来，深度学习无非就是将预先清洗好的数据，放进搭建好的框架结构中进行进一步分析处理，抽丝剥茧，最终最终期待该模型能够窥探该数据的本质，即规则，并能够在同类中进行良好的泛化。因此，无论如何，其中的重点，依旧是数据。\n",
    "\n",
    "### 1.2.1 创建数据容器\n",
    "\n",
    "在原文的书中，描述的是创建`NDArray`，怎么去准确描述而不产生歧义，这便成了一个问题，在主流的不同类型体系下，叫法也不一致，甚至于调用的方法也各不相同，但无论如何改变，都改变不了一个事实，那就是它们都是存储数据的容器。\n",
    "\n",
    "我们最常用的一个，就是`numpy`，在数据处理阶段，我们比较常见，而要想使用`numpy`创建一个数据容器，首先的安装，安装命令如下：\n",
    "```cmd\n",
    "pip install -i https://ptpi.douban.com/simple/ numpy\n",
    "```\n",
    "\n",
    "接下来，我们导入`numpy`库，并使用`arange`函数创建一个行向量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7b86d4d6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0  1  2  3  4  5  6  7  8  9 10 11]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 导入numpy库，并为其起一个别名np\n",
    "import numpy as np\n",
    "\n",
    "# 使用arange函数创建行向量\n",
    "x = np.arange(12)\n",
    "\n",
    "print(x)\n",
    "\n",
    "# 在 jupyter notebook 中，你也可以直接不使用 print 输出\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5dd4ad4d",
   "metadata": {},
   "source": [
    "从上面可以看出，通过`arange`生成了一个`[0,12)`左闭右开的一个行向量，在直接使用`jupyter notebook`本身输出时，可以看到该向量的数据类型是`array`类型。\n",
    "\n",
    "我们可以通过`shape`属性获取该向量的形状。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "542f3e3e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(12,)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c84b3ffe",
   "metadata": {},
   "source": [
    "我们还能够通过`size`属性来获取该实例中的元素总数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "28d058ac",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.size"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "915972d2",
   "metadata": {},
   "source": [
    "我们还可以使用`reshape`函数改变行向量`x`的形状，当然，改变形状必须是合法的，即两者的元素总数不变。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "a70d8586",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0,  1,  2,  3],\n",
       "       [ 4,  5,  6,  7],\n",
       "       [ 8,  9, 10, 11]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x1 = x.reshape((3, 4))\n",
    "x1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2c630e97",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3, 4)\n",
      "12\n"
     ]
    }
   ],
   "source": [
    "# 查看改变后的形状以及元素总数\n",
    "print(x1.shape)\n",
    "print(x1.size)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59072545",
   "metadata": {},
   "source": [
    "这里需要说明，上面说了可以直接使用`jupyter notebook`自己的输出，但是该方案有一个缺陷，那就是只会输出最后一个内容，前面的内容将会被覆盖，因此倘若需要输出多个内容，那么还是需要使用`print`函数。\n",
    "\n",
    "在`reshape`过程中，我们常见这样的形势`x.reshape(-1,1)`或者`x.reshape(1,-1)`，就是将数据`x`转换成行向量(列向量)，由于`x`的元素个数是已知的，上面的-1是能够通过元素个数和其他维度大小推断出来的。\n",
    "\n",
    "在深度学习中，我们会使用到一些特殊的数据初始化，比如创建一个形状为(2,3,4),所有元素都为0的张量。实际上，之前创建的向量和矩阵都是特殊的张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "5916c7de",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0.]],\n",
       "\n",
       "       [[0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0.]]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.zeros((2, 3, 4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d03381bf",
   "metadata": {},
   "source": [
    "类似地，我们可以创建各元素为1的张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "9cb847bb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 1., 1., 1.],\n",
       "       [1., 1., 1., 1.],\n",
       "       [1., 1., 1., 1.]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.ones((3,4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb6cb61f",
   "metadata": {},
   "source": [
    "倘若已经有了数据，且是通过列表(`list`)存储的，那么我们就可以直接通过`Python`的列表`list`创建与该列表一致的张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f137ff2c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2, 1, 2, 3],\n",
       "       [3, 5, 6, 1],\n",
       "       [7, 5, 1, 3]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = np.array([[2, 1, 2, 3], [3, 5, 6, 1], [7, 5, 1, 3]])\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de263b51",
   "metadata": {},
   "source": [
    "上面描述的如此多的内容，都是使用`numpy`完成的，但其实在`Pytorch`中，我们的数据类型都要转为对应类型，而在`Pytorch`中，一般的数据被称为张量。\n",
    "\n",
    "在前面，我们已经安装好了`pytorch`，现在就导入该包，并完成上面的内容，其实本质都是一样的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "3b0fbd2c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "torch_x = torch.arange(12)\n",
    "\n",
    "torch_x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84c17c04",
   "metadata": {},
   "source": [
    "从上面可以看到，在`pytorch`中，容器被称为`tensor`，即张量。\n",
    "\n",
    "接下来，我们也使用`torch`完成其他的工作。\n",
    "\n",
    "首先，查看`torch_x`的形状。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "1e15831a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([12])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch_x.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17d8d9b5",
   "metadata": {},
   "source": [
    "从上面可以看出，除了输出的形式不一样，其他的基本一致，接下来，我们调用`size`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "c8e99d94",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([12])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch_x.size()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f9893c3",
   "metadata": {},
   "source": [
    "与`numpy`有点不一样的是，`Pytorch`张量的`size`不是一个属性，而是一个方法，因此要在使用时是`torch_x.size()`，而非`torch_x.size`。\n",
    "\n",
    "下面我们使用`reshape`对`torch`的张量进行形状的改变，使用方法如下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ec79266c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0,  1,  2,  3],\n",
       "        [ 4,  5,  6,  7],\n",
       "        [ 8,  9, 10, 11]])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch_x1 = torch_x.reshape((3, 4))\n",
    "torch_x1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7978356d",
   "metadata": {},
   "source": [
    "查看`torch_x1`的形状和大小。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "ffe2451e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([3, 4])\n",
      "torch.Size([3, 4])\n"
     ]
    }
   ],
   "source": [
    "print(torch_x1.shape)\n",
    "print(torch_x1.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a568da68",
   "metadata": {},
   "source": [
    "进一步的尝试我们可以发现，`pytorch`张量的`size()`方法和`numpy`中的`size`其实是不一样的，在`pytorch`中的`size()`更像是`shape`属性的另一个调用形式。\n",
    "\n",
    "那`pytorch`的变量可以和之前一样调用`reshape(1,-1)`或者`reshape(-1,1)`吗？下面我们尝试一下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "523d7e88",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0],\n",
       "        [ 1],\n",
       "        [ 2],\n",
       "        [ 3],\n",
       "        [ 4],\n",
       "        [ 5],\n",
       "        [ 6],\n",
       "        [ 7],\n",
       "        [ 8],\n",
       "        [ 9],\n",
       "        [10],\n",
       "        [11]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch_x.reshape(-1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "02561b8d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch_x.reshape(1, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "844bfe8a",
   "metadata": {},
   "source": [
    "从上面可以看出，`pytorch`的`reshape`操作和`numpy`是一致的。\n",
    "\n",
    "接下来，我们创建一个形状为`(2,3,4)`，各元素全为0的张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "cce720ae",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0.]],\n",
       "\n",
       "        [[0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0.]]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.zeros((2, 3, 4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fe5fd0d",
   "metadata": {},
   "source": [
    "类似地，我们可以创建各元素为1的张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ee4f33fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1.]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.ones((3, 4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed8ecd49",
   "metadata": {},
   "source": [
    "我们也可以通过`Python`的列表(`list`)指定需要创建的`tensor`中每个元素的值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "24a981fc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[2, 1, 4, 3],\n",
       "        [1, 2, 3, 4],\n",
       "        [4, 3, 2, 1]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.tensor([[2, 1, 4, 3], [1, 2, 3, 4], [4, 3, 2, 1]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46b2d09a",
   "metadata": {},
   "source": [
    "有些情况下，我们需要随机⽣成数据容器中每个元素的值。下⾯我们创建⼀个形状为(3,4)的`tensor`。它的每个元素都随机采样于均值为0、标准差为1的正态分布。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "022ce887",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 3.3456, -1.4106, -0.2198, -1.1699],\n",
       "        [ 2.3045,  1.0154, -0.3354,  0.6758],\n",
       "        [ 0.9367,  0.3095, -1.8830,  0.9611]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.normal(mean=0, std=1, size=(3, 4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3269df84",
   "metadata": {},
   "source": [
    "### 1.2.2 运算\n",
    "\n",
    "科学计算的数据容器⽀持⼤量的运算符(`operator`)。例如，我们可以对之前创建的两个形状为(3,4)的`tensor`做按元素加法。所得结果形状不变。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "72f09cfc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.1116, 0.5664, 0.7239],\n",
      "        [0.9995, 0.9278, 0.2037]])\n",
      "tensor([[0.4575, 0.5238, 0.7656],\n",
      "        [0.6652, 0.1884, 0.8272]])\n",
      "tensor([[0.5691, 1.0902, 1.4895],\n",
      "        [1.6647, 1.1162, 1.0310]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.rand(2, 3)\n",
    "print(x)\n",
    "y = torch.rand(2, 3)\n",
    "print(y)\n",
    "print(x+y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32273557",
   "metadata": {},
   "source": [
    "按元素乘法(在`torch`也可以使用`torch.mul`计算元素乘法)："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b269635d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.3384, 0.0097, 0.7252],\n",
       "        [0.0183, 0.3628, 0.1490]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x * y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdcfade3",
   "metadata": {},
   "source": [
    "按元素除法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "619d2d6a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.8587, 0.4555, 0.9782],\n",
       "        [0.0754, 1.2396, 0.2269]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x / y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd31bc6b",
   "metadata": {},
   "source": [
    "按元素做指数运算："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "9003ceed",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1.8734, 1.1568, 2.3656],\n",
       "        [1.6370, 1.7177, 2.2489]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.exp()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1bd33031",
   "metadata": {},
   "source": [
    "除了按元素计算外，我们还可以使⽤`torch.mm`函数做矩阵乘法。下⾯将`x`与`y`的转置做矩阵乘法。由于`x`是2⾏3列的矩阵，`y`转置为3⾏2列的矩阵，因此两个矩阵相乘得到2⾏2列的矩阵。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "8f2f3222",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1.0732, 0.9841],\n",
       "        [0.2793, 0.5301]])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.mm(x, y.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ea2b6bc",
   "metadata": {},
   "source": [
    "我们也可以将多个`tensor`连结(`concatenate`)。下⾯分别在⾏上(维度0，即形状中的最左边元素)和列上(维度1，即形状中左起第⼆个元素)连结两个矩阵。可以看到，输出的第⼀个`tensor`在\n",
    "维度`0`的⻓度(`4`)为两个输⼊矩阵在维度`0`的⻓度之和(`3+3`)，而输出的第⼆个`tensor`在维\n",
    "度`1`的⻓度(`6`)为两个输⼊矩阵在维度`1`的⻓度之和(`4+4`)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "7a6ecd0e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.5390, 0.0663, 0.8422],\n",
       "        [0.0372, 0.6706, 0.1839],\n",
       "        [0.6278, 0.1456, 0.8610],\n",
       "        [0.4929, 0.5410, 0.8105]])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.cat([x, y], dim=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "fd2759c7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.5390, 0.0663, 0.8422, 0.6278, 0.1456, 0.8610],\n",
       "        [0.0372, 0.6706, 0.1839, 0.4929, 0.5410, 0.8105]])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.cat([x, y], dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd6e77c6",
   "metadata": {},
   "source": [
    "使⽤条件判断式可以得到元素为`True`或`False`的新的`tensor`。以`x == y`为例，如果`x`和`y`在相同位置的条件判断为真(值相等)，那么新的`tensor`在相同位置的值为`True`；反之为`False`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "44de8c99",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[False,  True, False],\n",
       "        [False, False, False]])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置 x, y 对应位置元素\n",
    "x[0, 1] = 1\n",
    "y[0, 1] = 1\n",
    "x == y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8f2caa5",
   "metadata": {},
   "source": [
    "对`tensor`中的所有元素求和得到只有⼀个元素的`tensor`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "3e5d44cb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(3.2729)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "629e8e8f",
   "metadata": {},
   "source": [
    "我们可以通过`norm`函数计算例⼦中`x`的$L_1,L_2$范数结果,同上例⼀样是单元素`tensor`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "b79610d3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(3.2729)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# x 的 L1 范数\n",
    "x.norm(p=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "2320ec6a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(1.5763)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# x 的 L2 范数\n",
    "x.norm(p=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "914bd466",
   "metadata": {},
   "source": [
    "除上面以外，我们还可以通过`dim`关键字选择求取对应范数的维度，比如`dim=0`时按列求取对应范数，当`dim=1`时按行求解对应范数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "822c7727",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0.5762, 1.6706, 1.0261])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 按列求解范数\n",
    "x.norm(p=1, dim=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "9ff4fa77",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([2.3813, 0.8917])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 按行求解范数\n",
    "x.norm(p=1, dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbc0b926",
   "metadata": {},
   "source": [
    "**`tensor`与`Numpy`中的`array`之间的转换**\n",
    "\n",
    "其中`tensor`变量可以通过`numpy`函数转为普通`numpy`中的`array`变量或者数组。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "48544a7e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.11157459, 0.5663688 , 0.7239244 ],\n",
       "       [0.9994676 , 0.92776114, 0.20373732]], dtype=float32)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "835a997b",
   "metadata": {},
   "source": [
    "当然，在使用`pytorch`进行机器学习编程时，其内部的数据容器都要保持一致，因此我们需要将`numpy`的`array`类型转为`tensor`类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "4d4be752",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.1116, 0.5664, 0.7239],\n",
       "        [0.9995, 0.9278, 0.2037]])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = x.numpy()\n",
    "torch.from_numpy(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a8841b8",
   "metadata": {},
   "source": [
    "我们也可以把`y.exp()、x.sum()、x.norm()`等分别改写为`torch.exp(y)、torch.sum(x)、torch.norm(x)`等。\n",
    "\n",
    "### 1.2.3 广播机制\n",
    "\n",
    "前⾯我们看到如何对两个形状相同的`tensor`做按元素运算。当对两个形状不同的`tensor`按元素运算时，可能会触发⼴播(`broadcasting`)机制：先适当复制元素使这两个`tensor`形状相同后再按元素运算。\n",
    "\n",
    "定义两个`tensor`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "907b26b0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[0],\n",
       "         [1],\n",
       "         [2]]),\n",
       " tensor([[0, 1]]))"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = torch.arange(3).reshape((3, 1))\n",
    "B = torch.arange(2).reshape((1, 2))\n",
    "A, B"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e95d3a6c",
   "metadata": {},
   "source": [
    "由于`A`和`B`分别是3⾏1列和1⾏2列的矩阵，如果要计算`A+B`，那么`A`中第⼀列的3个元素被⼴播(复制)到了第⼆列，而`B`中第⼀⾏的2个元素被⼴播(复制)到了第⼆⾏和第三⾏。如此，就可以对2个3⾏2列的矩阵按元素相加。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "2181b67e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0, 1],\n",
       "        [1, 2],\n",
       "        [2, 3]])"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A + B"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76154d6a",
   "metadata": {},
   "source": [
    "### 1.2.4 索引\n",
    "\n",
    "在`tensor`中，索引(`index`)代表了元素的位置。`tensor`的索引从0开始逐⼀递增。例如，⼀个3⾏2列的矩阵的⾏索引分别为0、1和2，列索引分别为0和1。\n",
    "\n",
    "在下⾯的例⼦中，我们指定了`tensor`的⾏索引截取范围[1:3]。依据左闭右开指定范围的惯例，它截取了矩阵`X`中⾏索引为1和2的两⾏。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "efc3cd4b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 4,  5,  6,  7],\n",
       "        [ 8,  9, 10, 11]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = torch.arange(12).reshape(3, 4)\n",
    "X[1:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "578d33bf",
   "metadata": {},
   "source": [
    "我们可以指定`tensor`中需要访问的单个元素的位置，如矩阵中⾏和列的索引，并为该元素重新赋值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "34b87ef9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0,  1,  2,  3],\n",
       "        [ 4,  5,  9,  7],\n",
       "        [ 8,  9, 10, 11]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X[1, 2] = 9\n",
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a787d49",
   "metadata": {},
   "source": [
    "当然，我们也可以截取⼀部分元素，并为它们重新赋值。在下⾯的例⼦中，我们为⾏索引为1的每⼀列元素重新赋值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6fb1b68f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0,  1,  2,  3],\n",
       "        [12, 12, 12, 12],\n",
       "        [ 8,  9, 10, 11]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X[1:2, :] = 12\n",
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0781e3b",
   "metadata": {},
   "source": [
    "### 1.2.5 运算的内存开销\n",
    "\n",
    "在前⾯的例⼦⾥我们对每个操作新开内存来存储运算结果。举个例⼦，即使像`Y = X + Y`这样的运算，我们也会新开内存，然后将`Y`指向新内存。为了演⽰这⼀点，我们可以使⽤`Python`⾃带的`id`函数：如果两个实例的`ID`⼀致，那么它们所对应的内存地址相同；反之则不同。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "6a44d013",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y = torch.arange(12).reshape(3, 4)\n",
    "\n",
    "before = id(Y)\n",
    "Y = Y + X\n",
    "id(Y) == before"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c269bcd",
   "metadata": {},
   "source": [
    "如果想指定结果到特定内存，我们可以使⽤前⾯介绍的索引来进⾏替换操作。在下⾯的例⼦中，我们先通过`zeros_like`创建和`Y`形状相同且元素为0的`tensor`，记为`Z`。接下来，我们把`X +\n",
    "Y`的结果通过`[:]`写进`Z`对应的内存中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "61cc2a3a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Z = Y.zero_()\n",
    "before = id(Z)\n",
    "Z[:] = X + Y\n",
    "id(Z) == before"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15f0810c",
   "metadata": {},
   "source": [
    "实际上，上例中我们还是为`X + Y`开了临时内存来存储计算结果，再复制到`Z`对应的内存。如果\n",
    "想避免这个临时内存开销，我们可以使⽤运算符全名函数中的`out`参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b6e48cdf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.add(X, Y, out=Z)\n",
    "id(Z) == before"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59e678df",
   "metadata": {},
   "source": [
    "如果`X`的值在之后的程序中不会复⽤，我们也可以⽤`X[:] = X + Y`或者`X += Y`来减少运算\n",
    "的内存开销。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "5ec6f241",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "before = id(X)\n",
    "X += Y\n",
    "id(X) == before"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aece2dc7",
   "metadata": {},
   "source": [
    "还可以直接在元素本身进行运算，使用`add_`方法，其效果和上面的是一样的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "cf5bc13e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.add_(Y)\n",
    "id(X) == before"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ef620e8",
   "metadata": {},
   "source": [
    "这里有一个点需要注意，`add_`是`in-place`操作，即原地运算，需要和`add`进行区分，`add`可以视为运算符，如`a.add(b)`等同于`a+b`，运算后需要指定结果存储位置，因此完整的运算语句应该为`a = a.add(b)`，`a = a + b`，而这两个运算分别等同于`a.add_(b)`和`a += b`。\n",
    "\n",
    "## 1.3 自动求解梯度\n",
    "\n",
    "在深度学习中，无论什么样的网络构型，都不可避免地需要使用到对函数求梯度(`gradient`)。本节将介绍如何使用`pytorch`提供的`autograd`模块来自动求梯度。如果对本节中的数学概念(如梯度)不是很熟悉，可以参阅附录中“数学基础”一节。\n",
    "\n",
    "下面，首先导入`torch`以及`torch`下面的`autograd`包。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "6ea86e7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch import autograd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "310ae3fe",
   "metadata": {},
   "source": [
    "### 1.3.1 简单实例\n",
    "\n",
    "我们首先来看一个简单的例子：对函数$\\pmb{y}=2\\pmb{x}^T\\pmb{x}$求关于列向量$\\pmb{x}$的梯度。我们先创建变量$\\pmb{x}$，并初始化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "82a5450d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0],\n",
       "        [1],\n",
       "        [2],\n",
       "        [3]])"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.arange(4).reshape((4, 1))\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5fe2067",
   "metadata": {},
   "source": [
    "为了求有关变量$\\pmb{x}$的梯度，我们需要先设置`requires_grad=True`来告诉系统我们需要对变量$\\pmb{x}$进行求导。\n",
    "\n",
    "在设置`requires_grad`，需要将矩阵$\\pmb{x}$的数据类型设置为`float`类型，只有连续的数值，才可求导。当然，一般时候初始化就是`float`类型，上面的使用的是`arange`进行初始化，因此我们需要手动的将其转为`float`类型。\n",
    "\n",
    ">此外，在设置是否需要求导时，方法多样，你可以在初始化时就进行设置，当然，这个时候你首先得确认这个张量中的数据是浮点型数据，你可以采用下面这种方式：\n",
    "```python\n",
    "x = torch.ones((2, 4), requires_grad=True)\n",
    "```\n",
    ">当然，你也可以先初始化数据，此时默认该数据是不需要求导的，然后采取下面格式设置为需要求导：\n",
    "```python\n",
    "x.requires_grad = True\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "e0d60f8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "x = x.float()\n",
    "x.requires_grad = True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "566e9272",
   "metadata": {},
   "source": [
    "下面定义函数运算$\\pmb{y}=2\\pmb{x}^T\\pmb{x}$，同时查看$\\pmb{y}$是否可导。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "55e91920",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = 2 * torch.mm(x.T, x)\n",
    "y.requires_grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fe27f8d",
   "metadata": {},
   "source": [
    "从上面可以发现$\\pmb{y}$不需要进行设置，是可导的，也就说明，当一个函数，变量矩阵可导，则结果也可导，有点像广播机制。\n",
    "\n",
    "接下来对$\\pmb{y}$执行反向传播，同时查看$\\pmb{x}$的梯度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "de6f4046",
   "metadata": {},
   "outputs": [],
   "source": [
    "y.backward()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "1b071a4d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.],\n",
       "        [ 4.],\n",
       "        [ 8.],\n",
       "        [12.]])"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9a0cab1",
   "metadata": {},
   "source": [
    "**要点注意 1**\n",
    "\n",
    "自动求解梯度的细节并未结束，在上面的描述中，设置张量可导的方法描述了两种，其实还有一种，当你使用提示时会发现，还有一个方法`requires_grad_`，该方法可以写成`x1.requires_grad_=True`，不会报错，但其实并没有任何效果，并不能使张量标记为可导。在`pytorch`中，该方法的正确使用为`x1.requires_grad_(True)`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "id": "dd19ea1c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1.]])"
      ]
     },
     "execution_count": 161,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x1 = torch.ones((2, 4))\n",
    "x1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03925de5",
   "metadata": {},
   "source": [
    "下面使用`x1.requires_grad_=True`方式设置该张量可导，可以发现，并未设置成功。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "id": "ae2c8196",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1.]])"
      ]
     },
     "execution_count": 155,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x1.requires_grad_ = True\n",
    "x1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b063b223",
   "metadata": {},
   "source": [
    "接下来使用`x1.requires_grad_(True)`方式，便可以成功。\n",
    "\n",
    ">这里需要重点注意，如果你先执行了上面的`x1.requires_grad_ = True`，在执行下面的代码时会报错，因为在上面相当于将`x1`中的`requires_grad_`方法设置成一个属性，且该属性为`True`。因此再调用`x1.requires_grad_(True)`时便会报错。所以你需要再次执行上面张量`x1`初始化语句，然后执行下面的代码。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "id": "b956415f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1.]], requires_grad=True)"
      ]
     },
     "execution_count": 162,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x1.requires_grad_(True)\n",
    "x1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b8be50b",
   "metadata": {},
   "source": [
    "**要点注意 2**\n",
    "\n",
    "下面执行函数的定义以及函数反向传播。这里又需要注意，在上面执行反向传播时，使用的语句是`y.backward()`，但是在这次的例子中，使用上面的方式执行`y1.backward()`的话，那么你只能收获报错。错误的原因在于，这次函数的结果并不是一个标量，而是一个张量，因此需要设置`output`类型，因此在反向传播时应该执行`y1.backward(torch.ones_like(y1))`。其实，稳妥起见，你可以将所有的反向传播都写成`y1.backward(torch.ones_like(y1))`，这样的话，不论函数结果是标量还是张量，都是正确的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "id": "aab0d276",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[16., 16., 16., 16.],\n",
       "        [16., 16., 16., 16.]])"
      ]
     },
     "execution_count": 163,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y1 = 2 * torch.mm(x1.T, x1)\n",
    "y1.backward(torch.ones_like(y1))\n",
    "x1.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24e07e80",
   "metadata": {},
   "source": [
    "### 1.3.2 训练模式和预测模式\n",
    "\n",
    "从上面我们可以看出，当一个数据容器(在这里是张量，即`tensor`)被告知需要进行求导，那么`pytorch`会记录并计算梯度。\n",
    "\n",
    "此外，当张量的`requires_grad`被设置为`True`时，那么该张量，或者说使用了该张量的数据模型就变成**训练模式**，在该模式下，张量可进行求导并将结果保存。当`requires_grad`未被设置为`True`时，我们则称之为**预测模式**，此时不能进行求导。\n",
    "\n",
    "**训练模式**与**预测模式**的理解，其实代入到机器学习模型中就能更好的理解。例如，在一个神经网络中，训练传播过程中需要对函数求导，因此此时就需要张量具备求导功能，这个模式就被称为**训练模式**；而在训练好模型之后，我们只需要通过该模型进行预测，此时并不存在对其中张量求导的需求，为了节省空间、提高运行速度，一般会关闭张量的求导功能，即设置`requires_grad=False`，而这个模式，就被称为**预测模式**。\n",
    "\n",
    "简而言之，**训练模式**即张量可被求导的模式，**预测模式**即张量不可被求导的模式。\n",
    "\n",
    "### 1.3.3 对`Python`控制流求梯度\n",
    "\n",
    "对于大多数的机器学习框架而言，都能对`Python`的控制流进行梯度求解。\n",
    "\n",
    "考虑下⾯程序，其中包含`Python`的条件和循环控制。需要强调的是，这⾥循环(`while`循环)迭\n",
    "代的次数和条件判断(`if`语句)的执⾏都取决于输⼊`a`的值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "id": "81573953",
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(a):\n",
    "    b = a * 2\n",
    "    while b.norm(p=2) < 500:\n",
    "        b = b * 2\n",
    "    if b.sum() > 0:\n",
    "        c = b\n",
    "    else:\n",
    "        c = 100 * b\n",
    "    return c"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f0c70e6",
   "metadata": {},
   "source": [
    "上文描述了梯度广播机制(自己取的名称，不确定是否有该机制，主要是便于理解)，因此只要输入是可求导的参数，则运算后的结果也是可求导的。\n",
    "\n",
    "下面我们初始化一个张量`a`，并设置为可导，调用函数`f()`计算，并对结果进行求导。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "id": "839cadf3",
   "metadata": {},
   "outputs": [],
   "source": [
    "a = torch.normal(mean=0, std=1, size=(2,3), requires_grad=True)\n",
    "c = f(a)\n",
    "c.backward(torch.ones_like(c))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cde9b7af",
   "metadata": {},
   "source": [
    "下面输出求张量`a`的求导结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "id": "6b6d441b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[25600., 25600., 25600.],\n",
       "        [25600., 25600., 25600.]])"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29a7dea6",
   "metadata": {},
   "source": [
    "我们来分析⼀下上⾯定义的f函数。事实上，给定任意输⼊`a`，其输出必然是 `f(a) = x * a`的\n",
    "形式，其中标量系数`x`的值取决于输⼊`a`。由于`c = f(a)`有关`a`的梯度为`x`，且值为`c / a`，我们可以像下⾯这样验证对本例中控制流求梯度的结果的正确性。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "id": "7b28a502",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[True, True, True],\n",
       "        [True, True, True]])"
      ]
     },
     "execution_count": 193,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a.grad == c / a"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "564476a9",
   "metadata": {},
   "source": [
    "## 1.4 拓展学习\n",
    "\n",
    "文章篇幅有限，无法去详细描述`pytorch`框架所具有的所有特性，但总之，学习是一个循序渐进的过程，同时也是一个永无止境的道路。\n",
    "\n",
    "在学习时，遇到问题多百度，基本上你遇到的问题，别人也遇到过，并且在网上也会有人给出答案，这种概率说为99%也不为过，倘若你遇到一个网上至今没有人遇到的问题，那么也不用灰心，说明你已经到达一个前所未有的高度。\n",
    "\n",
    "查阅官方文档是一个不错的习惯，当你在使用某个`API`遇到问题时。首先，科学技术可以打破壁垒，面对英文文档，即便英语不佳，也不用太过担心，现在的很多翻译软件翻译的结果可能远胜于一般的人工翻译，因此，即便英语不好也不用太过于担心，这也不是学不好的借口。\n",
    "\n",
    "就个人而言，兴趣是最好的老师，当你带着兴趣学习，那么就会充满动力，同时在学习的过程中应该带着疑问，求知欲，多尝试，而不要担心出错，错误是前进路上的老师。\n",
    "\n",
    "当然，初期也可以尽量的避免错误发生，严格按照实例进行学习理解，如果进行新的尝试时出现错误，也可以适当的忽略，浅尝辄止。这样建议的原因是学习初期过多的错误可能会打击学习的信心以及初期的兴趣，因此为了慢慢培养信心和兴趣，前期可以照葫芦画瓢，但你必须要摆脱一直照葫芦画瓢，随着不断地深入了解，要学会举一反三，比如示例是一个乘法，你用加法去尝试，示例使用地是`pytorch`所写，那你可以尝试使用`tensorflow`写一遍，这样不仅加深理解，同时让你对两个框架都有更深一步地了解。\n",
    "\n",
    "下一章，我们将进一步学习深度学习基础，希望大家继续努力。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}