10_intro_julia.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "NHG3EnKePvKi2VhD66Sm",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "# Introduction to Julia"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "pdQjoHJpBntjqta7HZIH",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     }
    }
   },
   "source": [
    "[Julia](https://julialang.org/) is a fast, dynamic, reproducible, composable, general and open source programming language that can be downloaded [here](https://julialang.org/downloads/). The full documentation of the current release (v1.7) is available [here](https://docs.julialang.org/en/v1/). This tutorial is designed to go over the basics of Julia, and to cover several topics on which Julia's speed is based. \n",
    "\n",
    "1. use/add/remove/develop package\n",
    "2. basic linear algebra\n",
    "3. just-in-time (JIT) compiler\n",
    "4. multiple dispatch\n",
    "5. type-stability"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "OPQOeWejCSDHz8cFoWME",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "## Package management\n",
    "\n",
    "In Julia, packages are loaded via `using` (like packages are loaded in python via `import`). We typically load all the packages we want to use at the start of the file as"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "BPd8NqXJYohPyMM1j7Qo",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": null
    }
   },
   "outputs": [],
   "source": [
    "using LinearAlgebra, Test, BenchmarkTools"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "BeMEFrfowtGnyEJEhRDb",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "To add/remove a package, we can follow [Pkg.jl](https://github.com/JuliaLang/Pkg.jl) and run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "xOMH2gmCHQYOVonpTq88",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     },
     "outputId": {
      "block": "mjua8ZCoCKuMFwTRz62A",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     }
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m\u001b[1m    Updating\u001b[22m\u001b[39m registry at `~/.julia/registries/General`\n",
      "\u001b[32m\u001b[1m    Updating\u001b[22m\u001b[39m git-repo `https://github.com/JuliaRegistries/General.git`\n",
      "\u001b[32m\u001b[1m    Updating\u001b[22m\u001b[39m registry at `~/.julia/registries/SLIMregistryJL`\n",
      "\u001b[32m\u001b[1m    Updating\u001b[22m\u001b[39m git-repo `https://github.com/slimgroup/SLIMregistryJL.git`\n",
      "\u001b[32m\u001b[1m   Resolving\u001b[22m\u001b[39m package versions...\n",
      "\u001b[32m\u001b[1m    Updating\u001b[22m\u001b[39m `~/.julia/environments/v1.7/Project.toml`\n",
      " \u001b[90m [42fd0dbc] \u001b[39m\u001b[93m~ IterativeSolvers v0.9.2 `https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git#master` ⇒ v0.9.2\u001b[39m\n",
      "\u001b[32m\u001b[1m    Updating\u001b[22m\u001b[39m `~/.julia/environments/v1.7/Manifest.toml`\n",
      " \u001b[90m [42fd0dbc] \u001b[39m\u001b[93m~ IterativeSolvers v0.9.2 `https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git#master` ⇒ v0.9.2\u001b[39m\n",
      "\u001b[32m\u001b[1mPrecompiling\u001b[22m\u001b[39m project...\n",
      "\u001b[32m  ✓ \u001b[39mIterativeSolvers\n",
      "\u001b[32m  ✓ \u001b[39mJOLI\n",
      "\u001b[32m  ✓ \u001b[39mGenSPGL\n",
      "\u001b[32m  ✓ \u001b[39mJUDI\n",
      "\u001b[32m  ✓ \u001b[39mSetIntersectionProjection\n",
      "\u001b[32m  ✓ \u001b[39mMECurvelets\n",
      "\u001b[32m  ✓ \u001b[39mImageGather\n",
      "\u001b[32m  ✓ \u001b[39mInvertibleNetworks\n",
      "  8 dependencies successfully precompiled in 16 seconds (356 already precompiled)\n"
     ]
    }
   ],
   "source": [
    "using Pkg\n",
    "Pkg.add(\"IterativeSolvers\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "ZkhMo7uh2Mil0TQNZQAD",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "Np3UwSHxaYXj2CwAoGeM",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m\u001b[1m    Updating\u001b[22m\u001b[39m `~/.julia/environments/v1.7/Project.toml`\n",
      " \u001b[90m [42fd0dbc] \u001b[39m\u001b[91m- IterativeSolvers v0.9.2\u001b[39m\n",
      "\u001b[32m\u001b[1m  No Changes\u001b[22m\u001b[39m to `~/.julia/environments/v1.7/Manifest.toml`\n"
     ]
    }
   ],
   "source": [
    "Pkg.rm(\"IterativeSolvers\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "FumQwwaxbQu9gH0uqqDz",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "Note that we can also install a Julia package at a specific version, or from a GitHub repository"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "wRLFPOGHdV0ZZibF2UPD",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "C0vf1RvYt1v0m7xfZkGt",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m\u001b[1m   Resolving\u001b[22m\u001b[39m package versions...\n",
      "\u001b[32m\u001b[1m    Updating\u001b[22m\u001b[39m `~/.julia/environments/v1.7/Project.toml`\n",
      " \u001b[90m [42fd0dbc] \u001b[39m\u001b[92m+ IterativeSolvers v0.9.2\u001b[39m\n",
      "\u001b[32m\u001b[1m  No Changes\u001b[22m\u001b[39m to `~/.julia/environments/v1.7/Manifest.toml`\n"
     ]
    }
   ],
   "source": [
    "Pkg.add(name=\"IterativeSolvers\", version=\"0.9.2\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "ZRxqSJsiKSjSl3Jt8cUu",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     },
     "outputId": {
      "block": "eYrReidMhIBwbRtkho6M",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     }
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m\u001b[1m    Updating\u001b[22m\u001b[39m git-repo `https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git`\n",
      "\u001b[32m\u001b[1m   Resolving\u001b[22m\u001b[39m package versions...\n",
      "\u001b[32m\u001b[1m    Updating\u001b[22m\u001b[39m `~/.julia/environments/v1.7/Project.toml`\n",
      " \u001b[90m [42fd0dbc] \u001b[39m\u001b[93m~ IterativeSolvers v0.9.2 ⇒ v0.9.2 `https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git#master`\u001b[39m\n",
      "\u001b[32m\u001b[1m    Updating\u001b[22m\u001b[39m `~/.julia/environments/v1.7/Manifest.toml`\n",
      " \u001b[90m [42fd0dbc] \u001b[39m\u001b[93m~ IterativeSolvers v0.9.2 ⇒ v0.9.2 `https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git#master`\u001b[39m\n",
      "\u001b[32m\u001b[1mPrecompiling\u001b[22m\u001b[39m project...\n",
      "\u001b[32m  ✓ \u001b[39mIterativeSolvers\n",
      "\u001b[32m  ✓ \u001b[39mJOLI\n",
      "\u001b[32m  ✓ \u001b[39mGenSPGL\n",
      "\u001b[32m  ✓ \u001b[39mJUDI\n",
      "\u001b[32m  ✓ \u001b[39mSetIntersectionProjection\n",
      "\u001b[32m  ✓ \u001b[39mImageGather\n",
      "\u001b[32m  ✓ \u001b[39mMECurvelets\n",
      "\u001b[32m  ✓ \u001b[39mInvertibleNetworks\n",
      "  8 dependencies successfully precompiled in 16 seconds (356 already precompiled)\n"
     ]
    }
   ],
   "source": [
    "Pkg.add(url=\"https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git\", rev=\"master\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "IYiGn2ZIHsjkl0d5bTWY",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "We can also do these on julia's interactive command-line REPL (read-eval-print loop) via\n",
    "```julia\n",
    "] add IterativeSolvers\n",
    "] rm IterativeSolvers\n",
    "] add https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git\n",
    "```\n",
    "etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "W36clnUXPRFCs6u3E5s0",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "## Linear algebra"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "JsvM9mAMzYD0vXqL1z5h",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "The syntax of creating multidimensional arrays is a bit similar to MATLAB. We use square brackets `[]` to set up a multidimensional array, and use semi-colon `;` to switch rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "4IBb50822NNF4nbpiaCG",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "oPNum2UBWcG33wpTw4Jk",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Matrix{Int64}:\n",
       " 1  2\n",
       " 3  4"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = [1 2; 3 4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "KzQ6y0OgHQGVLHRn36H8",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "Note that multidimensional arrays are stored in a column-major order, meaning that they fill up one column at a time. We can verify this by vectorizing `A` via `vec` or `[:]`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "bl6j20pYz44duaGYlftM",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "jfigfz2lr8TJpPkc9qUr",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4-element Vector{Int64}:\n",
       " 1\n",
       " 3\n",
       " 2\n",
       " 4"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A[:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "TgQyzRjZmF2MYwDlRbc4",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "pAwMBeXPpwNKXrgqUY5T",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4-element Vector{Int64}:\n",
       " 1\n",
       " 3\n",
       " 2\n",
       " 4"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vec(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "WuBE2ytQVqkOwffPAjaj",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "It is also quite simple to create a vector using square bracket, and Julia assumes 1D vector as column vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "AJXYJC3teG5k7KCgzvLF",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "iaLhBWdNb2eARCDANt7v",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Vector{Int64}:\n",
       " 1\n",
       " 2"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = [1, 2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "DGK0jLtdDWpqwraPcHV7",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "There are built-in basic linear algebraic operations, such as"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "vZom0KCJWmXSbl7L781w",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "soVwLko8xVpDZXN4J5VB",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Matrix{Float64}:\n",
       " -2.0   1.0\n",
       "  1.5  -0.5"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inv(A)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "NdtLeep4T377vCWbOzHN",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "0A7BLmBzUXuNHVYzyXne",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Vector{Int64}:\n",
       "  5\n",
       " 11"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A * x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "NMwRuOqttrMsxsIsKN7L",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "tgZYtecgR5f1OVSZNuFp",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Vector{Float64}:\n",
       " 0.0\n",
       " 0.5"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A \\ x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "sOJ8KspAaRZoIjILeUu7",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "y1eFavkEpotPGAIcnqC3",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-2.0"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "det(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "oe6QqpBxaUDdck3PGxbm",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "## Just-in-time (JIT) compilation\n",
    "\n",
    "Julia code is just-in-time compiled---i.e., every statement runs using compiled functions which are either compiled right before they are used, or cached compilations from before. This is different from e.g. C, which is compiled before executed. Due to the JIT compilation, the first run of a function/operation typically takes slightly longer time as it needs to compile first. An example is shown below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "1kZsHHiCUwcLQH7QirG6",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": null
    }
   },
   "outputs": [],
   "source": [
    "A = randn(Float32, 10^4, 10^4);\n",
    "b = randn(Float32, 10^4);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "gYDLOiHAEiebqiBK4P1C",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     },
     "outputId": {
      "block": "QVSmagjtWRWDWDLRL9Qx",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0.080792 seconds (290.82 k allocations: 16.171 MiB, 81.98% compilation time)\n"
     ]
    }
   ],
   "source": [
    "# first run\n",
    "@time A * b;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "uHTaagnPh6G05j0mpyAf",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     },
     "outputId": {
      "block": "MRKRRimbn1Zp15YHGd0a",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0.017111 seconds (2 allocations: 39.109 KiB)\n"
     ]
    }
   ],
   "source": [
    "# second run\n",
    "@time A * b;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "8mgnd43FXBcoUNEYEQPg",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "The first time of execution this \"Float32 matrix and Float 32 vector product\" takes longer time and more memory allocations as this is the first time to run such functionality in the current notebook. In order for robust estimate of the execution time and memory usage, we can use the `@benchmark` macro in [BenchmarkTools.jl](https://github.com/JuliaCI/BenchmarkTools.jl.git) package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "UrbUcSCok1EkDrrgNkrO",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     },
     "outputId": {
      "block": "8fM6zeFYU2DRnJpqxS7m",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: 205 samples with 1 evaluation.\n",
       " Range \u001b[90m(\u001b[39m\u001b[36m\u001b[1mmin\u001b[22m\u001b[39m … \u001b[35mmax\u001b[39m\u001b[90m):  \u001b[39m\u001b[36m\u001b[1m16.436 ms\u001b[22m\u001b[39m … \u001b[35m 27.088 ms\u001b[39m  \u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmin … max\u001b[90m): \u001b[39m0.00% … 0.00%\n",
       " Time  \u001b[90m(\u001b[39m\u001b[34m\u001b[1mmedian\u001b[22m\u001b[39m\u001b[90m):     \u001b[39m\u001b[34m\u001b[1m24.335 ms               \u001b[22m\u001b[39m\u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmedian\u001b[90m):    \u001b[39m0.00%\n",
       " Time  \u001b[90m(\u001b[39m\u001b[32m\u001b[1mmean\u001b[22m\u001b[39m ± \u001b[32mσ\u001b[39m\u001b[90m):   \u001b[39m\u001b[32m\u001b[1m24.451 ms\u001b[22m\u001b[39m ± \u001b[32m909.266 μs\u001b[39m  \u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmean ± σ\u001b[90m):  \u001b[39m0.00% ± 0.00%\n",
       "\n",
       "  \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m▃\u001b[39m▅\u001b[39m█\u001b[34m▆\u001b[39m\u001b[32m▁\u001b[39m\u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \n",
       "  \u001b[39m▂\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▂\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▁\u001b[39m▂\u001b[39m▁\u001b[39m▂\u001b[39m▅\u001b[39m▇\u001b[39m█\u001b[39m█\u001b[39m█\u001b[34m█\u001b[39m\u001b[32m█\u001b[39m\u001b[39m█\u001b[39m▇\u001b[39m▅\u001b[39m▆\u001b[39m▅\u001b[39m█\u001b[39m▄\u001b[39m▅\u001b[39m▃\u001b[39m \u001b[39m▃\n",
       "  16.4 ms\u001b[90m         Histogram: frequency by time\u001b[39m           26 ms \u001b[0m\u001b[1m<\u001b[22m\n",
       "\n",
       " Memory estimate\u001b[90m: \u001b[39m\u001b[33m39.11 KiB\u001b[39m, allocs estimate\u001b[90m: \u001b[39m\u001b[33m2\u001b[39m."
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@benchmark A * b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "uYr8Te1YVROXs2LLpVn3",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "## Multiple dispatch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "YGbd9s4sS5lt5mBitmkJ",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 2
     }
    }
   },
   "source": [
    "The core design to make Julia fast is type-stability through specialization via multiple-dispatch. A single function in Julia can take different types of input and compile the function in specialized ways. This \"specialization\" brings huge performance gains to Julia. Let's first go over a simple example, namely scalar multiplication, and use the macro `@which` to check the method being used"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "u5ZkBLAwRv5M7j25UPi4",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "YdkWPSwChQGm4nl4ZBDY",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "*(x::<b>T</b>, y::<b>T</b>)<i> where T<:Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8}</i> in Base at <a href=\"https://github.com/JuliaLang/julia/tree/bf534986350a991e4a1b29126de0342ffd76205e/base/int.jl#L88\" target=\"_blank\">int.jl:88</a>"
      ],
      "text/plain": [
       "*(x::T, y::T) where T<:Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8} in Base at int.jl:88"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@which 1 * 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "rRJvyj4tLEoUJ4PPgVCr",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 2
     }
    }
   },
   "source": [
    "In this example, we want to compute the multiplication of 2 integers. As we can see, the underlying method is running the multiplication of integers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "v3Biac2yHp3BpeJgu1Sq",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "WMcGFE2fQEd9H9vBvN4A",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "*(x::<b>Float64</b>, y::<b>Float64</b>) in Base at <a href=\"https://github.com/JuliaLang/julia/tree/bf534986350a991e4a1b29126de0342ffd76205e/base/float.jl#L405\" target=\"_blank\">float.jl:405</a>"
      ],
      "text/plain": [
       "*(x::Float64, y::Float64) in Base at float.jl:405"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@which 1.0 * 2.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "2T7liWcDr9NnK9O1YPKi",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 2
     }
    }
   },
   "source": [
    "If we multiply 2 floating point numbers, we can see that a different method is used, which is designed specifically for the type `double` (`Float64` in Julia)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "FpPubvtI0YX60txz6QVj",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "lYCTmgnCmlYK58ximqYG",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "*(x::<b>Number</b>, y::<b>Number</b>) in Base at <a href=\"https://github.com/JuliaLang/julia/tree/bf534986350a991e4a1b29126de0342ffd76205e/base/promotion.jl#L380\" target=\"_blank\">promotion.jl:380</a>"
      ],
      "text/plain": [
       "*(x::Number, y::Number) in Base at promotion.jl:380"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@which 1.0 * 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "FS63loH1TnB2lnm7IXzm",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "From here, we can see that Julia can't find a specialized way to compute `1.0 * 2` according to the type of inputs. In the end, it chooses to use the very generic `Number` to describe the type of `1.0` and `2` and to perform the multiplication. Note that this is one of the cases that we want to avoid during large-scale computations. We usually want to make sure that the type of input and output is the same in our functions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "7hfTCzCUjKFjmVCLMASX",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "This example demonstrates that the same function (multiplication `*` in this case) is executed via different compiled code blocks that are specialized for different kinds of input. This feature brings huge performance gain and potentially makes Julia to run almost as fast as C/Fortran (which are strictly typed). With that being said, we should also take advantage of this type-stability when implementing functions---i.e., dispatch a function to multiple types of input."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "521ePpQDDF8fLEYXWlVw",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "Let's show a simple example: we can define a `foo` function that works for different kinds of input. Let's first build it to take `String` inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "Di30bQaTb27WJY4W0CAr",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "V7QbdvaSCgssB7VolRpI",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "foo (generic function with 1 method)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "foo(x::String, y::String) = println(\"My inputs x and y are both strings!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "S26WKuQBdN9RdHlobVws",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "PnztgovBQVLvvK3RL0Eu",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "My inputs x and y are both strings!\n"
     ]
    }
   ],
   "source": [
    "foo(\"hello\", \"hi!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "0oVxs7rIYrlZpcmXx2wp",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "Notice that now `foo` is a generic function with 1 method (a.k.a. taking 2 `String` and output a sentence). Now the `foo` function won't work for numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "nr00jYTcsWWdq2R9B8Uz",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "0eTv0R89fqhJ5dzKhCfs",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "ename": "LoadError",
     "evalue": "MethodError: no method matching foo(::Int64, ::Int64)",
     "output_type": "error",
     "traceback": [
      "MethodError: no method matching foo(::Int64, ::Int64)",
      "",
      "Stacktrace:",
      " [1] top-level scope",
      "   @ In[23]:1",
      " [2] eval",
      "   @ ./boot.jl:373 [inlined]",
      " [3] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)",
      "   @ Base ./loading.jl:1196"
     ]
    }
   ],
   "source": [
    "foo(3, 4) # doesn't work here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "EsbPEvXr0UijWSDH87f2",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "To get `foo` to work on integer (Int) inputs, let's add `::Int` onto our input arguments when we declare `foo`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "5lkmvZobamV5rVj8lpgy",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "In9cTkEA7ZCJfisu2Vug",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "foo (generic function with 2 methods)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "foo(x::Int, y::Int) = println(\"My inputs x and y are both integers!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "ZPsAsVh3HtlqGB987FkI",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "woL7imvjet4IcizQqOF4",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "My inputs x and y are both integers!\n"
     ]
    }
   ],
   "source": [
    "foo(3, 4) # works now"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "sVdM4Vevd3kdPedEWZPC",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "Now `foo(::Int, ::Int)` works as well. Notice that it says `foo (generic function with 2 methods)`. This means `foo` is a general function but contains 2 methods: the first one works for 2 strings and the second one works for 2 numbers. This means that the functionality of `foo` on 2 strings isn't overwritten when you define how to run `foo` on another type of input(s). Instead, we just added an additional method to the generic function `foo`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "dcr7skUc0ZH77GatHTc7",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "R72tsBLYGb1Vp9H5qbyG",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "My inputs x and y are both strings!\n"
     ]
    }
   ],
   "source": [
    "foo(\"hello\", \"hi!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "M0tpJDB0586f6wqqDEGF",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "Then let's show an explanatory example to discuss how we should write code in Julia. Suppose we want to build a function to take the 2 norm of a scalar/vector/matrix (while we choose to ignore the existing norm functionality). A naive way to implement this would be"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "kao5EARr4ZnUzFcTrivk",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "NhIn5Ja3OSgE2HggdE5k",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "my2norm (generic function with 1 method)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function my2norm(x)\n",
    "    if isa(x, Number)\n",
    "        return abs(x)\n",
    "    elseif isa(x, Vector)\n",
    "        return sqrt(sum(x.^2))\n",
    "    elseif isa(x, Matrix)\n",
    "        return return maximum(svdvals(x))\n",
    "    else\n",
    "        println(\"my function is not defined for this kind of input\")\n",
    "        return -1\n",
    "    end\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "oX1uNI3g96pMG0qjHuqC",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "In the above code block, we defined a function `my2norm` that uses nested if-else to find the type of `x` and perform the corresponding operation (absolute value for scalar, euclidean norm for vector, and largest singular value for matrix)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "l0FNE8gx9hreKcFyHNt6",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "However, there is a smarter and cleaner way to write this function, which provides exactly the same functionality."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "fynU86SHuFeoV7y9DAB5",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "5vG3F9SVPfbeiI0i2ze3",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "mysmart2norm (generic function with 4 methods)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mysmart2norm(x::Number) = abs(x)\n",
    "mysmart2norm(x::Vector) = sqrt(sum(x.^2))\n",
    "mysmart2norm(x::Matrix) = maximum(svdvals(x))\n",
    "mysmart2norm(x) = begin println(\"my function is not defined for this kind of input\"); return -1; end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "6HZOlRAD7kZmpeSW1P7I",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     },
     "outputId": {
      "block": "HqVgsowEo3ESeSYgqS1q",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 3
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[32m\u001b[1mTest Passed\u001b[22m\u001b[39m\n",
       "  Expression: my2norm(x) == mysmart2norm(x)\n",
       "   Evaluated: 19.768341325704956 == 19.768341325704956"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = randn(10^2, 10^2);\n",
    "@test my2norm(x) == mysmart2norm(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "nWQwwEaf77NHypCw1dXP",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "This `mysmart2norm` will lead to cleaner and more maintainable code: when a new type comes to our sight (e.g. an abstract type defined by user), then we don't need to dig into the very long `my2norm` to add another set of `ifelse`. A simple `mysmart2norm(x::NewType)` will do the work. For example, check [here](https://github.com/slimgroup/JUDI.jl/blob/master/src/TimeModeling/Types/abstract.jl#L99) to see how [JUDI.jl](https://github.com/slimgroup/JUDI.jl) defines `norm` for `judiMultiSourceVector`, an abstract type defined for seismic data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "IFUSjvkbBP64GXSLb6w8",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "## Type stability\n",
    "\n",
    "The multiple dispatch feature makes Julia easy to use like Python, and fast to run like C. In order for Julia compiler to produce highly efficient code and show ultimate performance, we should write type-stable code---i.e., given the source code and the concrete type of all arguments, the compiler is able to infer the concrete type of every variable and expression within the method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "8wGy6Q5JvO1L4z9rutw5",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "Let's consider the following example. `funcN` takes a sequence of `Number` (`...` is the splatting operation) and it aims to compute the product of this entire array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "PdauIKOHpmLlxq0E0mvQ",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "Un5Ea5pR5LuCyOwU2p2h",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "funcN (generic function with 1 method)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function funcN(a::Number...)\n",
    "    c=copy(a[1])\n",
    "    L=length(a)\n",
    "    for l=2:L\n",
    "        c=c*a[l]\n",
    "    end\n",
    "    return c\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "viTpme5Mrvb45APVNrZz",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "Now we want to compute the product of the sequence `1, 1.5, 2, 2.5, 3`. We have 2 following options:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "15YqPa0evCeJkcypizWq",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 2
     },
     "outputId": {
      "block": "s6RhnpQqdbl15qHYemEC",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 2
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: 10000 samples with 413 evaluations.\n",
       " Range \u001b[90m(\u001b[39m\u001b[36m\u001b[1mmin\u001b[22m\u001b[39m … \u001b[35mmax\u001b[39m\u001b[90m):  \u001b[39m\u001b[36m\u001b[1m234.228 ns\u001b[22m\u001b[39m … \u001b[35m 50.863 μs\u001b[39m  \u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmin … max\u001b[90m): \u001b[39m0.00% … 99.35%\n",
       " Time  \u001b[90m(\u001b[39m\u001b[34m\u001b[1mmedian\u001b[22m\u001b[39m\u001b[90m):     \u001b[39m\u001b[34m\u001b[1m259.131 ns               \u001b[22m\u001b[39m\u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmedian\u001b[90m):    \u001b[39m0.00%\n",
       " Time  \u001b[90m(\u001b[39m\u001b[32m\u001b[1mmean\u001b[22m\u001b[39m ± \u001b[32mσ\u001b[39m\u001b[90m):   \u001b[39m\u001b[32m\u001b[1m272.421 ns\u001b[22m\u001b[39m ± \u001b[32m507.021 ns\u001b[39m  \u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmean ± σ\u001b[90m):  \u001b[39m1.86% ±  0.99%\n",
       "\n",
       "  \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m▄\u001b[39m█\u001b[39m \u001b[39m▁\u001b[39m \u001b[39m█\u001b[34m \u001b[39m\u001b[39m \u001b[39m▁\u001b[39m \u001b[39m \u001b[32m \u001b[39m\u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \n",
       "  \u001b[39m▃\u001b[39m▃\u001b[39m▂\u001b[39m▃\u001b[39m█\u001b[39m█\u001b[39m▄\u001b[39m█\u001b[39m▄\u001b[39m█\u001b[34m▅\u001b[39m\u001b[39m▅\u001b[39m█\u001b[39m▄\u001b[39m▅\u001b[32m▅\u001b[39m\u001b[39m▃\u001b[39m▃\u001b[39m▄\u001b[39m▃\u001b[39m▃\u001b[39m▅\u001b[39m▆\u001b[39m▄\u001b[39m▃\u001b[39m▂\u001b[39m▃\u001b[39m▃\u001b[39m▂\u001b[39m▂\u001b[39m▃\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▃\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▁\u001b[39m▁\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m \u001b[39m▃\n",
       "  234 ns\u001b[90m           Histogram: frequency by time\u001b[39m          390 ns \u001b[0m\u001b[1m<\u001b[22m\n",
       "\n",
       " Memory estimate\u001b[90m: \u001b[39m\u001b[33m96 bytes\u001b[39m, allocs estimate\u001b[90m: \u001b[39m\u001b[33m6\u001b[39m."
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x1 = (1, 1.5, 2, 2.5, 3)\n",
    "@benchmark funcN(x1...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "8ktvGX7hb5KL0meBjqHM",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 2
     },
     "outputId": {
      "block": "kuh5LmVplVF6Ydrmk1VR",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 2
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BenchmarkTools.Trial: 10000 samples with 803 evaluations.\n",
       " Range \u001b[90m(\u001b[39m\u001b[36m\u001b[1mmin\u001b[22m\u001b[39m … \u001b[35mmax\u001b[39m\u001b[90m):  \u001b[39m\u001b[36m\u001b[1m146.661 ns\u001b[22m\u001b[39m … \u001b[35m 29.191 μs\u001b[39m  \u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmin … max\u001b[90m): \u001b[39m0.00% … 99.30%\n",
       " Time  \u001b[90m(\u001b[39m\u001b[34m\u001b[1mmedian\u001b[22m\u001b[39m\u001b[90m):     \u001b[39m\u001b[34m\u001b[1m158.966 ns               \u001b[22m\u001b[39m\u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmedian\u001b[90m):    \u001b[39m0.00%\n",
       " Time  \u001b[90m(\u001b[39m\u001b[32m\u001b[1mmean\u001b[22m\u001b[39m ± \u001b[32mσ\u001b[39m\u001b[90m):   \u001b[39m\u001b[32m\u001b[1m171.676 ns\u001b[22m\u001b[39m ± \u001b[32m495.770 ns\u001b[39m  \u001b[90m┊\u001b[39m GC \u001b[90m(\u001b[39mmean ± σ\u001b[90m):  \u001b[39m4.99% ±  1.72%\n",
       "\n",
       "  \u001b[39m \u001b[39m \u001b[39m \u001b[39m▆\u001b[39m▁\u001b[39m█\u001b[39m \u001b[34m▅\u001b[39m\u001b[39m \u001b[39m▅\u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[32m \u001b[39m\u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \u001b[39m \n",
       "  \u001b[39m▃\u001b[39m▅\u001b[39m▄\u001b[39m█\u001b[39m█\u001b[39m█\u001b[39m▆\u001b[34m█\u001b[39m\u001b[39m▅\u001b[39m█\u001b[39m▄\u001b[39m▃\u001b[39m▃\u001b[39m▂\u001b[32m▃\u001b[39m\u001b[39m▃\u001b[39m▃\u001b[39m▃\u001b[39m▂\u001b[39m▃\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▁\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m▂\u001b[39m \u001b[39m▃\n",
       "  147 ns\u001b[90m           Histogram: frequency by time\u001b[39m          259 ns \u001b[0m\u001b[1m<\u001b[22m\n",
       "\n",
       " Memory estimate\u001b[90m: \u001b[39m\u001b[33m96 bytes\u001b[39m, allocs estimate\u001b[90m: \u001b[39m\u001b[33m6\u001b[39m."
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x2 = (1.0, 1.5, 2.0, 2.5, 3.0)\n",
    "@benchmark funcN(x2...)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "HTfcGFBL8SLPY9pjFkVF",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "Interesting! Why do they have different runtimes? Julia provides a macro `@code_warntype` to check whether the code is type-stable. Let's see."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "fabuUW88jVkuicxksRcN",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "5Qn1bd4k3672RgiuvvyI",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MethodInstance for funcN(::Int64, ::Float64, ::Int64, ::Float64, ::Int64)\n",
      "  from funcN(a::Number...) in Main at In[30]:1\n",
      "Arguments\n",
      "  #self#\u001b[36m::Core.Const(funcN)\u001b[39m\n",
      "  a\u001b[36m::Tuple{Int64, Float64, Int64, Float64, Int64}\u001b[39m\n",
      "Locals\n",
      "  @_3\u001b[33m\u001b[1m::Union{Nothing, Tuple{Int64, Int64}}\u001b[22m\u001b[39m\n",
      "  L\u001b[36m::Int64\u001b[39m\n",
      "  c\u001b[91m\u001b[1m::Union{Float64, Int64}\u001b[22m\u001b[39m\n",
      "  l\u001b[36m::Int64\u001b[39m\n",
      "Body\u001b[91m\u001b[1m::Union{Float64, Int64}\u001b[22m\u001b[39m\n",
      "\u001b[90m1 ─\u001b[39m %1  = Base.getindex(a, 1)\u001b[36m::Int64\u001b[39m\n",
      "\u001b[90m│  \u001b[39m       (c = Main.copy(%1))\n",
      "\u001b[90m│  \u001b[39m       (L = Main.length(a))\n",
      "\u001b[90m│  \u001b[39m %4  = (2:L::Core.Const(5))\u001b[36m::Core.Const(2:5)\u001b[39m\n",
      "\u001b[90m│  \u001b[39m       (@_3 = Base.iterate(%4))\n",
      "\u001b[90m│  \u001b[39m %6  = (@_3::Core.Const((2, 2)) === nothing)\u001b[36m::Core.Const(false)\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %7  = Base.not_int(%6)\u001b[36m::Core.Const(true)\u001b[39m\n",
      "\u001b[90m└──\u001b[39m       goto #4 if not %7\n",
      "\u001b[90m2 ┄\u001b[39m %9  = @_3\u001b[36m::Tuple{Int64, Int64}\u001b[39m\n",
      "\u001b[90m│  \u001b[39m       (l = Core.getfield(%9, 1))\n",
      "\u001b[90m│  \u001b[39m %11 = Core.getfield(%9, 2)\u001b[36m::Int64\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %12 = c\u001b[91m\u001b[1m::Union{Float64, Int64}\u001b[22m\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %13 = Base.getindex(a, l)\u001b[91m\u001b[1m::Union{Float64, Int64}\u001b[22m\u001b[39m\n",
      "\u001b[90m│  \u001b[39m       (c = %12 * %13)\n",
      "\u001b[90m│  \u001b[39m       (@_3 = Base.iterate(%4, %11))\n",
      "\u001b[90m│  \u001b[39m %16 = (@_3 === nothing)\u001b[36m::Bool\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %17 = Base.not_int(%16)\u001b[36m::Bool\u001b[39m\n",
      "\u001b[90m└──\u001b[39m       goto #4 if not %17\n",
      "\u001b[90m3 ─\u001b[39m       goto #2\n",
      "\u001b[90m4 ┄\u001b[39m       return c\n",
      "\n"
     ]
    }
   ],
   "source": [
    "@code_warntype funcN(x1...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "iooxa": {
     "id": {
      "block": "FB7Z2F3AcapLM0iLRHJC",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     },
     "outputId": {
      "block": "64wH9EBBHsqG6AsdgA8V",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MethodInstance for funcN(::Float64, ::Float64, ::Float64, ::Float64, ::Float64)\n",
      "  from funcN(a::Number...) in Main at In[30]:1\n",
      "Arguments\n",
      "  #self#\u001b[36m::Core.Const(funcN)\u001b[39m\n",
      "  a\u001b[36m::NTuple{5, Float64}\u001b[39m\n",
      "Locals\n",
      "  @_3\u001b[33m\u001b[1m::Union{Nothing, Tuple{Int64, Int64}}\u001b[22m\u001b[39m\n",
      "  L\u001b[36m::Int64\u001b[39m\n",
      "  c\u001b[36m::Float64\u001b[39m\n",
      "  l\u001b[36m::Int64\u001b[39m\n",
      "Body\u001b[36m::Float64\u001b[39m\n",
      "\u001b[90m1 ─\u001b[39m %1  = Base.getindex(a, 1)\u001b[36m::Float64\u001b[39m\n",
      "\u001b[90m│  \u001b[39m       (c = Main.copy(%1))\n",
      "\u001b[90m│  \u001b[39m       (L = Main.length(a))\n",
      "\u001b[90m│  \u001b[39m %4  = (2:L::Core.Const(5))\u001b[36m::Core.Const(2:5)\u001b[39m\n",
      "\u001b[90m│  \u001b[39m       (@_3 = Base.iterate(%4))\n",
      "\u001b[90m│  \u001b[39m %6  = (@_3::Core.Const((2, 2)) === nothing)\u001b[36m::Core.Const(false)\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %7  = Base.not_int(%6)\u001b[36m::Core.Const(true)\u001b[39m\n",
      "\u001b[90m└──\u001b[39m       goto #4 if not %7\n",
      "\u001b[90m2 ┄\u001b[39m %9  = @_3\u001b[36m::Tuple{Int64, Int64}\u001b[39m\n",
      "\u001b[90m│  \u001b[39m       (l = Core.getfield(%9, 1))\n",
      "\u001b[90m│  \u001b[39m %11 = Core.getfield(%9, 2)\u001b[36m::Int64\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %12 = c\u001b[36m::Float64\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %13 = Base.getindex(a, l)\u001b[36m::Float64\u001b[39m\n",
      "\u001b[90m│  \u001b[39m       (c = %12 * %13)\n",
      "\u001b[90m│  \u001b[39m       (@_3 = Base.iterate(%4, %11))\n",
      "\u001b[90m│  \u001b[39m %16 = (@_3 === nothing)\u001b[36m::Bool\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %17 = Base.not_int(%16)\u001b[36m::Bool\u001b[39m\n",
      "\u001b[90m└──\u001b[39m       goto #4 if not %17\n",
      "\u001b[90m3 ─\u001b[39m       goto #2\n",
      "\u001b[90m4 ┄\u001b[39m       return c\n",
      "\n"
     ]
    }
   ],
   "source": [
    "@code_warntype funcN(x2...)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "Zt11kt15JYsmLwb2CYpk",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "A red line in the output indicates type-instability in the code. So what happened? In the function, we first read the variable `c` as the first entry of the sequence. In `x1`, the first entry is an `Int64`, so `c` will get the type of `Int64` as well initially. However, `c` later multiplies with `Float64` entries (namely `1.5` and `2.5`). Therefore, it is forced to change its type of `Float64`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "iooxa": {
     "id": {
      "block": "2Omw0q3tu6GpnLQtsBvt",
      "project": "4qWjnDwuuPUd70Df3LsC",
      "version": 1
     }
    }
   },
   "source": [
    "When we are very sure of the input and output types of a function we expect (like in this case we know everything ends up to be `Float64`), we should keep in mind to write type-stable code so that Julia doesn't find a hard time to struggle on the type of the variables."
   ]
  }
 ],
 "metadata": {
  "iooxa": {
   "id": {
    "block": "75XEhpExCShLZopzZkJe",
    "project": "4qWjnDwuuPUd70Df3LsC",
    "version": 6
   }
  },
  "kernelspec": {
   "display_name": "Julia 1.7.2",
   "language": "julia",
   "name": "julia-1.7"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}