data_loading.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DlN1gzqNsDYY"
   },
   "source": [
    "# Data loading and preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WJ4glUc_NN9W"
   },
   "source": [
    "Here we walk through the necessary steps to get your data into ready for scvi-tools."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 382
    },
    "id": "N3oQkal_sLcr",
    "outputId": "2ab53643-9881-4376-9f0e-d22ab67c6b9c"
   },
   "outputs": [],
   "source": [
    "!pip install --quiet scvi-colab\n",
    "from scvi_colab import install\n",
    "install()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "VpgMddaEr-lS"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Global seed set to 0\n",
      "/Users/jhong/miniforge3/envs/scvi-env/lib/python3.9/site-packages/docrep/decorators.py:43: SyntaxWarning: 'param_size_factor_key' is not a valid key!\n",
      "  doc = func(self, args[0].__doc__, *args[1:], **kwargs)\n"
     ]
    }
   ],
   "source": [
    "import scvi\n",
    "import scanpy as sc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "80gW_yfkuzB9"
   },
   "source": [
    "## Loading data\n",
    "\n",
    "scvi-tools supports the [AnnData](https://anndata.readthedocs.io/en/latest/) data format, which also underlies [Scanpy](https://scanpy.readthedocs.io/en/stable/). AnnData is quite similar to other popular single cell objects like that of [Seurat](https://github.com/satijalab/seurat/wiki) and [SingleCellExperiment](https://bioconductor.org/packages/release/bioc/vignettes/SingleCellExperiment/inst/doc/intro.html). In particular, it allows cell-level and feature-level metadata to coexist in the same data structure as the molecular counts.\n",
    "\n",
    "It's also now possible to automatically convert these R-based objects to AnnData within a Jupyter notebook. See the following [tutorial](https://github.com/LuckyMD/Code_snippets/blob/master/Seurat_to_anndata.ipynb) for more information.\n",
    "\n",
    "scvi-tools has a number of convenience methods for loading data from `.csv`, `.loom`, and `.h5ad` formats. To load outputs from Cell Ranger, please use Scanpy's [reading functionality](https://scanpy.readthedocs.io/en/latest/api.html#reading).\n",
    "\n",
    "Let us now download an AnnData object (`.h5ad` format) and load it using scvi-tools."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5-_VxGJ9xWaW"
   },
   "source": [
    "### PBMC3k"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 208
    },
    "id": "obl1PpsLu2EX",
    "outputId": "39369067-518e-4a11-fa0f-860463e283f4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2022-02-18 12:12:58--  http://falexwolf.de/data/pbmc3k_raw.h5ad\n",
      "Resolving falexwolf.de (falexwolf.de)... 85.13.135.70\n",
      "Connecting to falexwolf.de (falexwolf.de)|85.13.135.70|:80... connected.\n",
      "HTTP request sent, awaiting response... 301 Moved Permanently\n",
      "Location: https://falexwolf.de/data/pbmc3k_raw.h5ad [following]\n",
      "--2022-02-18 12:12:58--  https://falexwolf.de/data/pbmc3k_raw.h5ad\n",
      "Connecting to falexwolf.de (falexwolf.de)|85.13.135.70|:443... connected.\n",
      "HTTP request sent, awaiting response... 301 Moved Permanently\n",
      "Location: https://falexwolf.me/data/pbmc3k_raw.h5ad [following]\n",
      "--2022-02-18 12:12:59--  https://falexwolf.me/data/pbmc3k_raw.h5ad\n",
      "Resolving falexwolf.me (falexwolf.me)... 75.2.60.5\n",
      "Connecting to falexwolf.me (falexwolf.me)|75.2.60.5|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 5855727 (5.6M) [application/octet-stream]\n",
      "Saving to: ‘pbmc3k_raw.h5ad’\n",
      "\n",
      "pbmc3k_raw.h5ad     100%[===================>]   5.58M  10.6MB/s    in 0.5s    \n",
      "\n",
      "2022-02-18 12:13:00 (10.6 MB/s) - ‘pbmc3k_raw.h5ad’ saved [5855727/5855727]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget 'http://falexwolf.de/data/pbmc3k_raw.h5ad'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "UX4nKs0vwV3S"
   },
   "outputs": [],
   "source": [
    "pbmc3k = scvi.data.read_h5ad(\"pbmc3k_raw.h5ad\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "id": "yM_lXgv4xAvv",
    "outputId": "07385465-c290-47e2-d732-89c327909468"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AnnData object with n_obs × n_vars = 2700 × 32738\n",
       "    var: 'gene_ids'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pbmc3k"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Gw44EIlExDUc"
   },
   "source": [
    "This is a fairly simple object, it just contains the count data and the ENSEMBL ids for the genes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 237
    },
    "id": "JmqvrG8AxCrO",
    "outputId": "8252b5ae-9f5d-434b-e308-766f95f6b994"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gene_ids</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>index</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>MIR1302-10</th>\n",
       "      <td>ENSG00000243485</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>FAM138A</th>\n",
       "      <td>ENSG00000237613</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>OR4F5</th>\n",
       "      <td>ENSG00000186092</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RP11-34P13.7</th>\n",
       "      <td>ENSG00000238009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RP11-34P13.8</th>\n",
       "      <td>ENSG00000239945</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     gene_ids\n",
       "index                        \n",
       "MIR1302-10    ENSG00000243485\n",
       "FAM138A       ENSG00000237613\n",
       "OR4F5         ENSG00000186092\n",
       "RP11-34P13.7  ENSG00000238009\n",
       "RP11-34P13.8  ENSG00000239945"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pbmc3k.var.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1mhNtRfhxX1_"
   },
   "source": [
    "### PBMC5k\n",
    "\n",
    "As another example, let's download a dataset from 10x Genomics. This data was obtained from a CITE-seq experiment, so it also contains protein count data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 208
    },
    "id": "22rXIMVgxVVy",
    "outputId": "321a081f-8223-406a-c1ae-bb7283dc3645"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2022-02-18 12:13:01--  https://cf.10xgenomics.com/samples/cell-exp/3.0.2/5k_pbmc_protein_v3/5k_pbmc_protein_v3_filtered_feature_bc_matrix.h5\n",
      "Resolving cf.10xgenomics.com (cf.10xgenomics.com)... 104.18.1.173, 104.18.0.173\n",
      "Connecting to cf.10xgenomics.com (cf.10xgenomics.com)|104.18.1.173|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 17129253 (16M) [binary/octet-stream]\n",
      "Saving to: ‘5k_pbmc_protein_v3_filtered_feature_bc_matrix.h5’\n",
      "\n",
      "5k_pbmc_protein_v3_ 100%[===================>]  16.33M  9.91MB/s    in 1.6s    \n",
      "\n",
      "2022-02-18 12:13:03 (9.91 MB/s) - ‘5k_pbmc_protein_v3_filtered_feature_bc_matrix.h5’ saved [17129253/17129253]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget https://cf.10xgenomics.com/samples/cell-exp/3.0.2/5k_pbmc_protein_v3/5k_pbmc_protein_v3_filtered_feature_bc_matrix.h5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "TypCa4tExtyp",
    "outputId": "5cf6a4c5-de7d-4c7e-8114-47ce4fcfe19f"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Variable names are not unique. To make them unique, call `.var_names_make_unique`.\n"
     ]
    }
   ],
   "source": [
    "pbmc5k = sc.read_10x_h5(\n",
    "    \"5k_pbmc_protein_v3_filtered_feature_bc_matrix.h5\", \n",
    "    gex_only=False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Yg4KB0idx-9z"
   },
   "source": [
    "It's often helpful to give the gene names unique names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "fK451tYtx-cH"
   },
   "outputs": [],
   "source": [
    "pbmc5k.var_names_make_unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "43BWkucQye4C"
   },
   "source": [
    "We can see that `adata.X` contains the concatenated gene and protein expression data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "QsmoVmGmyF8T",
    "outputId": "bdb5b25d-9559-4b66-dde2-b6fb8a058282"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Antibody Capture', 'Gene Expression'], dtype='object')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pbmc5k.var.feature_types.astype(\"category\").cat.categories"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oOYhcGtOyFHP"
   },
   "source": [
    "We can use scvi-tools to organize this object, which places the protein expression in `adata.obms[\"protein_expression]`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "id": "fmjHhbY2yEYT",
    "outputId": "6886fc06-6066-4191-9f45-694ab6d960f3"
   },
   "outputs": [],
   "source": [
    "scvi.data.organize_cite_seq_10x(pbmc5k)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 69
    },
    "id": "GhoSe7nayxbJ",
    "outputId": "10ff1a69-2950-435a-e5bc-f9aa6a648c3e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AnnData object with n_obs × n_vars = 5247 × 33538\n",
       "    var: 'gene_ids', 'feature_types', 'genome'\n",
       "    obsm: 'protein_expression'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pbmc5k"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VItecTR9zL7w"
   },
   "source": [
    "### Concatenate the datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "iX6wZkSYzKH-"
   },
   "outputs": [],
   "source": [
    "adata = pbmc5k.concatenate(pbmc3k)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EQdWs4BPz3E2"
   },
   "source": [
    "Notice that the resulting AnnData has a batch key in `.obs`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "iIhwaX1az1rB",
    "outputId": "f21bf106-6cee-4475-d3a0-0b65fed6995c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>batch</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>GTAGCTAAGTTCATGC-1-0</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CACTGAAAGACGGTCA-1-0</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AAGGAATCAGTCGCAC-1-0</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CCACTTGAGCGTTACT-1-0</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GAGGGATGGGAAAT-1-1</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     batch\n",
       "GTAGCTAAGTTCATGC-1-0     0\n",
       "CACTGAAAGACGGTCA-1-0     0\n",
       "AAGGAATCAGTCGCAC-1-0     0\n",
       "CCACTTGAGCGTTACT-1-0     0\n",
       "GAGGGATGGGAAAT-1-1       1"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adata.obs.sample(n=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TH0sTU2j0gM0"
   },
   "source": [
    "## Preprocessing the data\n",
    "\n",
    "It is common to remove outliers, and even perform feature selection before model fitting. We prefer the [Scanpy preprocessing module](https://scanpy.readthedocs.io/en/stable/api/index.html#module-scanpy.pp) at this stage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 86
    },
    "id": "eUN2B0pm0iah",
    "outputId": "28a116f8-c3fd-4f47-adcf-4c619aaa2799"
   },
   "outputs": [],
   "source": [
    "sc.pp.filter_genes(adata, min_counts=3)\n",
    "sc.pp.filter_cells(adata, min_counts=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AlMB0Tq601IQ"
   },
   "source": [
    "As it is popular to normalize the data for many methods, we can use Scanpy for this; however, it's important to keep the count information intact for scvi-tools models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "id": "lwdhWVBe00mW"
   },
   "outputs": [],
   "source": [
    "adata.layers[\"counts\"] = adata.X.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4X9Ruf2D1HGR"
   },
   "source": [
    "Now we can proceed with common normalization methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "T81mge-w1KHF"
   },
   "outputs": [],
   "source": [
    "sc.pp.normalize_total(adata, target_sum=1e4)\n",
    "sc.pp.log1p(adata)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "abt2SyCv1Rk5"
   },
   "source": [
    "We can store the normalized values in `.raw` to keep them safe in the event the anndata gets subsetted feature-wise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "id": "ZbwQTY_h1QT2"
   },
   "outputs": [],
   "source": [
    "adata.raw = adata"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lygvBJwWxSqk"
   },
   "source": [
    "## Register the data with scvi-tools\n",
    "\n",
    "Now that we have an AnnData object, we need to alert scvi-tools of all the interesting data in our object. For example, now that we have batches in our AnnData, we can alert the models that we'd like to perform batch correction. Also, because we have the count data in a layer, we can use the `layer` argument.\n",
    "\n",
    "Normally, we set up the data right before using a model, thus we would call the `setup_anndata` method specific to that model. However, we are not using any particular model here since we are just demonstrating data usage and handling in this tutorial. We will use the SCVI model's `setup_anndata` method here and in what follows for sake of example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7ivqMBFZ1ren"
   },
   "source": [
    "### Basic case"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 225
    },
    "id": "V8eKOa4Uy1WP",
    "outputId": "b26c6733-e4f7-46be-fdec-3cd467f6a7c2"
   },
   "outputs": [],
   "source": [
    "scvi.model.SCVI.setup_anndata(adata, layer=\"counts\", batch_key=\"batch\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pmIEOmbD0E8T"
   },
   "source": [
    "Notice the info messages notify us that batches were detected in the data. Just to demonstrate what happens if we don't include this option:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 191
    },
    "id": "C3ZWlowC0D5V",
    "outputId": "c36f3e22-0c87-49f3-c4c9-93429593474a"
   },
   "outputs": [],
   "source": [
    "scvi.model.SCVI.setup_anndata(adata, layer=\"counts\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XKd16FQX0PXa"
   },
   "source": [
    "Now integration-based tasks can no longer be performed in subsequent models because there is no knowledge of such information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BVIcVMz41sxi"
   },
   "source": [
    "### CITE-seq case"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tBrrUMyu104c"
   },
   "source": [
    "As PBMC5k is a CITE-seq dataset, we can use scvi-tools to register the protein expression. Note that totalVI is the only current model that uses the protein expression. The usage of registered items is model specific. As another example, registering the labels in the AnnData object will not affect totalVI or scVI, but is necessary to run scANVI.\n",
    "\n",
    "We have not preprocessed the `pbmc5k` object, which we do recommend. We show how to run `setup_anndata` in this case for illustrative purposes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 225
    },
    "id": "i7REZQil1o1u",
    "outputId": "4a81b824-c6ca-46a9-830b-651406650cb4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mINFO    \u001b[0m Using column names from columns of adata.obsm\u001b[1m[\u001b[0m\u001b[32m'protein_expression'\u001b[0m\u001b[1m]\u001b[0m                 \n"
     ]
    }
   ],
   "source": [
    "scvi.model.TOTALVI.setup_anndata(pbmc5k, protein_expression_obsm_key=\"protein_expression\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_KQVSyVR3Dk_"
   },
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "\n",
    "Warning\n",
    "\n",
    "After `setup_anndata` has been run, the adata object should not be modified. In other words, the very next step in the workflow is to initialize and train the model of interest (e.g., scVI, totalVI). If you do modify the adata, it's ok, just run `setup_anndata` again -- and then reinitialize the model.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0GuvUEFnF1NT"
   },
   "source": [
    "### Viewing the scvi-tools data setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 644
    },
    "id": "lJngLWo_F0qG",
    "outputId": "fd5eba95-05bc-4ca6-c75d-343b1502add1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mINFO    \u001b[0m Computing empirical prior initialization for protein background.                    \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Anndata setup with scvi-tools version <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.15</span>.0a0.\n",
       "</pre>\n"
      ],
      "text/plain": [
       "Anndata setup with scvi-tools version \u001b[1;36m0.15\u001b[0m.0a0.\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-style: italic\">         Summary Statistics         </span>\n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\">     Summary Stat Key     </span>┃<span style=\"font-weight: bold\"> Value </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\">         n_cells          </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\"> 5247  </span>│\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\">          n_vars          </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\"> 33538 </span>│\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\">         n_labels         </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\">   1   </span>│\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\">         n_batch          </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\">   1   </span>│\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\"> n_extra_categorical_covs </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\">   0   </span>│\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\"> n_extra_continuous_covs  </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\">   0   </span>│\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\">        n_proteins        </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\">  32   </span>│\n",
       "└──────────────────────────┴───────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[3m         Summary Statistics         \u001b[0m\n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1m    Summary Stat Key    \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mValue\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33m        n_cells         \u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m5247 \u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33m         n_vars         \u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m33538\u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33m        n_labels        \u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m  1  \u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33m        n_batch         \u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m  1  \u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33mn_extra_categorical_covs\u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m  0  \u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33mn_extra_continuous_covs \u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m  0  \u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33m       n_proteins       \u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m 32  \u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "└──────────────────────────┴───────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-style: italic\">                   Data Registry                   </span>\n",
       "┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Registry Key </span>┃<span style=\"font-weight: bold\">       scvi-tools Location        </span>┃\n",
       "┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\">      X       </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\">             adata.X              </span>│\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\">    labels    </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\">    adata.obs['_scvi_labels']     </span>│\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\">    batch     </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\">     adata.obs['_scvi_batch']     </span>│\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\">   proteins   </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\"> adata.obsm['protein_expression'] </span>│\n",
       "└──────────────┴──────────────────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[3m                   Data Registry                   \u001b[0m\n",
       "┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mRegistry Key\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      scvi-tools Location       \u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33m     X      \u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m            adata.X             \u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33m   labels   \u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m   adata.obs['_scvi_labels']    \u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33m   batch    \u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m    adata.obs['_scvi_batch']    \u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33m  proteins  \u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128madata.obsm['protein_expression']\u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "└──────────────┴──────────────────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-style: italic\">                     labels State Registry                      </span>\n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\">      Source Location      </span>┃<span style=\"font-weight: bold\"> Categories </span>┃<span style=\"font-weight: bold\"> scvi-tools Encoding </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\"> adata.obs['_scvi_labels'] </span>│<span style=\"color: #008000; text-decoration-color: #008000\">     0      </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\">          0          </span>│\n",
       "└───────────────────────────┴────────────┴─────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[3m                     labels State Registry                      \u001b[0m\n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1m     Source Location     \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mCategories\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mscvi-tools Encoding\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33madata.obs['_scvi_labels']\u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[32m \u001b[0m\u001b[32m    0     \u001b[0m\u001b[32m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m         0         \u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "└───────────────────────────┴────────────┴─────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-style: italic\">                     batch State Registry                      </span>\n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\">     Source Location      </span>┃<span style=\"font-weight: bold\"> Categories </span>┃<span style=\"font-weight: bold\"> scvi-tools Encoding </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│<span style=\"color: #0087ff; text-decoration-color: #0087ff\"> adata.obs['_scvi_batch'] </span>│<span style=\"color: #008000; text-decoration-color: #008000\">     0      </span>│<span style=\"color: #af00d7; text-decoration-color: #af00d7\">          0          </span>│\n",
       "└──────────────────────────┴────────────┴─────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[3m                     batch State Registry                      \u001b[0m\n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1m    Source Location     \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mCategories\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mscvi-tools Encoding\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│\u001b[38;5;33m \u001b[0m\u001b[38;5;33madata.obs['_scvi_batch']\u001b[0m\u001b[38;5;33m \u001b[0m│\u001b[32m \u001b[0m\u001b[32m    0     \u001b[0m\u001b[32m \u001b[0m│\u001b[38;5;128m \u001b[0m\u001b[38;5;128m         0         \u001b[0m\u001b[38;5;128m \u001b[0m│\n",
       "└──────────────────────────┴────────────┴─────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = scvi.model.TOTALVI(pbmc5k)\n",
    "model.view_anndata_setup(pbmc5k)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "data_loading.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}