test_app.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Na0MC_1nzDbi"
      },
      "source": [
        "# Testing your app\n",
        "\n",
        "Once your app is running you can query it. You can simply do that by navigating to the URL that ngrok gave you above or through code in a different python session. For example, once the frontend app is running, you can navigate to:\n",
        "http://YOUR_SERVER_DOMAIN/search?query=hello+world where YOUR_SERVER_DOMAIN is something like XXXX-XX-XX-XX-XX.ngrok.io, which is printed above in Colab or that is your external IP on GCP.\n",
        "\n",
        "The code below shows how to issue a query from python. This is also how our testing code will issue queries to your search engine, so make sure to test your search engine this way after you deploy it to GCP and before submission. Command line instructions for deploying your search engine to GCP are available at `run_frontend_in_gcp.sh`. Note that we will not only issue training queries to your search engine, but also test queries, i.e. queries that you've never seen before."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "_Ol1OArbtK-X"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "EM5ePrRHojbG"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "with open('queries_train.json', 'rt') as f:\n",
        "  queries = json.load(f)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "gWimZWCOy3Ei"
      },
      "outputs": [],
      "source": [
        "def average_precision_at_40(true_list, predicted_list, k=40):\n",
        "    true_set = frozenset(true_list)\n",
        "    predicted_list = predicted_list[:k]\n",
        "    precisions = []\n",
        "    for i,doc_id in enumerate(predicted_list):        \n",
        "        if doc_id in true_set:\n",
        "            prec = (len(precisions)+1) / (i+1)            \n",
        "            precisions.append(prec)\n",
        "    if len(precisions) == 0:\n",
        "        return 0.0\n",
        "    return round(sum(precisions)/len(precisions),3)\n",
        "\n",
        "def average_precision(true_list, predicted_list):\n",
        "    true_set = frozenset(true_list)\n",
        "    precisions = []\n",
        "    for i,doc_id in enumerate(predicted_list):        \n",
        "        if doc_id in true_set:\n",
        "            prec = (len(precisions)+1) / (i+1)            \n",
        "            precisions.append(prec)\n",
        "    if len(precisions) == 0:\n",
        "        return 0.0\n",
        "    return round(sum(precisions)/len(precisions),3)\n",
        "\n",
        "def recall(true_list, predicted_list):\n",
        "    true_set = frozenset(true_list)\n",
        "    rec = 0\n",
        "    for i in predicted_list:\n",
        "      if i in true_set:\n",
        "        rec += 1\n",
        "    return rec/len(true_set)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "id": "dYmNTq9u0ChK",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d577019e-2afc-49a2-f22a-3a55fd3c4a7d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "best marvel movie -> 4\n",
            "How do kids come to world? -> 14\n",
            "Information retrieval -> 14\n",
            "LinkedIn -> 7\n",
            "How to make coffee? -> 4\n",
            "Ritalin -> 4\n",
            "How to make wine at home? -> 16\n",
            "Most expensive city in the world -> 15\n",
            "India -> 27\n",
            "how to make money fast? -> 8\n",
            "Netflix -> 18\n",
            "Apple computer -> 6\n",
            "The Simpsons -> 4\n",
            "World cup -> 4\n",
            "How to lose weight? -> 17\n",
            "Java -> 5\n",
            "Air Jordan -> 8\n",
            "how to deal with depression? -> 12\n",
            "How do you make gold -> 7\n",
            "Marijuana -> 4\n",
            "How to make hummus -> 10\n",
            "Winter -> 31\n",
            "Rick and Morty -> 14\n",
            "Natural Language processing -> 9\n",
            "World Cup 2022 -> 14\n",
            "Dolly the sheep -> 6\n",
            "Cigarettes -> 4\n",
            "What is the best place to live in? -> 16\n",
            "Elon musk -> 13\n",
            "How do you breed flowers? -> 8\n",
            "[('best marvel movie', 2.219882011413574, 0.5, 0.5, 0.022222222222222223), ('How do kids come to world?', 2.740659713745117, 0.127, 0.127, 0.04), ('Information retrieval', 1.0459835529327393, 0.979, 0.979, 0.3), ('LinkedIn', 0.10203075408935547, 1.0, 1.0, 0.4666666666666667), ('How to make coffee?', 1.434638261795044, 0.639, 0.639, 0.06521739130434782), ('Ritalin', 0.08860898017883301, 0.639, 0.639, 0.07894736842105263), ('How to make wine at home?', 2.8392598628997803, 0.134, 0.134, 0.04), ('Most expensive city in the world', 3.4662537574768066, 1.0, 1.0, 0.021739130434782608), ('India', 0.9410107135772705, 0.537, 0.537, 0.04081632653061224), ('how to make money fast?', 2.1770012378692627, 0.681, 0.681, 0.09090909090909091), ('Netflix', 0.1485450267791748, 0.766, 0.766, 0.2647058823529412), ('Apple computer', 0.8379561901092529, 1.0, 1.0, 0.07142857142857142), ('The Simpsons', 0.22732758522033691, 1.0, 1.0, 0.02564102564102564), ('World cup', 2.763754367828369, 0.833, 0.833, 0.045454545454545456), ('How to lose weight?', 0.8217732906341553, 0.696, 0.696, 0.09090909090909091), ('Java', 0.16182255744934082, 0.887, 0.887, 0.08695652173913043), ('Air Jordan', 1.1966516971588135, 0.833, 0.833, 0.06896551724137931), ('how to deal with depression?', 0.921539306640625, 0.276, 0.276, 0.061224489795918366), ('How do you make gold', 2.2193241119384766, 0.0, 0.0, 0.0), ('Marijuana', 0.11546134948730469, 0.5, 0.5, 0.0425531914893617), ('How to make hummus', 1.3304378986358643, 0.528, 0.528, 0.06818181818181818), ('Winter', 0.6601948738098145, 0.521, 0.521, 0.10638297872340426), ('Rick and Morty', 0.2563650608062744, 0.865, 0.865, 0.4482758620689655), ('Natural Language processing', 2.214942455291748, 0.925, 0.925, 0.125), ('World Cup 2022', 2.810328245162964, 0.817, 0.817, 0.18518518518518517), ('Dolly the sheep', 0.1973097324371338, 0.887, 0.887, 0.0975609756097561), ('Cigarettes', 0.12864327430725098, 1.0, 1.0, 0.06666666666666667), ('What is the best place to live in?', 5.477948427200317, 0.0, 0.0, 0.0), ('Elon musk', 0.10497736930847168, 0.801, 0.801, 0.21739130434782608), ('How do you breed flowers?', 0.46424078941345215, 0.887, 0.887, 0.08333333333333333)]\n",
            "average recall is 0.11074450522192315\n",
            "average precision is 0.6752666666666668\n",
            "average precision at 40 is 0.6752666666666668\n",
            "average time is 1.3371624151865642\n"
          ]
        }
      ],
      "source": [
        "import requests\n",
        "from time import time\n",
        "\n",
        "url = 'http://35.225.7.222:8080'\n",
        "\n",
        "qs_res = []\n",
        "for q, true_wids in queries.items():\n",
        "  duration, ap = None, None\n",
        "  t_start = time()\n",
        "  try:\n",
        "    res = requests.get(url + '/search', {'query': q}, timeout=35)\n",
        "    duration = time() - t_start\n",
        "    if res.status_code == 200:\n",
        "      pred_wids, _ = zip(*res.json())\n",
        "      apa40 = average_precision_at_40(true_wids, pred_wids)\n",
        "      ap = average_precision(true_wids, pred_wids)\n",
        "      r = recall(true_wids, pred_wids)\n",
        "  except:\n",
        "    pass\n",
        "  print(f\"{q} -> {len(pred_wids)}\")\n",
        "  \n",
        "  qs_res.append((q, duration, apa40, ap, r))\n",
        "\n",
        "print(qs_res)\n",
        "sum_duration = 0\n",
        "sum_apa40 = 0\n",
        "sum_ap = 0\n",
        "sum_r = 0\n",
        "for tup in qs_res:\n",
        "  sum_duration += tup[1]\n",
        "  sum_apa40 += tup[2]\n",
        "  sum_ap += tup[3]\n",
        "  sum_r += tup[4]\n",
        "\n",
        "avg_duration = sum_duration/len(qs_res)\n",
        "avg_apa40 = sum_apa40/len(qs_res)\n",
        "avg_ap = sum_ap/len(qs_res)\n",
        "avg_r = sum_r/len(qs_res)\n",
        "\n",
        "print(f\"average recall is {avg_r}\")\n",
        "print(f\"average precision is {avg_ap}\")\n",
        "print(f\"average precision at 40 is {avg_apa40}\")\n",
        "print(f\"average time is {avg_duration}\")"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "try:\n",
        "  page_rank = requests.post(url + '/get_pagerank', json=[22634890, 23154189, 306742])\n",
        "  print(page_rank.json())\n",
        "  page_view = requests.post(url + '/get_pageview', json=[22634890, 23154189, 306742])\n",
        "  print(page_view.json())\n",
        "except:\n",
        "  print(\"some error\")"
      ],
      "metadata": {
        "id": "FwH6iCESiKHW",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "705c52bf-3504-4bfe-925e-f955f0505310"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[5.485321527478058, 0.8112487603091939, 4.720355599766869]\n",
            "[558, 475, 4641]\n"
          ]
        }
      ]
    }
  ]
}