hyperparameter_tuning.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Hyperparameter Tuning with HyperDrive"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "from azureml.core import Workspace, Experiment\n",
        "from azureml.core.compute import ComputeTarget, AmlCompute\n",
        "from azureml.core.compute_target import ComputeTargetException\n",
        "from azureml.widgets import RunDetails\n",
        "from azureml.train.sklearn import SKLearn\n",
        "from azureml.train.hyperdrive.run import PrimaryMetricGoal\n",
        "from azureml.train.hyperdrive.policy import BanditPolicy\n",
        "from azureml.train.hyperdrive.sampling import RandomParameterSampling\n",
        "from azureml.train.hyperdrive.runconfig import HyperDriveConfig\n",
        "from azureml.train.hyperdrive.parameter_expressions import choice, uniform\n",
        "import os\n",
        "import shutil\n",
        "import joblib"
      ],
      "outputs": [],
      "execution_count": 1,
      "metadata": {
        "gather": {
          "logged": 1620805676532
        }
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Dataset\n",
        "\n",
        "### Source of the data set \n",
        "For this project we are using files from [Kaggle](https://www.kaggle.com/andrewmvd/heart-failure-clinical-data). In this dataset are data on cardiovascular diseases (CVDs). Which are the number one cause of death worldwide, claiming the lives of an estimated 17 million people each year. This represents approximately 31% of all deaths worldwide.\n",
        "Heart failure is one of the common events caused by CVDs. This dataset contains 12 characteristics that can be used to predict mortality from heart failure.\n",
        "In order for people with cardiovascular disease or at high cardiovascular risk (due to the presence of one or more risk factors such as hypertension, diabetes, hyperlipidaemia or established disease) to receive early detection and treatment, these datasets attempt to improve prediction.\n",
        "\n",
        "### Content of the data set\n",
        "The dataset contains 12 features that can be used to predict mortality from heart failure:\n",
        "- age: Age of the patient\n",
        "- amaemia: Decrease of red blood cells or hemoglobin\n",
        "- creatinine_phosphokinase: Level of the CPK enzyme in the blood (mcg/L)\n",
        "- diabetes: If the patient has diabetes\n",
        "- ejection_fraction: Percentage of blood leaving the heart at each contraction\n",
        "- high_blood_pressure: If the patient has hypertension\n",
        "- platelets: Platelets in the blood (kiloplatelets/mL)\n",
        "- serum_creatinine: Level of serum creatinine in the blood (mg/dL)\n",
        "- serum_sodium: Level of serum sodium in the blood (mEq/L)\n",
        "- sex: Woman or man (Gender at birth)\n",
        "- smoking: patient smokes or not\n",
        "- time: Follow-up period (days)\n",
        "\n",
        "### Target\n",
        "Our goal is to develop a machine learning algorithm that can detect whether a person is likely to die from heart failure. This will help in diagnosis and early prevention. For this, the above mentioned 12 features in the dataset are used to develop a model for the detection.\n",
        "\n",
        "### Attention!\n",
        "This is an experiment that was developed in the course of a test for the Udacity learning platform. Do not use this model in a medical environment or for acute indications. Always consult your doctor for medical questions or the medical emergency service in acute cases!"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "ws = Workspace.from_config()\n",
        "experiment_name = 'hyperdrive_experiment_heart_failure'\n",
        "\n",
        "experiment=Experiment(ws, experiment_name)\n",
        "\n",
        "print('Workspace name: ' + ws.name, \n",
        "      'Azure region: ' + ws.location, \n",
        "      'Subscription id: ' + ws.subscription_id, \n",
        "      'Resource group: ' + ws.resource_group, sep = '\\n')\n",
        "\n",
        "run = experiment.start_logging()"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Workspace name: quick-starts-ws-144621\n",
            "Azure region: southcentralus\n",
            "Subscription id: 610d6e37-4747-4a20-80eb-3aad70a55f43\n",
            "Resource group: aml-quickstarts-144621\n"
          ]
        }
      ],
      "execution_count": 2,
      "metadata": {
        "gather": {
          "logged": 1620805679657
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Dfine compute claster name\n",
        "compute_cluster_name= \"comput-hyper\"\n",
        "\n",
        "#Check if compute cluster already exists\n",
        "\n",
        "try:\n",
        "    # Using exist cluster\n",
        "    compute_cluster=ComputeTarget(workspace=ws, name=compute_cluster_name)\n",
        "    print(\"Found existing cluster, use it\")\n",
        "except ComputeTargetException:\n",
        "    #Create compute cluster\n",
        "    print(\"Creating new cluster\")\n",
        "    compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_DS3_v2',max_nodes=10)\n",
        "    compute_cluster = ComputeTarget.create(ws, compute_cluster_name, compute_config)\n",
        "\n",
        "# Wait for loading\n",
        "compute_cluster.wait_for_completion(show_output=True)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Creating new cluster\n",
            "Creating...\n",
            "SucceededProvisioning operation finished, operation \"Succeeded\"\n",
            "Succeeded\n",
            "AmlCompute wait for completion finished\n",
            "\n",
            "Minimum number of nodes requested have been provisioned\n"
          ]
        }
      ],
      "execution_count": 3,
      "metadata": {
        "gather": {
          "logged": 1620805696246
        }
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Hyperdrive Configuration\n",
        "\n",
        "The model used here is a logistic regression model that is trained with a custom script train.py. \n",
        "The dataset is fetched from [HERE](https://raw.githubusercontent.com/Petopp/Udacity_Final_Project/main/heart_failure_clinical_records_dataset.csv) as a dataset. The hyperparameters chosen for the scikit-learn model are regularisation strength (C) and maximum iterations (max_iter). The trained model is evaluated against 25% data selected from the original dataset. The remaining data is used to train the model.\n",
        "\n",
        "Hyperparameter tuning with HyperDrive requires several steps: \n",
        "- define the parameter search space\n",
        "- define a sampling method\n",
        "- selecting a primary metric for optimisation \n",
        "- selecting an early stop policy.\n",
        "\n",
        "The parameter sampling method used for this project is Random Sampling. It randomly selects the best hyperparameters for the model so that the entire search space does not need to be searched. The Random Sampling method saves time and is much faster than Grid Sampling and Bayesian Sampling, which are only recommended if you have a budget to explore the entire search space.\n",
        "\n",
        "The early stop policy used in this project is the Bandit policy, which is based on a slack factor (in this case 0.1) and a scoring interval (in this case 1). This policy stops runs where the primary metric is not within the specified slip factor, compared to the run with the best performance. This will save time and resources as runs that may not produce good results would be terminated early.\n"
      ],
      "metadata": {
        "collapsed": true,
        "gather": {
          "logged": 1598531923519
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create an early termination policy.\n",
        "early_termination_policy = BanditPolicy(\n",
        "    evaluation_interval=1,\n",
        "    slack_factor= 0.1\n",
        ")\n",
        "\n",
        "# Create the different params that will be needed during training\n",
        "param_sampling = RandomParameterSampling(\n",
        "    {\n",
        "        \"--C\": uniform(0.001,100),\n",
        "        \"--max_iter\": choice(50, 90, 125, 170)\n",
        "    }\n",
        ")\n",
        "\n",
        "if \"training\" not in os.listdir():\n",
        "    os.mkdir(\"./training\")\n",
        "    \n",
        "script_folder = './training'\n",
        "os.makedirs(script_folder, exist_ok=True)\n",
        "\n",
        "shutil.copy('./train.py', script_folder)\n",
        "\n",
        "# Create a SKLearn estimator for use with train.py\n",
        "estimator = SKLearn(\n",
        "    source_directory= script_folder,\n",
        "    compute_target= compute_cluster,\n",
        "    entry_script= \"train.py\",\n",
        "    vm_size=\"Standard_D2_V2\",\n",
        "    vm_priority=\"lowpriority\"\n",
        ")\n",
        "\n",
        "# Create a HyperDriveConfig using the estimator, hyperparameter sampler, and policy.\n",
        "hyperdrive_run_config = HyperDriveConfig(\n",
        "    estimator=estimator,\n",
        "    hyperparameter_sampling= param_sampling,\n",
        "    policy= early_termination_policy,\n",
        "    primary_metric_name= \"Accuracy\",\n",
        "    primary_metric_goal= PrimaryMetricGoal.MAXIMIZE,\n",
        "    max_total_runs=5,\n",
        "    max_concurrent_runs=5\n",
        ")"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "'SKLearn' estimator is deprecated. Please use 'ScriptRunConfig' from 'azureml.core.script_run_config' with your own defined environment or the AzureML-Tutorial curated environment.\n",
            "'enabled' is deprecated. Please use the azureml.core.runconfig.DockerConfiguration object with the 'use_docker' param instead.\n"
          ]
        }
      ],
      "execution_count": 4,
      "metadata": {
        "gather": {
          "logged": 1620805696433
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Submit the experiment\n",
        "hyperdrive_run=experiment.submit(config=hyperdrive_run_config)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:root:If 'script' has been provided here and a script file name has been specified in 'run_config', 'script' provided in ScriptRunConfig initialization will take precedence.\n"
          ]
        }
      ],
      "execution_count": 5,
      "metadata": {
        "gather": {
          "logged": 1620805699221
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Run Details"
      ],
      "metadata": {
        "collapsed": true,
        "gather": {
          "logged": 1598544898497
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "RunDetails(hyperdrive_run).show()"
      ],
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "_HyperDriveWidget(widget_settings={'childWidgetDisplay': 'popup', 'send_telemetry': False, 'log_level': 'INFO'…",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d0c684f8b1eb42bd942ed0ec65ce18d6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "application/aml.mini.widget.v1": "{\"status\": \"Completed\", \"workbench_run_details_uri\": \"https://ml.azure.com/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76?wsid=/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourcegroups/aml-quickstarts-144621/workspaces/quick-starts-ws-144621&tid=660b3398-b80e-49d2-bc5b-ac1dc93b5254\", \"run_id\": \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76\", \"run_properties\": {\"run_id\": \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76\", \"created_utc\": \"2021-05-12T07:48:17.229838Z\", \"properties\": {\"primary_metric_config\": \"{\\\"name\\\": \\\"Accuracy\\\", \\\"goal\\\": \\\"maximize\\\"}\", \"resume_from\": \"null\", \"runTemplate\": \"HyperDrive\", \"azureml.runsource\": \"hyperdrive\", \"platform\": \"AML\", \"ContentSnapshotId\": \"97423971-e158-463d-abd2-88868329b3ba\", \"score\": \"0.84\", \"best_child_run_id\": \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_3\", \"best_metric_status\": \"Succeeded\"}, \"tags\": {\"_aml_system_max_concurrent_jobs\": \"5\", \"max_concurrent_jobs\": \"5\", \"_aml_system_max_total_jobs\": \"5\", \"max_total_jobs\": \"5\", \"_aml_system_max_duration_minutes\": \"10080\", \"max_duration_minutes\": \"10080\", \"_aml_system_policy_config\": \"{\\\"name\\\": \\\"BANDIT\\\", \\\"properties\\\": {\\\"evaluation_interval\\\": 1, \\\"delay_evaluation\\\": 0, \\\"slack_factor\\\": 0.1}}\", \"policy_config\": \"{\\\"name\\\": \\\"BANDIT\\\", \\\"properties\\\": {\\\"evaluation_interval\\\": 1, \\\"delay_evaluation\\\": 0, \\\"slack_factor\\\": 0.1}}\", \"_aml_system_generator_config\": \"{\\\"name\\\": \\\"RANDOM\\\", \\\"parameter_space\\\": {\\\"--C\\\": [\\\"uniform\\\", [0.001, 100]], \\\"--max_iter\\\": [\\\"choice\\\", [[50, 90, 125, 170]]]}}\", \"generator_config\": \"{\\\"name\\\": \\\"RANDOM\\\", \\\"parameter_space\\\": {\\\"--C\\\": [\\\"uniform\\\", [0.001, 100]], \\\"--max_iter\\\": [\\\"choice\\\", [[50, 90, 125, 170]]]}}\", \"_aml_system_primary_metric_config\": \"{\\\"name\\\": \\\"Accuracy\\\", \\\"goal\\\": \\\"maximize\\\"}\", \"primary_metric_config\": \"{\\\"name\\\": \\\"Accuracy\\\", \\\"goal\\\": \\\"maximize\\\"}\", \"_aml_system_platform_config\": \"{\\\"ServiceAddress\\\": \\\"https://southcentralus.experiments.azureml.net\\\", \\\"ServiceArmScope\\\": \\\"subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/hyperdrive_experiment_heart_failure\\\", \\\"SubscriptionId\\\": \\\"610d6e37-4747-4a20-80eb-3aad70a55f43\\\", \\\"ResourceGroupName\\\": \\\"aml-quickstarts-144621\\\", \\\"WorkspaceName\\\": \\\"quick-starts-ws-144621\\\", \\\"ExperimentName\\\": \\\"hyperdrive_experiment_heart_failure\\\", \\\"Definition\\\": {\\\"Overrides\\\": {\\\"script\\\": \\\"train.py\\\", \\\"arguments\\\": [], \\\"target\\\": \\\"comput-hyper\\\", \\\"framework\\\": \\\"Python\\\", \\\"communicator\\\": \\\"None\\\", \\\"maxRunDurationSeconds\\\": null, \\\"nodeCount\\\": 1, \\\"priority\\\": null, \\\"environment\\\": {\\\"name\\\": null, \\\"version\\\": null, \\\"environmentVariables\\\": {\\\"EXAMPLE_ENV_VAR\\\": \\\"EXAMPLE_VALUE\\\"}, \\\"python\\\": {\\\"userManagedDependencies\\\": true, \\\"interpreterPath\\\": \\\"python\\\", \\\"condaDependenciesFile\\\": null, \\\"baseCondaEnvironment\\\": null, \\\"condaDependencies\\\": {\\\"name\\\": \\\"project_environment\\\", \\\"dependencies\\\": [\\\"python=3.6.2\\\", {\\\"pip\\\": [\\\"azureml-defaults\\\"]}], \\\"channels\\\": [\\\"anaconda\\\", \\\"conda-forge\\\"]}}, \\\"docker\\\": {\\\"enabled\\\": true, \\\"baseImage\\\": \\\"sklearn:0.20.3-cpu\\\", \\\"baseDockerfile\\\": null, \\\"sharedVolumes\\\": true, \\\"shmSize\\\": \\\"2g\\\", \\\"arguments\\\": [], \\\"baseImageRegistry\\\": {\\\"address\\\": \\\"viennaprivate.azurecr.io\\\", \\\"username\\\": null, \\\"password\\\": null, \\\"registryIdentity\\\": null}, \\\"platform\\\": {\\\"os\\\": \\\"Linux\\\", \\\"architecture\\\": \\\"amd64\\\"}}, \\\"spark\\\": {\\\"repositories\\\": [], \\\"packages\\\": [], \\\"precachePackages\\\": false}, \\\"databricks\\\": {\\\"mavenLibraries\\\": [], \\\"pypiLibraries\\\": [], \\\"rcranLibraries\\\": [], \\\"jarLibraries\\\": [], \\\"eggLibraries\\\": []}, \\\"r\\\": null, \\\"inferencingStackVersion\\\": null}, \\\"history\\\": {\\\"outputCollection\\\": true, \\\"snapshotProject\\\": true, \\\"directoriesToWatch\\\": [\\\"logs\\\"]}, \\\"spark\\\": {\\\"configuration\\\": {\\\"spark.app.name\\\": \\\"Azure ML Experiment\\\", \\\"spark.yarn.maxAppAttempts\\\": 1}}, \\\"docker\\\": {\\\"useDocker\\\": false, \\\"sharedVolumes\\\": true, \\\"arguments\\\": [], \\\"shmSize\\\": \\\"2g\\\"}, \\\"hdi\\\": {\\\"yarnDeployMode\\\": \\\"cluster\\\"}, \\\"tensorflow\\\": {\\\"workerCount\\\": 1, \\\"parameterServerCount\\\": 1}, \\\"mpi\\\": {\\\"processCountPerNode\\\": 1, \\\"nodeCount\\\": 1}, \\\"pytorch\\\": {\\\"communicationBackend\\\": \\\"nccl\\\", \\\"processCount\\\": null, \\\"nodeCount\\\": 1}, \\\"paralleltask\\\": {\\\"maxRetriesPerWorker\\\": 0, \\\"workerCountPerNode\\\": 1, \\\"terminalExitCodes\\\": null}, \\\"dataReferences\\\": {}, \\\"data\\\": {}, \\\"outputData\\\": {}, \\\"sourceDirectoryDataStore\\\": null, \\\"amlcompute\\\": {\\\"vmSize\\\": \\\"Standard_D2_V2\\\", \\\"vmPriority\\\": \\\"lowpriority\\\", \\\"retainCluster\\\": false, \\\"name\\\": null, \\\"clusterMaxNodeCount\\\": 1}, \\\"credentialPassthrough\\\": false, \\\"command\\\": \\\"\\\"}, \\\"TargetDetails\\\": null, \\\"SnapshotId\\\": \\\"97423971-e158-463d-abd2-88868329b3ba\\\", \\\"TelemetryValues\\\": {\\\"amlClientType\\\": \\\"azureml-sdk-train\\\", \\\"amlClientModule\\\": \\\"[Scrubbed]\\\", \\\"amlClientFunction\\\": \\\"[Scrubbed]\\\", \\\"tenantId\\\": \\\"660b3398-b80e-49d2-bc5b-ac1dc93b5254\\\", \\\"amlClientRequestId\\\": \\\"8578dfdc-9bfc-424a-a561-7b8671f6a171\\\", \\\"amlClientSessionId\\\": \\\"f272226c-654a-424c-a43b-2216104e8718\\\", \\\"subscriptionId\\\": \\\"610d6e37-4747-4a20-80eb-3aad70a55f43\\\", \\\"estimator\\\": \\\"SKLearn\\\", \\\"samplingMethod\\\": \\\"RANDOM\\\", \\\"terminationPolicy\\\": \\\"Bandit\\\", \\\"primaryMetricGoal\\\": \\\"maximize\\\", \\\"maxTotalRuns\\\": 5, \\\"maxConcurrentRuns\\\": 5, \\\"maxDurationMinutes\\\": 10080, \\\"vmSize\\\": \\\"Standard_D2_V2\\\"}}}\", \"platform_config\": \"{\\\"ServiceAddress\\\": \\\"https://southcentralus.experiments.azureml.net\\\", \\\"ServiceArmScope\\\": \\\"subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/hyperdrive_experiment_heart_failure\\\", \\\"SubscriptionId\\\": \\\"610d6e37-4747-4a20-80eb-3aad70a55f43\\\", \\\"ResourceGroupName\\\": \\\"aml-quickstarts-144621\\\", \\\"WorkspaceName\\\": \\\"quick-starts-ws-144621\\\", \\\"ExperimentName\\\": \\\"hyperdrive_experiment_heart_failure\\\", \\\"Definition\\\": {\\\"Overrides\\\": {\\\"script\\\": \\\"train.py\\\", \\\"arguments\\\": [], \\\"target\\\": \\\"comput-hyper\\\", \\\"framework\\\": \\\"Python\\\", \\\"communicator\\\": \\\"None\\\", \\\"maxRunDurationSeconds\\\": null, \\\"nodeCount\\\": 1, \\\"priority\\\": null, \\\"environment\\\": {\\\"name\\\": null, \\\"version\\\": null, \\\"environmentVariables\\\": {\\\"EXAMPLE_ENV_VAR\\\": \\\"EXAMPLE_VALUE\\\"}, \\\"python\\\": {\\\"userManagedDependencies\\\": true, \\\"interpreterPath\\\": \\\"python\\\", \\\"condaDependenciesFile\\\": null, \\\"baseCondaEnvironment\\\": null, \\\"condaDependencies\\\": {\\\"name\\\": \\\"project_environment\\\", \\\"dependencies\\\": [\\\"python=3.6.2\\\", {\\\"pip\\\": [\\\"azureml-defaults\\\"]}], \\\"channels\\\": [\\\"anaconda\\\", \\\"conda-forge\\\"]}}, \\\"docker\\\": {\\\"enabled\\\": true, \\\"baseImage\\\": \\\"sklearn:0.20.3-cpu\\\", \\\"baseDockerfile\\\": null, \\\"sharedVolumes\\\": true, \\\"shmSize\\\": \\\"2g\\\", \\\"arguments\\\": [], \\\"baseImageRegistry\\\": {\\\"address\\\": \\\"viennaprivate.azurecr.io\\\", \\\"username\\\": null, \\\"password\\\": null, \\\"registryIdentity\\\": null}, \\\"platform\\\": {\\\"os\\\": \\\"Linux\\\", \\\"architecture\\\": \\\"amd64\\\"}}, \\\"spark\\\": {\\\"repositories\\\": [], \\\"packages\\\": [], \\\"precachePackages\\\": false}, \\\"databricks\\\": {\\\"mavenLibraries\\\": [], \\\"pypiLibraries\\\": [], \\\"rcranLibraries\\\": [], \\\"jarLibraries\\\": [], \\\"eggLibraries\\\": []}, \\\"r\\\": null, \\\"inferencingStackVersion\\\": null}, \\\"history\\\": {\\\"outputCollection\\\": true, \\\"snapshotProject\\\": true, \\\"directoriesToWatch\\\": [\\\"logs\\\"]}, \\\"spark\\\": {\\\"configuration\\\": {\\\"spark.app.name\\\": \\\"Azure ML Experiment\\\", \\\"spark.yarn.maxAppAttempts\\\": 1}}, \\\"docker\\\": {\\\"useDocker\\\": false, \\\"sharedVolumes\\\": true, \\\"arguments\\\": [], \\\"shmSize\\\": \\\"2g\\\"}, \\\"hdi\\\": {\\\"yarnDeployMode\\\": \\\"cluster\\\"}, \\\"tensorflow\\\": {\\\"workerCount\\\": 1, \\\"parameterServerCount\\\": 1}, \\\"mpi\\\": {\\\"processCountPerNode\\\": 1, \\\"nodeCount\\\": 1}, \\\"pytorch\\\": {\\\"communicationBackend\\\": \\\"nccl\\\", \\\"processCount\\\": null, \\\"nodeCount\\\": 1}, \\\"paralleltask\\\": {\\\"maxRetriesPerWorker\\\": 0, \\\"workerCountPerNode\\\": 1, \\\"terminalExitCodes\\\": null}, \\\"dataReferences\\\": {}, \\\"data\\\": {}, \\\"outputData\\\": {}, \\\"sourceDirectoryDataStore\\\": null, \\\"amlcompute\\\": {\\\"vmSize\\\": \\\"Standard_D2_V2\\\", \\\"vmPriority\\\": \\\"lowpriority\\\", \\\"retainCluster\\\": false, \\\"name\\\": null, \\\"clusterMaxNodeCount\\\": 1}, \\\"credentialPassthrough\\\": false, \\\"command\\\": \\\"\\\"}, \\\"TargetDetails\\\": null, \\\"SnapshotId\\\": \\\"97423971-e158-463d-abd2-88868329b3ba\\\", \\\"TelemetryValues\\\": {\\\"amlClientType\\\": \\\"azureml-sdk-train\\\", \\\"amlClientModule\\\": \\\"[Scrubbed]\\\", \\\"amlClientFunction\\\": \\\"[Scrubbed]\\\", \\\"tenantId\\\": \\\"660b3398-b80e-49d2-bc5b-ac1dc93b5254\\\", \\\"amlClientRequestId\\\": \\\"8578dfdc-9bfc-424a-a561-7b8671f6a171\\\", \\\"amlClientSessionId\\\": \\\"f272226c-654a-424c-a43b-2216104e8718\\\", \\\"subscriptionId\\\": \\\"610d6e37-4747-4a20-80eb-3aad70a55f43\\\", \\\"estimator\\\": \\\"SKLearn\\\", \\\"samplingMethod\\\": \\\"RANDOM\\\", \\\"terminationPolicy\\\": \\\"Bandit\\\", \\\"primaryMetricGoal\\\": \\\"maximize\\\", \\\"maxTotalRuns\\\": 5, \\\"maxConcurrentRuns\\\": 5, \\\"maxDurationMinutes\\\": 10080, \\\"vmSize\\\": \\\"Standard_D2_V2\\\"}}}\", \"_aml_system_resume_child_runs\": \"null\", \"resume_child_runs\": \"null\", \"_aml_system_all_jobs_generated\": \"true\", \"all_jobs_generated\": \"true\", \"_aml_system_cancellation_requested\": \"false\", \"cancellation_requested\": \"false\", \"_aml_system_progress_metadata_evaluation_timestamp\": \"\\\"2021-05-12T07:48:17.958020\\\"\", \"progress_metadata_evaluation_timestamp\": \"\\\"2021-05-12T07:48:17.958020\\\"\", \"_aml_system_progress_metadata_digest\": \"\\\"4160b7b97e8c0b210cc6d5764eb8ab771c9a84eaed3a7791a3954e527afe66d3\\\"\", \"progress_metadata_digest\": \"\\\"4160b7b97e8c0b210cc6d5764eb8ab771c9a84eaed3a7791a3954e527afe66d3\\\"\", \"_aml_system_progress_metadata_active_timestamp\": \"\\\"2021-05-12T07:48:17.958020\\\"\", \"progress_metadata_active_timestamp\": \"\\\"2021-05-12T07:48:17.958020\\\"\", \"_aml_system_optimizer_state_artifact\": \"null\", \"_aml_system_outdated_optimizer_state_artifacts\": \"\\\"[]\\\"\", \"_aml_system_HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_0\": \"{\\\"--C\\\": 37.25840360760336, \\\"--max_iter\\\": 90}\", \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_0\": \"{\\\"--C\\\": 37.25840360760336, \\\"--max_iter\\\": 90}\", \"_aml_system_HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_1\": \"{\\\"--C\\\": 89.2474155729707, \\\"--max_iter\\\": 125}\", \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_1\": \"{\\\"--C\\\": 89.2474155729707, \\\"--max_iter\\\": 125}\", \"_aml_system_HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_2\": \"{\\\"--C\\\": 59.40375278971442, \\\"--max_iter\\\": 50}\", \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_2\": \"{\\\"--C\\\": 59.40375278971442, \\\"--max_iter\\\": 50}\", \"_aml_system_HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_3\": \"{\\\"--C\\\": 97.2861169940756, \\\"--max_iter\\\": 125}\", \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_3\": \"{\\\"--C\\\": 97.2861169940756, \\\"--max_iter\\\": 125}\", \"_aml_system_HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_4\": \"{\\\"--C\\\": 75.26162920486995, \\\"--max_iter\\\": 90}\", \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_4\": \"{\\\"--C\\\": 75.26162920486995, \\\"--max_iter\\\": 90}\", \"_aml_system_final_best_metric_update_retry_count\": \"1\", \"final_best_metric_update_retry_count\": \"1\"}, \"end_time_utc\": \"2021-05-12T07:53:49.636105Z\", \"status\": \"Completed\", \"log_files\": {\"azureml-logs/hyperdrive.txt\": \"https://mlstrg144621.blob.core.windows.net/azureml/ExperimentRun/dcid.HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76/azureml-logs/hyperdrive.txt?sv=2019-02-02&sr=b&sig=%2FnNOsh%2FYlpeMfrViBN1pWgeXAH76a0uT9gF%2BiUgG%2BdU%3D&st=2021-05-12T07%3A43%3A51Z&se=2021-05-12T15%3A53%3A51Z&sp=r\"}, \"log_groups\": [[\"azureml-logs/hyperdrive.txt\"]], \"run_duration\": \"0:05:32\", \"run_number\": \"2\", \"run_queued_details\": {\"status\": \"Completed\", \"details\": null}, \"hyper_parameters\": {\"--C\": [\"uniform\", [0.001, 100]], \"--max_iter\": [\"choice\", [[50, 90, 125, 170]]]}}, \"child_runs\": [{\"run_id\": \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_4\", \"run_number\": 3, \"metric\": 0.8, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2021-05-12T07:52:03.572206Z\", \"end_time\": \"2021-05-12T07:52:58.038177Z\", \"created_time\": \"2021-05-12T07:48:48.659913Z\", \"created_time_dt\": \"2021-05-12T07:48:48.659913Z\", \"duration\": \"0:04:09\", \"hyperdrive_id\": \"20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76\", \"arguments\": null, \"param_--C\": 75.26162920486995, \"param_--max_iter\": 90, \"best_metric\": 0.8}, {\"run_id\": \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_0\", \"run_number\": 4, \"metric\": 0.81333333, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2021-05-12T07:52:06.202981Z\", \"end_time\": \"2021-05-12T07:53:07.096218Z\", \"created_time\": \"2021-05-12T07:48:48.730504Z\", \"created_time_dt\": \"2021-05-12T07:48:48.730504Z\", \"duration\": \"0:04:18\", \"hyperdrive_id\": \"20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76\", \"arguments\": null, \"param_--C\": 37.25840360760336, \"param_--max_iter\": 90, \"best_metric\": 0.81333333}, {\"run_id\": \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_3\", \"run_number\": 5, \"metric\": 0.84, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2021-05-12T07:52:04.434574Z\", \"end_time\": \"2021-05-12T07:53:02.925809Z\", \"created_time\": \"2021-05-12T07:48:48.939496Z\", \"created_time_dt\": \"2021-05-12T07:48:48.939496Z\", \"duration\": \"0:04:13\", \"hyperdrive_id\": \"20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76\", \"arguments\": null, \"param_--C\": 97.2861169940756, \"param_--max_iter\": 125, \"best_metric\": 0.84}, {\"run_id\": \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_1\", \"run_number\": 6, \"metric\": 0.81333333, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2021-05-12T07:52:17.15264Z\", \"end_time\": \"2021-05-12T07:53:15.570325Z\", \"created_time\": \"2021-05-12T07:48:49.01914Z\", \"created_time_dt\": \"2021-05-12T07:48:49.01914Z\", \"duration\": \"0:04:26\", \"hyperdrive_id\": \"20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76\", \"arguments\": null, \"param_--C\": 89.2474155729707, \"param_--max_iter\": 125, \"best_metric\": 0.84}, {\"run_id\": \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_2\", \"run_number\": 7, \"metric\": 0.81333333, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2021-05-12T07:52:06.500022Z\", \"end_time\": \"2021-05-12T07:53:10.102348Z\", \"created_time\": \"2021-05-12T07:48:49.202064Z\", \"created_time_dt\": \"2021-05-12T07:48:49.202064Z\", \"duration\": \"0:04:20\", \"hyperdrive_id\": \"20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76\", \"arguments\": null, \"param_--C\": 59.40375278971442, \"param_--max_iter\": 50, \"best_metric\": 0.84}], \"children_metrics\": {\"categories\": [0], \"series\": {\"Regularization Strength:\": [{\"categories\": [3, 4, 5, 6, 7], \"mode\": \"markers\", \"name\": \"Regularization Strength:\", \"stepped\": false, \"type\": \"scatter\", \"data\": [75.26162920486995, 37.25840360760336, 97.2861169940756, 89.2474155729707, 59.40375278971442]}, {\"categories\": [3, 4, 5, 6, 7], \"mode\": \"lines\", \"name\": \"Regularization Strength:_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [75.26162920486995, 75.26162920486995, 97.2861169940756, 97.2861169940756, 97.2861169940756]}], \"Max iterations:\": [{\"categories\": [3, 4, 5, 6, 7], \"mode\": \"markers\", \"name\": \"Max iterations:\", \"stepped\": false, \"type\": \"scatter\", \"data\": [90, 90, 125, 125, 50]}, {\"categories\": [3, 4, 5, 6, 7], \"mode\": \"lines\", \"name\": \"Max iterations:_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [90, 90, 125, 125, 125]}], \"Accuracy\": [{\"categories\": [3, 4, 5, 6, 7], \"mode\": \"markers\", \"name\": \"Accuracy\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.8, 0.8133333333333334, 0.84, 0.8133333333333334, 0.8133333333333334]}, {\"categories\": [3, 4, 5, 6, 7], \"mode\": \"lines\", \"name\": \"Accuracy_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.8, 0.8133333333333334, 0.84, 0.84, 0.84]}]}, \"metricName\": null, \"primaryMetricName\": \"Accuracy\", \"showLegend\": false}, \"run_metrics\": [{\"name\": \"best_child_by_primary_metric\", \"run_id\": \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76\", \"categories\": [0], \"series\": [{\"data\": [{\"metric_name\": [\"Accuracy\", \"Accuracy\"], \"timestamp\": [\"2021-05-12 07:53:17.806370+00:00\", \"2021-05-12 07:53:48.592196+00:00\"], \"run_id\": [\"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_4\", \"HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_3\"], \"metric_value\": [0.8, 0.84], \"final\": [false, true]}]}]}], \"run_logs\": \"[2021-05-12T07:48:18.051691][GENERATOR][INFO]Trying to sample '5' jobs from the hyperparameter space\\r\\n[2021-05-12T07:48:18.256019][GENERATOR][INFO]Successfully sampled '5' jobs, they will soon be submitted to the execution target.\\r\\n[2021-05-12T07:48:17.619740][API][INFO]Experiment created\\r\\n[2021-05-12T07:48:47.453887][GENERATOR][INFO]Max number of jobs '5' reached for experiment.\\r\\n[2021-05-12T07:48:47.592346][GENERATOR][INFO]All jobs generated.\\r\\n[2021-05-12T07:48:47.9484149Z][SCHEDULER][INFO]Scheduling job, id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_0'\\r\\n[2021-05-12T07:48:47.9524926Z][SCHEDULER][INFO]Scheduling job, id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_1'\\r\\n[2021-05-12T07:48:47.9898295Z][SCHEDULER][INFO]Scheduling job, id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_3'\\r\\n[2021-05-12T07:48:47.9608310Z][SCHEDULER][INFO]Scheduling job, id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_2'\\r\\n[2021-05-12T07:48:48.7710174Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_4'\\r\\n[2021-05-12T07:48:47.9777512Z][SCHEDULER][INFO]Scheduling job, id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_4'\\r\\n[2021-05-12T07:48:48.8321492Z][SCHEDULER][INFO]Successfully patched a child run. Id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_4'\\r\\n[2021-05-12T07:48:48.8445851Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_0'\\r\\n[2021-05-12T07:48:48.9020494Z][SCHEDULER][INFO]Successfully patched a child run. Id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_0'\\r\\n[2021-05-12T07:48:49.0506061Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_3'\\r\\n[2021-05-12T07:48:49.1391407Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_1'\\r\\n[2021-05-12T07:48:49.1405184Z][SCHEDULER][INFO]Successfully patched a child run. Id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_3'\\r\\n[2021-05-12T07:48:49.2077024Z][SCHEDULER][INFO]Successfully patched a child run. Id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_1'\\r\\n[2021-05-12T07:48:49.3166034Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_2'\\r\\n[2021-05-12T07:48:49.3919036Z][SCHEDULER][INFO]Successfully patched a child run. Id='HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_2'\\r\\n[2021-05-12T07:52:17.583467][ENFORCER][INFO]Jobs [https://southcentralus.experiments.azureml.net/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/**SCRUBBED**/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_0, https://southcentralus.experiments.azureml.net/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/**SCRUBBED**/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_1, https://southcentralus.experiments.azureml.net/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/**SCRUBBED**/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_2, https://southcentralus.experiments.azureml.net/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/**SCRUBBED**/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_3, https://southcentralus.experiments.azureml.net/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/**SCRUBBED**/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_4] do not contain any metrics with the primary metric name at this moment, policy cannot be applied.\\r\\n[2021-05-12T07:52:47.647521][ENFORCER][INFO]Jobs [https://southcentralus.experiments.azureml.net/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/**SCRUBBED**/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_0, https://southcentralus.experiments.azureml.net/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/**SCRUBBED**/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_1, https://southcentralus.experiments.azureml.net/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/**SCRUBBED**/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_2, https://southcentralus.experiments.azureml.net/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/**SCRUBBED**/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_3, https://southcentralus.experiments.azureml.net/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourceGroups/aml-quickstarts-144621/providers/Microsoft.MachineLearningServices/workspaces/quick-starts-ws-144621/experiments/**SCRUBBED**/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_4] do not contain any metrics with the primary metric name at this moment, policy cannot be applied.\\r\\n[2021-05-12T07:53:49.828449][CONTROLLER][INFO]Experiment was 'ExperimentStatus.RUNNING', is 'ExperimentStatus.FINISHED'.\\n\\nRun is completed.\", \"graph\": {}, \"widget_settings\": {\"childWidgetDisplay\": \"popup\", \"send_telemetry\": false, \"log_level\": \"INFO\", \"sdk_version\": \"1.27.0\"}, \"loading\": false}"
          },
          "metadata": {}
        }
      ],
      "execution_count": 6,
      "metadata": {
        "gather": {
          "logged": 1620805699453
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "hyperdrive_run.wait_for_completion(show_output= True)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "RunId: HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76\n",
            "Web View: https://ml.azure.com/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76?wsid=/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourcegroups/aml-quickstarts-144621/workspaces/quick-starts-ws-144621&tid=660b3398-b80e-49d2-bc5b-ac1dc93b5254\n",
            "\n",
            "Streaming azureml-logs/hyperdrive.txt\n",
            "=====================================\n",
            "\n",
            "\"<START>[2021-05-12T07:48:18.051691][GENERATOR][INFO]Trying to sample '5' jobs from the hyperparameter space<END>\\n\"\"<START>[2021-05-12T07:48:18.256019][GENERATOR][INFO]Successfully sampled '5' jobs, they will soon be submitted to the execution target.<END>\\n\"\"<START>[2021-05-12T07:48:17.619740][API][INFO]Experiment created<END>\\n\"\n",
            "\n",
            "Execution Summary\n",
            "=================\n",
            "RunId: HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76\n",
            "Web View: https://ml.azure.com/runs/HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76?wsid=/subscriptions/610d6e37-4747-4a20-80eb-3aad70a55f43/resourcegroups/aml-quickstarts-144621/workspaces/quick-starts-ws-144621&tid=660b3398-b80e-49d2-bc5b-ac1dc93b5254\n",
            "\n"
          ]
        },
        {
          "output_type": "execute_result",
          "execution_count": 7,
          "data": {
            "text/plain": "{'runId': 'HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76',\n 'target': 'comput-hyper',\n 'status': 'Completed',\n 'startTimeUtc': '2021-05-12T07:48:17.325863Z',\n 'endTimeUtc': '2021-05-12T07:53:49.636105Z',\n 'properties': {'primary_metric_config': '{\"name\": \"Accuracy\", \"goal\": \"maximize\"}',\n  'resume_from': 'null',\n  'runTemplate': 'HyperDrive',\n  'azureml.runsource': 'hyperdrive',\n  'platform': 'AML',\n  'ContentSnapshotId': '97423971-e158-463d-abd2-88868329b3ba',\n  'score': '0.84',\n  'best_child_run_id': 'HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76_3',\n  'best_metric_status': 'Succeeded'},\n 'inputDatasets': [],\n 'outputDatasets': [],\n 'logFiles': {'azureml-logs/hyperdrive.txt': 'https://mlstrg144621.blob.core.windows.net/azureml/ExperimentRun/dcid.HD_20b1fe66-bbfd-42e9-a6c0-d9bbb6ceef76/azureml-logs/hyperdrive.txt?sv=2019-02-02&sr=b&sig=%2FnNOsh%2FYlpeMfrViBN1pWgeXAH76a0uT9gF%2BiUgG%2BdU%3D&st=2021-05-12T07%3A43%3A51Z&se=2021-05-12T15%3A53%3A51Z&sp=r'},\n 'submittedBy': 'ODL_User 144621'}"
          },
          "metadata": {}
        }
      ],
      "execution_count": 7,
      "metadata": {
        "gather": {
          "logged": 1620806040931
        }
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Best Model"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "# Get best model and diplay all details\n",
        "best_run= hyperdrive_run.get_best_run_by_primary_metric()\n",
        "best_run_metrics=best_run.get_metrics()\n",
        "print(best_run.get_details()['runDefinition']['arguments'])\n",
        "print(best_run.get_file_names())\n",
        "print('Best Run Accuracy:',best_run_metrics['Accuracy'])"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['--C', '97.2861169940756', '--max_iter', '125']\n",
            "['azureml-logs/55_azureml-execution-tvmps_b3d8a370fdab6acc496b1fa398220948b9ae8dd605d8df21bbd0582f1cc744bc_d.txt', 'azureml-logs/65_job_prep-tvmps_b3d8a370fdab6acc496b1fa398220948b9ae8dd605d8df21bbd0582f1cc744bc_d.txt', 'azureml-logs/70_driver_log.txt', 'azureml-logs/75_job_post-tvmps_b3d8a370fdab6acc496b1fa398220948b9ae8dd605d8df21bbd0582f1cc744bc_d.txt', 'azureml-logs/process_info.json', 'azureml-logs/process_status.json', 'logs/azureml/106_azureml.log', 'logs/azureml/job_prep_azureml.log', 'logs/azureml/job_release_azureml.log', 'outputs/model.joblib']\n",
            "Best Run Accuracy: 0.84\n"
          ]
        }
      ],
      "execution_count": 8,
      "metadata": {
        "gather": {
          "logged": 1620806042335
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "best_run.get_file_names()"
      ],
      "outputs": [
        {
          "output_type": "execute_result",
          "execution_count": 9,
          "data": {
            "text/plain": "['azureml-logs/55_azureml-execution-tvmps_b3d8a370fdab6acc496b1fa398220948b9ae8dd605d8df21bbd0582f1cc744bc_d.txt',\n 'azureml-logs/65_job_prep-tvmps_b3d8a370fdab6acc496b1fa398220948b9ae8dd605d8df21bbd0582f1cc744bc_d.txt',\n 'azureml-logs/70_driver_log.txt',\n 'azureml-logs/75_job_post-tvmps_b3d8a370fdab6acc496b1fa398220948b9ae8dd605d8df21bbd0582f1cc744bc_d.txt',\n 'azureml-logs/process_info.json',\n 'azureml-logs/process_status.json',\n 'logs/azureml/106_azureml.log',\n 'logs/azureml/job_prep_azureml.log',\n 'logs/azureml/job_release_azureml.log',\n 'outputs/model.joblib']"
          },
          "metadata": {}
        }
      ],
      "execution_count": 9,
      "metadata": {
        "gather": {
          "logged": 1620806042566
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Save the best model\n",
        "model=best_run.register_model(model_name='heart-failure-sklearn', model_path='outputs/model.joblib')\n",
        "best_run.download_file('/outputs/model.joblib', 'hyperdrive_model.joblib')"
      ],
      "outputs": [],
      "execution_count": 10,
      "metadata": {
        "gather": {
          "logged": 1620806043352
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Cleaning\n",
        "compute_cluster.delete()\n"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Current provisioning state of AmlCompute is \"Deleting\"\n",
            "\n",
            "Current provisioning state of AmlCompute is \"Deleting\"\n",
            "\n",
            "Current provisioning state of AmlCompute is \"Deleting\"\n",
            "\n",
            "Current provisioning state of AmlCompute is \"Deleting\"\n",
            "\n",
            "Current provisioning state of AmlCompute is \"Deleting\"\n",
            "\n",
            "Current provisioning state of AmlCompute is \"Deleting\"\n",
            "\n",
            "Current provisioning state of AmlCompute is \"Deleting\"\n",
            "\n",
            "Current provisioning state of AmlCompute is \"Deleting\"\n",
            "\n",
            "Current provisioning state of AmlCompute is \"Deleting\"\n",
            "\n"
          ]
        }
      ],
      "execution_count": 12,
      "metadata": {
        "gather": {
          "logged": 1620806113317
        }
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "outputs": [],
      "execution_count": null,
      "metadata": {}
    }
  ],
  "metadata": {
    "kernel_info": {
      "name": "python3"
    },
    "kernelspec": {
      "name": "python3",
      "language": "python",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python",
      "version": "3.6.9",
      "mimetype": "text/x-python",
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "pygments_lexer": "ipython3",
      "nbconvert_exporter": "python",
      "file_extension": ".py"
    },
    "microsoft": {
      "host": {
        "AzureML": {
          "notebookHasBeenCompleted": true
        }
      }
    },
    "nteract": {
      "version": "nteract-front-end@1.0.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}