Notebooks/Rekognition_Demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Amazon Rekognition Demo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "Copyright [2017]-[2017] Amazon.com, Inc. or its affiliates. All Rights Reserved.\n",
    "\n",
    "Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at\n",
    "\n",
    "http://aws.amazon.com/apache2.0/\n",
    "\n",
    "or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Prerequisites:**\n",
    "\n",
    "The user or role that executes the commands must have permissions in AWS Identity and Access Management (IAM) to perform those actions. AWS provides a set of managed policies that help you get started quickly. For our example, you need to apply the following minimum managed policies to your user or role:\n",
    "\n",
    "* AmazonRekognitionFullAccess\n",
    "* AmazonDynamoDBFullAccess\n",
    "* AmazonS3FullAccess\n",
    "\n",
    "Be aware that we recommend you follow AWS IAM best practices for production implementations, which is out of scope fof this workshop.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "An error occurred (ResourceInUseException) when calling the CreateTable operation: Table already exists: rekognition_actors\n",
      "mkdir: tmp: File exists\n"
     ]
    }
   ],
   "source": [
    "# initialise Notebook\n",
    "import boto3\n",
    "from IPython.display import HTML, display\n",
    "from PIL import Image, ImageDraw, ImageFont\n",
    "from io import BytesIO\n",
    "import time\n",
    "\n",
    "# key variable initiation\n",
    "rekognition = boto3.client('rekognition', region_name='eu-west-1')\n",
    "dynamodb = boto3.resource('dynamodb', region_name='eu-west-1')\n",
    "s3 = boto3.client('s3')\n",
    "\n",
    "# blue, green, red, grey\n",
    "colors = ((220,220,220),(242,168,73),(76,182,252),(52,194,123))\n",
    "\n",
    "# helper function to lookup names in dynamoDB\n",
    "actors = dynamodb.Table('rekognition_actors')\n",
    "def get_actor_name(actorId):\n",
    "    response = actors.get_item(\n",
    "            Key={\n",
    "                'actorId': actorId\n",
    "            },\n",
    "            AttributesToGet=['actorFullName']                  \n",
    "        )\n",
    "    \n",
    "    if 'Item' in response:\n",
    "        return (response['Item']['actorFullName'])\n",
    "    else:\n",
    "        return ('no match found in actors lookup')\n",
    "    \n",
    "# create Amazon DynamoDB table \n",
    "!aws dynamodb create-table --table-name rekognition_actors \\\n",
    "--attribute-definitions AttributeName=actorId,AttributeType=S \\\n",
    "--key-schema AttributeName=actorId,KeyType=HASH \\\n",
    "--provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1 \\\n",
    "--region eu-west-1   \n",
    "\n",
    "# create temporary directory\n",
    "!mkdir tmp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Object and scene detection\n",
    "\n",
    "This features allows you to detect thousands of objects, scenes and concepts in your images and provide labels describing the objects detected.\n",
    "\n",
    "This example presents an image to the object and scene detection endpoint to determine the lables for the image. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><tr><td width='50%'><img src='tmp/image?1512402487.196725' ></td><td>                 <table><tr><td>Blossom</td><td>93.37197875976562</td></tr><tr><td>Flora</td><td>93.37197875976562</td></tr><tr><td>Flower</td><td>93.37197875976562</td></tr><tr><td>Lily</td><td>93.37197875976562</td></tr><tr><td>Plant</td><td>93.37197875976562</td></tr><tr><td>Pond Lily</td><td>93.37197875976562</td></tr><tr><td>Soil</td><td>54.79717254638672</td></tr><tr><td>Art</td><td>54.27285385131836</td></tr><tr><td>Flower Arrangement</td><td>54.27285385131836</td></tr><tr><td>Ikebana</td><td>54.27285385131836</td></tr><tr><td>Jar</td><td>54.27285385131836</td></tr><tr><td>Ornament</td><td>54.27285385131836</td></tr><tr><td>Pottery</td><td>54.27285385131836</td></tr><tr><td>Vase</td><td>54.27285385131836</td></tr><tr><td>Crocus</td><td>52.84606170654297</td></tr><tr><td>Anthurium</td><td>50.84963607788086</td></tr><tr><td>Outdoors</td><td>50.73622512817383</td></tr></table></td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "bucket = 'mlrekognitiondemo'\n",
    "key = 'images/P1010138.JPG'\n",
    "#key = 'images/P4050012.JPG'\n",
    "\n",
    "# Download image from Amazon S3 for display\n",
    "image_location = 'tmp/image'\n",
    "s3.download_file(bucket, key, image_location)\n",
    "image_bin=Image.open(image_location)\n",
    "\n",
    "# retrieve labels from Amazon Rekognition service\n",
    "ret = rekognition.detect_labels(\n",
    "    Image={\n",
    "        \"S3Object\": {\n",
    "            \"Bucket\": bucket,\n",
    "            \"Name\" : key,\n",
    "        }\n",
    "    },\n",
    "    #MaxLabels=10,\n",
    "    # MinConfidence=85\n",
    ")\n",
    "\n",
    "inner = \"<table>\"\n",
    "for i in ret['Labels']:\n",
    "    inner += \"<tr><td>\" + i['Name'] + \"</td><td>\" + str(i['Confidence']) + \"</td></tr>\"\n",
    "display(HTML(\"<table><tr><td width='50%'><img src='tmp/image?\"+str(time.time())+\"' ></td><td> \\\n",
    "                \"+inner+\"</table></td></tr></table>\"))  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Facial analysis\n",
    "\n",
    "Using Amazon DetectFaces API, you can detect faces in an image and key facial characteristics.\n",
    "\n",
    "As faces are detected on an image you get a number of attributes about the face:\n",
    "\n",
    "\t- gender\n",
    "\t- age\n",
    "\t- sentiment of the person\n",
    "    - coordinates of key features\n",
    "    - image quality\n",
    "    - etc.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bucket = 'mlrekognitiondemo'\n",
    "key = 'images/Werner_Vogels.jpeg'\n",
    "OUTPUT_BLACKLIST = (\"BoundingBox\",\"Landmarks\",\"Pose\",\"Confidence\")\n",
    "\n",
    "# detect faces using Amazon Rekognition detect faces API\n",
    "ret = rekognition.detect_faces(\n",
    "    Image={\n",
    "        \"S3Object\": {\n",
    "            \"Bucket\": bucket,\n",
    "            \"Name\" : key,\n",
    "        }\n",
    "    },\n",
    "    Attributes=['ALL'],\n",
    ")\n",
    "\n",
    "# Output image and labels\n",
    "\n",
    "image_location = 'tmp/image'\n",
    "s3.download_file(bucket, key, image_location)\n",
    "\n",
    "inner = \"<table>\"\n",
    "for line in ret['FaceDetails']:   \n",
    "    for label in line.keys():\n",
    "        if not label in OUTPUT_BLACKLIST:\n",
    "            if not type(line[label]) is list:\n",
    "                inner += \"<tr><td>\" + label + \"</td><td>\" + str(line[label]) + \"</td></tr>\"\n",
    "            elif type(line[label]) is list:    \n",
    "                inner += \"<tr><td><b>\" + label + \":</b></td><td>\" + '' + \"</td></tr>\"\n",
    "                for entities in line[label]:\n",
    "                    inner += \"<tr><td>\" + '' + \"</td><td>\" + str(entities) + \"</td></tr>\"\n",
    "\n",
    "display(HTML(\"<table><tr><td width='40%'><img src='tmp/image?\"+str(time.time())+\"' ></td><td> \\\n",
    "                \"+inner+\"</table></td></tr></table>\")) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Detect multiple faces within images\n",
    "\n",
    "bucket = 'mlrekognitiondemo'\n",
    "key = 'images/faces.png'\n",
    "\n",
    "# retrieve face boxes from Rekognition service\n",
    "\n",
    "ret = rekognition.detect_faces(\n",
    "    Image={\n",
    "        \"S3Object\": {\n",
    "            \"Bucket\": bucket,\n",
    "            \"Name\" : key,\n",
    "        }\n",
    "    },\n",
    "    Attributes=['DEFAULT'],\n",
    ")\n",
    "\n",
    "boxes = []\n",
    "\n",
    "# Output labels\n",
    "\n",
    "faces = ret['FaceDetails']\n",
    "for face in faces:\n",
    "    boxes.append (face['BoundingBox'])\n",
    "\n",
    "# donwload image from S3\n",
    "image_location ='tmp/image'\n",
    "s3.download_file(bucket, key, image_location)\n",
    "\n",
    "image_out = Image.open(\"tmp/image\")\n",
    "\n",
    "# apply face boxes to image \n",
    "draw = ImageDraw.Draw(image_out)\n",
    "width, height = image_out.size\n",
    "col = 0\n",
    "line= 10\n",
    "inner = \"<table>\"\n",
    "for box in boxes:\n",
    "    inner += \"<tr><td><b>Face \" + str(col+1) + \":<b></td><td>\"\n",
    "    x1 = int(box['Left'] * width)\n",
    "    y1 = int(box['Top'] * height)\n",
    "    x2 = int(box['Left'] * width + box['Width'] * width)\n",
    "    y2 = int(box['Top'] * height + box['Height']  * height)\n",
    "    for label in box.keys():\n",
    "        inner += \"<tr><td>\" + label + \"</td><td>\" + str(box[label]) + \"</td></tr>\"\n",
    "        \n",
    "    for l in range(line):\n",
    "        draw.rectangle((x1-l,y1-l,x2+l,y2+l),outline=colors[col])\n",
    "    col += 1\n",
    "\n",
    "image_out.save('tmp/image',format=\"PNG\")\n",
    "   \n",
    "#display(image_out)\n",
    "\n",
    "display(HTML(\"<table><tr><td width='60%'><img src='tmp/image?\"+str(time.time())+\"' ></td><td> \\\n",
    "                \"+inner+\"</table></td></tr></table>\"))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Face comparison\n",
    "\n",
    "Given two images with faces, Rekognition will compare the largest face from the source image and find similarity with faces found in the target image. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# retrieve and resize images for display\n",
    "\n",
    "bucket = 'mlrekognitiondemo'\n",
    "key1 = 'images/coty.jpg'\n",
    "key2 = 'images/Werner_Vogels.jpeg'\n",
    "\n",
    "s3.download_file(bucket, key1, 'tmp/image1')\n",
    "s3.download_file(bucket, key2, 'tmp/image2')\n",
    "\n",
    "# Compare faces using Rekognition\n",
    "\n",
    "ret = rekognition.compare_faces(\n",
    "    SourceImage={\n",
    "        \"S3Object\": {\n",
    "            \"Bucket\": bucket,\n",
    "            \"Name\" : key2,\n",
    "        }\n",
    "    \n",
    "    },\n",
    "    TargetImage={\n",
    "        \"S3Object\": {\n",
    "            \"Bucket\": bucket,\n",
    "            \"Name\" : key1,\n",
    "        }    \n",
    "    }\n",
    ")\n",
    "\n",
    "# And highlight match on the image\n",
    "\n",
    "image_out = Image.open(\"tmp/image1\")\n",
    "draw = ImageDraw.Draw(image_out)\n",
    "\n",
    "# get image size\n",
    "width, height = image_out.size\n",
    "\n",
    "col = 2\n",
    "line= 5\n",
    "\n",
    "box= ret['FaceMatches'][0]['Face']['BoundingBox']\n",
    "x1 = int(box['Left'] * width)\n",
    "y1 = int(box['Top'] * height)\n",
    "x2 = int(box['Left'] * width + box['Width'] * width)\n",
    "y2 = int(box['Top'] * height + box['Height']  * height)\n",
    "        \n",
    "for l in range(line):\n",
    "        draw.rectangle((x1-l,y1-l,x2+l,y2+l),outline=colors[col])\n",
    "    \n",
    "image_out.save('tmp/image',format=\"PNG\") \n",
    "\n",
    "inner = \"<table>\"\n",
    "inner += \"<tr><td><b>Face match:<b></td><td>\"\n",
    "inner += \"<tr><td>Similarity: </td><td>\" + str(ret['FaceMatches'][0]['Similarity']) + \"</td></tr>\"\n",
    "for label in box.keys():\n",
    "        inner += \"<tr><td>\" + label + \"</td><td>\" + str(box[label]) + \"</td></tr>\"\n",
    "\n",
    "display(HTML(\"<table><tr><td><img src='tmp/image2' height='250' width='250'></td><td><img src='tmp/image' height='250' width='250'></td><td>\"+inner+\"</table></td></tr></table>\"))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Face recognition\n",
    "\n",
    "With FaceRekognition, you can search your image collection for similar faces by storing faces, using the IndexFaces API operation. \n",
    "You can then use the SearchFaces operation to return high-confidence matches. A face collection is an index of faces that you own and manage.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "aws rekognition create-collection --collection-id actors --region eu-west-1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load face definitions into index\n",
    "\n",
    "faces = [\n",
    "{\"objectId\":\"images/z5oj3dys.jpg\",\"actorFullName\":\"Alexandra Maria Lara\"},\n",
    "{\"objectId\":\"images/z6od3gys.jpg\",\"actorFullName\":\"Alexandra Maria Lara\"},\n",
    "{\"objectId\":\"images/z7ad3fys.jpg\",\"actorFullName\":\"Alexandra Maria Lara\"},    \n",
    "{\"objectId\":\"images/z5kj4dss.png\",\"actorFullName\":\"Alexandra Maria Lara\"},   \n",
    "{\"objectId\":\"images/z6hj7fss.jpg\",\"actorFullName\":\"Alexandra Maria Lara\"}, \n",
    "{\"objectId\":\"images/z6hd7hsl.jpg\",\"actorFullName\":\"Alexandra Maria Lara\"},      \n",
    "{\"objectId\":\"images/i3bvrk3c.jpg\",\"actorFullName\":\"Matthias Schweighoefer\"},\n",
    "{\"objectId\":\"images/idbvyk3c.png\",\"actorFullName\":\"Matthias Schweighoefer\"},\n",
    "{\"objectId\":\"images/i2bvrg7d.jpg\",\"actorFullName\":\"Matthias Schweighoefer\"},\n",
    "{\"objectId\":\"images/d5bgrg6d.jpg\",\"actorFullName\":\"Matthias Schweighoefer\"}, \n",
    "{\"objectId\":\"images/d2sgrg8a.jpg\",\"actorFullName\":\"Matthias Schweighoefer\"},     \n",
    "{\"objectId\":\"images/hq1rszje.jpg\",\"actorFullName\":\"Franz Hagn\"},\n",
    "{\"objectId\":\"images/hq2r5yje.png\",\"actorFullName\":\"Franz Hagn\"},\n",
    "{\"objectId\":\"images/06e3ykz8.jpg\",\"actorFullName\":\"Katrin Bauerfeind\"}, \n",
    "{\"objectId\":\"images/hgx13oet.jpg\",\"actorFullName\":\"Tom Beck\"},\n",
    "{\"objectId\":\"images/06e3gkf8.jpg\",\"actorFullName\":\"Tony Garrn\"},\n",
    "{\"objectId\":\"images/evd9sm37.png\",\"actorFullName\":\"Karoline Herfurth\"},\n",
    "{\"objectId\":\"images/evd2am47.png\",\"actorFullName\":\"Karoline Herfurth\"},  \n",
    "{\"objectId\":\"images/eva8sm27.jpg\",\"actorFullName\":\"Karoline Herfurth\"},    \n",
    "]\n",
    "\n",
    "s3_image_bucket = 'mlrekognitiondemo'\n",
    "face_table = dynamodb.Table('rekognition_actors')\n",
    "\n",
    "# function to write reference table in DynamoDB\n",
    "def populate_lookup_table(actorFullName,actorId):\n",
    "    print ('Populating DynamoDB index for ' + actorFullName + ' as ' + actorId)\n",
    "    response = face_table.put_item(\n",
    "                Item={\n",
    "                      'actorId': actorId,\n",
    "                      'actorFullName': actorFullName,\n",
    "                      }\n",
    "                )\n",
    "    return response\n",
    "        \n",
    "def populate_rekognition_collection(objectId):\n",
    "    ret = rekognition.index_faces(\n",
    "    CollectionId=rekognition_face_index,\n",
    "    Image={\n",
    "                'S3Object': {\n",
    "                    'Bucket': s3_image_bucket,\n",
    "                    'Name':  objectId\n",
    "                }\n",
    "            }\n",
    "        )\n",
    "    if ret['ResponseMetadata']['HTTPStatusCode'] == 200:\n",
    "        return ret['FaceRecords'][0]['Face']['FaceId']\n",
    "    else:\n",
    "        return False\n",
    "    \n",
    "\n",
    "# function to populate faces into index\n",
    "def populate_index(all_faces,rekognition_face_index):\n",
    "    for f in all_faces:\n",
    "        ret = populate_rekognition_collection(f['objectId'])\n",
    "        if ret == False:\n",
    "            print ('some error')\n",
    "        else:\n",
    "            ret = populate_lookup_table(f['actorFullName'],ret)\n",
    "        \n",
    "              \n",
    "    print ('done')\n",
    "\n",
    "\n",
    "\n",
    "    \n",
    "rekognition_face_index = 'actors' \n",
    "populate_index(faces,rekognition_face_index)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "sourceurl = 'http://www.presseportal.de/pm/8337/3336775'\n",
    "source = 'obs/Amazon.de/www.stephan-rabold.com'\n",
    "imgurl = 'http://cache.pressmailing.net/thumbnail/story_big/eb0a2593-b7a8-4f3f-8b31-5581004c3426/drehstart-fuer-you-are-wanted-der-ersten-deutschen-amazon-originals-serie-von-und-mit-matthias-schwe'\n",
    "\n",
    "bucket = 'mlrekognitiondemo'\n",
    "key1 = 'images/yawcast'\n",
    "\n",
    "# open image\n",
    "s3.download_file(bucket, key1, 'tmp/image')\n",
    "image_bin=Image.open('tmp/image')\n",
    "\n",
    "# resize image for display\n",
    "image_out=Image.open('tmp/image')\n",
    "size = 900,600\n",
    "image_out.thumbnail(size, Image.ANTIALIAS)\n",
    "\n",
    "# add resized image to image to stream and file for later processing\n",
    "stream = BytesIO()\n",
    "image_bin.save(stream,format=\"JPEG\")\n",
    "image_binary = stream.getvalue()\n",
    "image_bin.save('tmp/image',format=\"JPEG\")\n",
    "\n",
    "# Detect face boxes using Rekognition\n",
    "\n",
    "def face_detection(image_input):\n",
    "    response = rekognition.detect_faces(\n",
    "        Image={\n",
    "            'Bytes':image_input\n",
    "        }                                        \n",
    "        )\n",
    "    all_faces=response['FaceDetails']\n",
    "    \n",
    "    # initialise list object \n",
    "    boxes = []\n",
    "  \n",
    "    # populate list for each face\n",
    "    for face in all_faces:\n",
    "        boxes.append (face['BoundingBox'])\n",
    "    return boxes\n",
    "\n",
    "boxes = face_detection(image_binary)\n",
    "\n",
    "# And highlight them on the image\n",
    "\n",
    "image_out = Image.open(\"tmp/image\")\n",
    "draw = ImageDraw.Draw(image_out)\n",
    "\n",
    "# get image size\n",
    "width, height = image_out.size\n",
    " \n",
    "# apply face boxes to image and store coordinates for later processing\n",
    "def get_coordinates(face_boxes, image_width, image_height):\n",
    "    # initialise list object \n",
    "    coordinates = [] \n",
    "    for box in face_boxes:\n",
    "        x1 = int(box['Left'] * image_width)-5\n",
    "        y1 = int(box['Top'] * image_height)-5\n",
    "        x2 = int(box['Left'] * image_width + box['Width'] * image_width)+5\n",
    "        y2 = int(box['Top'] * image_height + box['Height']  * image_height)+5\n",
    "        if x1 < 0 : x1=0\n",
    "        if y1 < 0 : y1=0\n",
    "        if x2 < 0 : x2=image_width\n",
    "        if y2 < 0 : y2=image_height \n",
    "            \n",
    "        coordinates.append((x1,y1,x2,y2))\n",
    "        \n",
    "    return coordinates\n",
    "    \n",
    "\n",
    "coordinates = get_coordinates(boxes,width,height)\n",
    "    \n",
    "col = 0\n",
    "line= 5     \n",
    "for xy in coordinates: \n",
    "    for l in range(line):\n",
    "        draw.rectangle((xy[0]-l,xy[1]-l,xy[2]+l,xy[3]+l),outline=colors[col])\n",
    "    col += 1\n",
    "        \n",
    "size = 900,600\n",
    "image_out.thumbnail(size, Image.ANTIALIAS)        \n",
    "\n",
    "display(image_out)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Search faces in index\n",
    "\n",
    "def get_face_data(image_binary):\n",
    "    \n",
    "    try:\n",
    "        response = rekognition.search_faces_by_image(\n",
    "            CollectionId='actors',\n",
    "            Image={\n",
    "                'Bytes':image_binary\n",
    "            }                                       \n",
    "            )\n",
    "    \n",
    "        if len(response['FaceMatches']) > 0:\n",
    "            for o in response['FaceMatches']:\n",
    "                print (o['Face']['FaceId'],o['Face']['Confidence'],get_actor_name(o['Face']['FaceId']))\n",
    "            return response['FaceMatches'][0]['Face']['FaceId'],response['FaceMatches'][0]['Face']['Confidence']\n",
    "        else:\n",
    "            return ('no match detected',0)\n",
    "    except Exception as e:\n",
    "        #print (e)\n",
    "        return ('no face detected',0)\n",
    "        \n",
    "    \n",
    "# initialise list object \n",
    "ret = []    \n",
    "\n",
    "# iterate through list of boxes and detect individual face\n",
    "for box in coordinates:\n",
    "        \n",
    "        image_crop = image_bin.crop(box)\n",
    "        display (image_crop)\n",
    "        \n",
    "        # add cropped image to temporary stream\n",
    "        stream2 = BytesIO() \n",
    "        image_crop.save(stream2,format=\"JPEG\")\n",
    "        image_region_binary = stream2.getvalue()    \n",
    "        stream2.close()\n",
    "        \n",
    "        try:\n",
    "            # get external ID name of cropped image\n",
    "            retdata=get_face_data(image_region_binary),box\n",
    "            ret.append (retdata)            \n",
    "        except Exception as e:\n",
    "            print (e, box)\n",
    "            ret.append ((('no face',0),box))\n",
    "            pass\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Output matches on image\n",
    "\n",
    "font = ImageFont.load_default()\n",
    "img_out = Image.open(\"tmp/image\")\n",
    "draw = ImageDraw.Draw(img_out)\n",
    "\n",
    "font = ImageFont.truetype(\"/Library/Fonts/Arial.ttf\", 22)\n",
    "\n",
    "col = 0\n",
    "xpos = 0\n",
    "ypos = 30\n",
    "line = 5\n",
    "for f in ret:\n",
    "    \n",
    "    actor_name=get_actor_name(f[0][0]) \n",
    "    xpos = len(actor_name)*2.5\n",
    "    draw.text((f[1][0]-xpos,f[1][1]-ypos),actor_name,colors[col],font=font)\n",
    "    for l in range(line):\n",
    "        draw.rectangle((f[1][0]-l,f[1][1]-l,f[1][2]+l,f[1][3]+l),outline=colors[col])\n",
    "    col += 1\n",
    "    \n",
    "display(img_out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Celebrity detection\n",
    "\n",
    "Amazon Rekognition can recognize thousands of celebrities in a wide range of categories, such as entertainment and media, sports, business, and politics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "aws rekognition recognize-celebrities \\\n",
    "--image '{\"S3Object\":{\"Bucket\":\"mlrekognitiondemo\",\"Name\":\"images/yawcast\"}}' --region eu-west-1 \\\n",
    "--query 'CelebrityFaces[*].{Name:Name,Confidence:MatchConfidence,Details:Urls,Face:Face.BoundingBox}' "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Content moderation\n",
    "\n",
    "Provides image moderation to suggestive or explicit content that may not be appropriate for your audience based on two category and eight child category labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bucket = 'mlrekognitiondemo'\n",
    "key = 'images/david-full-front.jpg'\n",
    "key = 'images/david-pants.jpg'\n",
    "\n",
    "# retrieve labels from Rekognition service\n",
    "ret = rekognition.detect_moderation_labels(\n",
    "    Image={\n",
    "        \"S3Object\": {\n",
    "            \"Bucket\": bucket,\n",
    "            \"Name\" : key,\n",
    "        }\n",
    "    }\n",
    ")\n",
    "\n",
    "# Output image and labels\n",
    "image_location = 'tmp/image'\n",
    "s3.download_file(bucket, key, image_location)\n",
    "\n",
    "inner = \"<table>\"\n",
    "for i in ret['ModerationLabels']:\n",
    "    if i['ParentName'] is '': \n",
    "        inner += \"<tr><td><b>\" + i['Name'] + \" (parent)</b></td><td>\" + str(i['Confidence']) + \"</td></tr>\"\n",
    "    else:\n",
    "        inner += \"<tr><td>\" + i['Name'] + \"</td><td>\" + str(i['Confidence']) + \"</td></tr>\"\n",
    "display(HTML(\"<table><tr><td width='50%'><img src='tmp/image?\"+str(time.time())+\"' ></td><td> \\\n",
    "                \"+inner+\"</table></td></tr></table>\"))  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Text detection\n",
    "\n",
    "Detect text in images and convert it into machine-readable text.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><tr><td width='50%'><img src='tmp/image?1512402530.9996932' ></td><td>                 <table><tr><td><b>Text</td><td><b>Confidence %</td><td><b>Type</td><td><b>Id</td><td><b>Parent Id</td></tr><tr><td>IT'S</td><td>97.4773178100586</td><td>LINE</td><td>0</td><td></td></tr><tr><td>MONDAY</td><td>92.79122924804688</td><td>LINE</td><td>1</td><td></td></tr><tr><td>but keep</td><td>94.46160125732422</td><td>LINE</td><td>2</td><td></td></tr><tr><td>Smiling</td><td>99.04523468017578</td><td>LINE</td><td>3</td><td></td></tr><tr><td>IT'S</td><td>97.4773178100586</td><td>WORD</td><td>4</td><td>0</td></tr><tr><td>MONDAY</td><td>92.79122924804688</td><td>WORD</td><td>5</td><td>1</td></tr><tr><td>but</td><td>96.57958984375</td><td>WORD</td><td>6</td><td>2</td></tr><tr><td>keep</td><td>92.34361267089844</td><td>WORD</td><td>7</td><td>2</td></tr><tr><td>Smiling</td><td>99.04523468017578</td><td>WORD</td><td>8</td><td>3</td></tr></table></td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "bucket = 'console-sample-images-dub'\n",
    "key = 'coffee_monday.jpg'\n",
    "#key = 'license_plate.jpg'\n",
    "\n",
    "# retrieve labels from Rekognition service\n",
    "ret = rekognition.detect_text(\n",
    "    Image={\n",
    "        \"S3Object\": {\n",
    "            \"Bucket\": bucket,\n",
    "            \"Name\" : key,\n",
    "        }\n",
    "    }\n",
    ")\n",
    "\n",
    "# Output image and labels\n",
    "image_location = 'tmp/image'\n",
    "s3.download_file(bucket, key, image_location)\n",
    "\n",
    "\n",
    "inner = \"<table><tr><td><b>Text</td><td><b>Confidence %</td><td><b>Type</td><td><b>Id</td><td><b>Parent Id</td></tr>\"\n",
    "for i in ret['TextDetections']:\n",
    "    if not 'ParentId' in i :\n",
    "        inner += \"<tr><td>\" + i['DetectedText'] + \"</td><td>\" + str(i['Confidence']) + \"</td><td>\" + str(i['Type']) + \"</td><td>\" + str(i['Id']) + \"</td><td></td></tr>\"\n",
    "    else:\n",
    "        inner += \"<tr><td>\" + i['DetectedText'] + \"</td><td>\" + str(i['Confidence']) + \"</td><td>\" + str(i['Type']) + \"</td><td>\" + str(i['Id']) + \"</td><td>\" + str(i['ParentId']) +\"</td></tr>\"\n",
    "display(HTML(\"<table><tr><td width='50%'><img src='tmp/image?\"+str(time.time())+\"' ></td><td> \\\n",
    "                \"+inner+\"</table></td></tr></table>\"))  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://eu-west-1.console.aws.amazon.com/rekognition/home?region=eu-west-1#/usage-metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "all the dependencies that need to be installed:\n",
    "\n",
    "pip3 install boto3\n",
    "\n",
    "pip3 install pillow\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}