1901010101-OGI.ipynb

1901010101-ogi.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/odtmusisi19/1fbea1b94eb526d77084a1d551f454c3/1901010101-ogi.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jzRKNe1iSlNW"
      },
      "source": [
        "# Google Colab: Access Webcam for Images and Video\n",
        "This notebook will go through how to access and run code on images and video taken using your webcam.  \n",
        "\n",
        "For this purpose of this tutorial we will be using OpenCV's Haar Cascade to do face detection on our Webcam image and video."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "Fj9YcAnsT4B_"
      },
      "outputs": [],
      "source": [
        "# import dependencies\n",
        "from IPython.display import display, Javascript, Image\n",
        "from google.colab.output import eval_js\n",
        "from base64 import b64decode, b64encode\n",
        "import cv2\n",
        "import numpy as np\n",
        "import PIL\n",
        "import io\n",
        "import html\n",
        "import time"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MnTDlBQDfXiP",
        "outputId": "f4b18761-b302-4f1f-e634-9a4f48697c8d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ],
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L6pCmkJrUC9g"
      },
      "source": [
        "## Helper Functions\n",
        "Below are a few helper function to make converting between different image data types and formats. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "09b_0FAnUa9y"
      },
      "outputs": [],
      "source": [
        "# function to convert the JavaScript object into an OpenCV image\n",
        "def js_to_image(js_reply):\n",
        "  \"\"\"\n",
        "  Params:\n",
        "          js_reply: JavaScript object containing image from webcam\n",
        "  Returns:\n",
        "          img: OpenCV BGR image\n",
        "  \"\"\"\n",
        "  # decode base64 image\n",
        "  image_bytes = b64decode(js_reply.split(',')[1])\n",
        "  # convert bytes to numpy array\n",
        "  jpg_as_np = np.frombuffer(image_bytes, dtype=np.uint8)\n",
        "  # decode numpy array into OpenCV BGR image\n",
        "  img = cv2.imdecode(jpg_as_np, flags=1)\n",
        "\n",
        "  return img\n",
        "\n",
        "# function to convert OpenCV Rectangle bounding box image into base64 byte string to be overlayed on video stream\n",
        "def bbox_to_bytes(bbox_array):\n",
        "  \"\"\"\n",
        "  Params:\n",
        "          bbox_array: Numpy array (pixels) containing rectangle to overlay on video stream.\n",
        "  Returns:\n",
        "        bytes: Base64 image byte string\n",
        "  \"\"\"\n",
        "  # convert array into PIL image\n",
        "  bbox_PIL = PIL.Image.fromarray(bbox_array, 'RGBA')\n",
        "  iobuf = io.BytesIO()\n",
        "  # format bbox into png for return\n",
        "  bbox_PIL.save(iobuf, format='png')\n",
        "  # format return string\n",
        "  bbox_bytes = 'data:image/png;base64,{}'.format((str(b64encode(iobuf.getvalue()), 'utf-8')))\n",
        "\n",
        "  return bbox_bytes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MaZIOR4WaT64"
      },
      "source": [
        "## Haar Cascade Classifier\n",
        "For this tutorial we will run a simple object detection algorithm called Haar Cascade on our images and video fetched from our webcam. OpenCV has a pre-trained Haar Cascade face detection model. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "ZpA68lTrcvZs",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8a9f3ba9-3241-44d5-befc-980941d70a02"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "< cv2.CascadeClassifier 0x7fa5ed328c90>\n"
          ]
        }
      ],
      "source": [
        "# initialize the Haar Cascade face detection model\n",
        "# from google.colab import files\n",
        "# body = files.upload()\n",
        "face_cascade = cv2.CascadeClassifier(cv2.samples.findFile(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml'))\n",
        "eye_cascade = cv2.CascadeClassifier(cv2.samples.findFile(cv2.data.haarcascades + 'haarcascade_eye.xml'))\n",
        "smile_cascade = cv2.CascadeClassifier(cv2.samples.findFile(cv2.data.haarcascades + 'haarcascade_smile.xml'))\n",
        "fullbody = cv2.CascadeClassifier(cv2.samples.findFile(\"/content/drive/MyDrive/Colab Notebooks/haarcascades/haarcascade_fullbody.xml\"))\n",
        "lowerbody = cv2.CascadeClassifier(cv2.samples.findFile(\"/content/drive/MyDrive/Colab Notebooks/haarcascades/haarcascade_lowerbody.xml\"))\n",
        "upperbody = cv2.CascadeClassifier(cv2.samples.findFile(\"/content/drive/MyDrive/Colab Notebooks/haarcascades/haarcascade_upperbody.xml\"))\n",
        "profileface = cv2.CascadeClassifier(cv2.samples.findFile(\"/content/drive/MyDrive/Colab Notebooks/haarcascades/haarcascade_profileface.xml\"))\n",
        "\n",
        "\n",
        "print(fullbody)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fRFoJo6QT94w"
      },
      "source": [
        "## Webcam Images\n",
        "Running code on images taken from webcam is fairly straight-forward. We will utilize code within Google Colab's **Code Snippets** that has a variety of useful code functions to perform various tasks.\n",
        "\n",
        "We will be using the code snippet for **Camera Capture** to utilize your computer's webcam."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "X9Eb-KWAfvug"
      },
      "outputs": [],
      "source": [
        "from google.colab.patches import cv2_imshow\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 94,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 684
        },
        "id": "u6pdu43TfPaj",
        "outputId": "f4b0ffc0-b27f-41fd-d8b3-be48df9a41e2"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<PIL.Image.Image image mode=RGB size=1600x748 at 0x7FA5BC2804F0>"
            ],