x-pyannote.ipynb

pyannote.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "name": "pyannote.ipynb",
      "gpuType": "A100",
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/StanAngeloff/91480fac18a74d8aff3e4cf566cfd0ff/pyannote.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import locale\n",
        "\n",
        "def getpreferredencoding(do_setlocale = True):\n",
        "    return \"UTF-8\"\n",
        "\n",
        "locale.getpreferredencoding = getpreferredencoding"
      ],
      "metadata": {
        "id": "rudtRKL9P7KK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lbtdzoCjO7DU"
      },
      "outputs": [],
      "source": [
        "!pip install \\\n",
        "  git+https://github.com/pyannote/pyannote-audio.git@7379f1c82be093078354449100e1a84cbdfbafdf \\\n",
        "  git+https://github.com/openai/whisper.git@248b6cb124225dd263bb9bd32d060b6517e067f8 \\\n",
        "  torch==2.0.0+cu118 torchvision==0.15.1+cu118 torchaudio==2.0.1 \\\n",
        "    --extra-index-url https://download.pytorch.org/whl/cu118"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "\n",
        "torch.cuda.is_available()"
      ],
      "metadata": {
        "id": "J_Ss89GhRlvq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "\n",
        "drive.mount('/content/gdrive')"
      ],
      "metadata": {
        "id": "10-Kt8ghQFK1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from huggingface_hub import notebook_login\n",
        "\n",
        "notebook_login()"
      ],
      "metadata": {
        "id": "KOvvVyKSPJck"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import soundfile as sf\n",
        "\n",
        "sf.available_formats()"
      ],
      "metadata": {
        "id": "oNpve47cPJ9n"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "75IRpeQnxJYW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import whisper\n",
        "import torch\n",
        "\n",
        "device = torch.device(\"cuda\")\n",
        "\n",
        "model = whisper.load_model(\"large\", device=device)"
      ],
      "metadata": {
        "id": "x0CLqnvDwg8P"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from pyannote.audio import Pipeline\n",
        "from pyannote.audio.pipelines.utils.hook import ProgressHook\n",
        "\n",
        "#device = torch.device(\"cuda\")\n",
        "\n",
        "speaker_diarization = Pipeline.from_pretrained(\n",
        "    \"pyannote/[email protected]\",\n",
        "    use_auth_token=True\n",
        ")\n",
        "\n",
        "speaker_diarization.to(device)"
      ],
      "metadata": {
        "id": "3rKnEyruPKwu"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!nvidia-smi"
      ],
      "metadata": {
        "id": "yzH_lXFOv2ur"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cp \"/content/gdrive/MyDrive/Recordings/Day 1/Session 1.ogg\" /content/target.ogg"
      ],
      "metadata": {
        "id": "CHGdyOCFxCiQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "05L9PUV9xb8k"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "with ProgressHook() as hook:\n",
        "    who_speaks_when = speaker_diarization(\n",
        "        \"/content/target.ogg\",\n",
        "        num_speakers=2,\n",
        "       #min_speakers=5,\n",
        "       #max_speakers=9,\n",
        "        hook=hook\n",
        "    )"
      ],
      "metadata": {
        "id": "PEH4cv5VPMfC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from pyannote.core import Segment\n",
        "from pyannote.audio import Audio\n",
        "\n",
        "speakers = who_speaks_when.rename_labels({\n",
        "    #\"SPEAKER_00\": \"David\",\n",
        "    #\"SPEAKER_01\": \"Stan\",\n",
        "})\n",
        "\n",
        "crop = Segment(0, 999999999)\n",
        "#crop = Segment(5 * 60.0, 10 * 60.0)\n",
        "audio = Audio(sample_rate=16000, mono=\"downmix\")\n",
        "\n",
        "def float_to_timestamp(float_time):\n",
        "    hours, remainder = divmod(float_time, 3600)\n",
        "    minutes, seconds = divmod(remainder, 60)\n",
        "    return \"{:02}:{:02}:{:04.1f}\".format(int(hours), int(minutes), seconds)\n",
        "\n",
        "for segment, _, speaker in speakers.crop(crop).itertracks(yield_label=True):\n",
        "    waveform, sample_rate = audio.crop(\"/content/target.ogg\", segment)\n",
        "    text = model.transcribe(waveform.squeeze().numpy(), language=\"en\", initial_prompt=\"\"\"\n",
        "A recorded conversation between Company A consisting of Stan & David and Client consisting of … discussing a new project requirements and demonstrating … and current standard operating procedures in …, including Campaign Planning and Digital Marketing, Customer Service, IT Process, Finance.\n",
        "\"\"\".strip())[\"text\"]\n",
        "    print(f\"{float_to_timestamp(segment.start)}-{float_to_timestamp(segment.end)} {speaker.strip()}: {text.strip()}\")"
      ],
      "metadata": {
        "id": "JbC9Qj0k5fuF"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}

Here's a tutorial on how to use GPT-4 for analyzing and generating detailed reports from the transcribed notes of lengthy meetings.

Begin by setting up GPT-4 with the following prompt, which instructs the AI to analyze the meeting notes meticulously:

You are a helpful bot that analyses meetings. Your input is a partial conversation log in chronological order in the format "(Person Name): (What was said…)" without the quotes.

From Company A we have: Stan and David.
From Client we have: ….

The meeting objective is for Company A (Us) to understand and scope a new project for Client.

Your job is to analyse everything that has been said in the meeting.
You will output extensive and very detailed meeting minutes. Do not create numbered lists in your output. You will output paragraphs explaining all important aspects discussed during the meeting. You will include detailed information of each high-level conversation point.
You will not summarize the conversation.

You will not omit actions which Company A needs to take.
You will not omit features, enhancements, apps, systems, etc. which Client talks about.
You will not omit any previous features or apps which Client may want to bring back.
You will not omit actions or operations the system will not allow.
You will not omit anything that Client considers important.
You will not omit anything that is legally mandatory.

You will not omit any details regarding the payments, income, bonuses, discounts, campaigns, ….

When you see ACRONYM, interpret this as … (ACRONYM).

When you are finished, you will output 'Fin.' on a line of its own.

The output generated will be the detailed meeting minutes. For effective processing, consider breaking down the input content and feed approximately 3000 tokens at a time into GPT-4. This approach helps prevent the loss of essential information.

After you have the comprehensive minutes, compile them for further analysis. You can then instruct GPT-4 to summarize high-level topics, as shown in the following prompt:

You are a helpful bot that analyses meetings.
Your input is a detailed report of everything which was discussed during the meeting.

Your will analyse everything that has been said in the meeting.
Your output is a top-level list of every subject, topic, idea, requirement, operating procedure, process being discussed.
You will not create sub-lists.
You will attempt to group list items together when they are related and come up with a description that describes the group.

GPT-4's output here should offer a solid, categorical list of topics. Use your judgement to refine this list further by combining similar items and restructuring the list for conciseness.

Once you have the high-level overview, proceed to create in-depth reports for each topic using the unchanged initial detailed meeting report. For this, employ the following prompt structure for each topic:

You are a helpful bot that analyses meetings.
Your input is a detailed report of everything which was discussed during the meeting.

I will give you a list of high-level topics, subjects, ideas, etc. which were discussed at the meeting. I will include additional context information in brackets.
For each item on the list you will analyse everything that has been said in the meeting regarding the item. Your output will follow the format:

\`\`\`
## LIST ITEM

Long exhaustive description of LIST ITEM sourced from meeting (multiple paragraphs if needed).

Current Operating Procedures:

- Operating Procedure 1…
- Operating Procedure 2…
- Operating Procedure 3…
- Operating Procedure 4…
- Add as many as needed

Requirements:

- Requirement 1
- Requirement 2
- Requirement 3
- Requirement 4
- Add as many as needed

Action Points:

1. Action Point 1 - short introduction followed by longer description.
  - Sub-action 1.1 - description
  - Sub-action 1.2 - description
  - Add as many as needed
2. Action Point 2
3. Add as many as needed

A final conclusion and summary paragraph for LIST ITEM.
\`\`\`

Here is the list I promised:

- TOPIC 1
- TOPIC 2

When working topic by topic, it's advisable not to ask GPT-4 to handle more than two items simultaneously as it may struggle to maintain focus. One topic at a time is typically the most manageable approach.

---

When you're done with the process, you're going to end up with a report that translates your meeting's chaos into a neatly organized playbook of what's next. Basically, it's like having the ultimate meeting notes that don't just remind you what was said, but what you gotta do about it.