Created
March 10, 2025 10:04
-
-
Save snowch/95146268d671bd4464c0870189f6bdf9 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "cells": [ | |
| { | |
| "cell_type": "markdown", | |
| "id": "0c9e547f", | |
| "metadata": { | |
| "id": "0c9e547f" | |
| }, | |
| "source": [ | |
| "[](https://colab.research.google.com/github/langchain-ai/langchain-academy/blob/main/module-3/streaming-interruption.ipynb) [](https://academy.langchain.com/courses/take/intro-to-langgraph/lessons/58239464-lesson-1-streaming)" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "id": "319adfec-2d0a-49f2-87f9-275c4a32add2", | |
| "metadata": { | |
| "id": "319adfec-2d0a-49f2-87f9-275c4a32add2" | |
| }, | |
| "source": [ | |
| "# Streaming\n", | |
| "\n", | |
| "## Review\n", | |
| "\n", | |
| "In module 2, covered a few ways to customize graph state and memory.\n", | |
| "\n", | |
| "We built up to a Chatbot with external memory that can sustain long-running conversations.\n", | |
| "\n", | |
| "## Goals\n", | |
| "\n", | |
| "This module will dive into `human-in-the-loop`, which builds on memory and allows users to interact directly with graphs in various ways.\n", | |
| "\n", | |
| "To set the stage for `human-in-the-loop`, we'll first dive into streaming, which provides several ways to visualize graph output (e.g., node state or chat model tokens) over the course of execution." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 1, | |
| "id": "db024d1f-feb3-45a0-a55c-e7712a1feefa", | |
| "metadata": { | |
| "id": "db024d1f-feb3-45a0-a55c-e7712a1feefa" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "%%capture --no-stderr\n", | |
| "%pip install --quiet -U langgraph langchain_openai langgraph_sdk" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "id": "70d7e41b-c6ba-4e47-b645-6c110bede549", | |
| "metadata": { | |
| "id": "70d7e41b-c6ba-4e47-b645-6c110bede549" | |
| }, | |
| "source": [ | |
| "## Streaming\n", | |
| "\n", | |
| "LangGraph is built with [first class support for streaming](https://langchain-ai.github.io/langgraph/concepts/low_level/#streaming).\n", | |
| "\n", | |
| "Let's set up our Chatbot from Module 2, and show various way to stream outputs from the graph during execution." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 2, | |
| "id": "5b430d92-f595-4322-a56e-06de7485daa8", | |
| "metadata": { | |
| "id": "5b430d92-f595-4322-a56e-06de7485daa8", | |
| "outputId": "b3664fb5-a33d-4007-93a9-c15acc5afba0", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "name": "stdout", | |
| "output_type": "stream", | |
| "text": [ | |
| "OPENAI_API_KEY: ··········\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "import os, getpass\n", | |
| "\n", | |
| "def _set_env(var: str):\n", | |
| " if not os.environ.get(var):\n", | |
| " os.environ[var] = getpass.getpass(f\"{var}: \")\n", | |
| "\n", | |
| "_set_env(\"OPENAI_API_KEY\")" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "id": "4d0682fc", | |
| "metadata": { | |
| "id": "4d0682fc" | |
| }, | |
| "source": [ | |
| "Note that we use `RunnableConfig` with `call_model` to enable token-wise streaming. This is [only needed with python < 3.11](https://langchain-ai.github.io/langgraph/how-tos/streaming-tokens/). We include in case you are running this notebook in CoLab, which will use python 3.x." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 3, | |
| "id": "2d7321e0-0d99-4efe-a67b-74c12271859b", | |
| "metadata": { | |
| "id": "2d7321e0-0d99-4efe-a67b-74c12271859b", | |
| "outputId": "7de676ef-a812-4bb7-a148-2b932b91abe4", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 350 | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQMAAAFNCAIAAACL4Z2AAAAAAXNSR0IArs4c6QAAIABJREFUeJzt3WdcU+fbB/A7kyzC3nsoblFR3IKCG1txax11rypad617710HFlHcVtFKLQ4c4GilDhQVlL1HIISE7OR5cXhS/phEpUlOItf3w4uQs64D+eU+8z4EpVKJAGj0iHgXAIBRgCQAgCAJANSCJACAIAkA1IIkAIAQQmS8CwD/iVyqLM0TC3gyAU+mkCOJSIF3RZ9mRieSKQQGm8xkkx08zPAupxYBzieYIqlI+S6Zl/laUJghtHczY7LJDDbJwpZiGkmgkSpKJAKejEQmZL8ReLVi+rQ29/Vn4lsVJMH0/HWjIuuNwMmT5tWK6e7HwLuc/0QqUWa95ue8rcl9V9M1zKZ5JzZelUASTMmHF4KbMUUd+1p37GuNdy06JuTLH14rryyV9v3OwcKWYvgCIAkm43EcR1yj6BluSyQR8K5FX6rKpVcPF3YfYuvdxtAbS5AE0/D4OodKI3YIscK7EEP449eitj0tXXzphlwoJMEE/BldbONs1jG0UcQAE3e8yL0Zo3U3C4MtEc4nGLunNyss7SiNKgYIoUFTnNL/qS7KFBlsiZAEo5bzpkYkUHQeaIN3ITgYNs/16a0KidBAx4UhCUbt/uXStj0Nt4VgbHz9WYlXyw2zLEiC8Up9zHNrwmDb4HBI0Ui0CGQXZgi5ZVIDLAuSYLwyUvjdvrHFuwqc9Qy3e5VUZYAFQRKMVGGmSCpRUGkG/QctXbr0999/b8CEISEhhYWFeqgIuTdjvEzk6mPO9UASjFTWa753K5aBF/r27dsGTFVcXMzl6uvDSiAgj+aM7Dc1epr/vwuC8wnG6dqRwqDhdnraSYiNjT1z5kxBQQGNRmvfvv2iRYscHBwCAgKwoSwW6969e3K5/NixY3/++WdpaamFhUWvXr3mz59Pp9OxpoNAIHh6esbExEyePPnQoUPYhL169dq5c6fOq01LruYUSbqG6fkAmhIYpf0R75UKvcz52bNnHTp0uHz5cl5e3qtXr6ZOnTpp0iSlUllSUtKhQ4dz585xuVylUnny5MnAwMD4+PicnJzHjx/3799/+/bt2Bx++umnYcOGzZ8//59//ikrK7t582aHDh3evn3L5/P1UXBees3lA/n6mHNdcH+CMarhyRnmJKSfy4syMjLMzMzCwsLIZLKrq+uWLVuKiooQQhYWFgghBoOBvRgwYECXLl18fX0RQu7u7n379n348KFqJvn5+cePH8fGZDKZCCE2m4290DmmBVlQJdPHnOuCJBgjAU/GYOvrXxMQEEAgEKZOnfrNN98EBgY6Ozvb2KjZ8LC0tIyLi9uwYUNpaalMJqupqWEw/r0C3MPDA4uBATDZJAFPru+lwB6zMVIqkRldX/8aT0/PqKgoV1fX/fv3DxkyZNKkSa9fv/54tO3bt0dGRo4cOfLYsWNnzpwZOnRo3aEsluH25okkggGOoUESjBHDnKTX00lNmjTZsGHDrVu3jhw5QiKRIiIiJBJJ3RHkcvnVq1cnTpw4cOBAFxcXW1tbPp+vv3q0E1TJSGS9X4gOSTBGTDa5hqevLePXr1+npKQghEgkUocOHWbNmsXlcjkcDjYUO5aoUCjkcrlq+0cgEDx48ED7YUb9HYSs4cmZbJKeZq4CSTBGBCLyaM4QVutl4/jRo0cLFy68c+dOfn5+WlrauXPnnJycHB0dzczMzMzMnj17lpaWRiAQ/Pz8rl+/np+f//79+4iIiG7duvF4vOzsbJmsfkTZbDZCKCkpKTMzUx8FCwVyRw+aPuZcFyTBSDEtyBmv9LJBMnny5KFDh+7Zs2f48OFz5sxRKpX79u0jEAgIoUmTJt2+fXv27NlCoXDVqlVyuXzkyJHLly8fPXr0nDlzHB0dJ0yYUFpaWm+GzZs379q16+7du7dt26aPgt8/r7Z303sS4Myakcp+U/PqITdsmjPeheDv8NKMKeu9KVT97ipAm2CkPJszJCIlavRfU8VZoibtzPUdAzifYMQIyN2P/uQGR8ttOiEhIR9vtWNHfkgkjbuYV69e1dOpgBcvXkRERKgdJJFIqFSq2kFeXl5RUVGa5vnw9/Kugw1xQS5sHRm1I0szJq/zppip/0YsKipS++8Ti8UUCoVIVN/gOzo6ahr0H4nFYtUxqHr4fD6DwVC7XAqFYmdnp3aq7FTB60dVgw2yiQhJMGpv/6rmc2Ud+zWum5hV4qNLOvaztnY0xL1KsJ9g1JoHmvMqpG//4uFdCA5uny7xaMEwTAwgCSagzxj7lKSq3DQh3oUY1MNrHBqL1KyjucGWCFtHpuH3I4Wtult4tcS5G13DeHSdw7Ikt+lu0K4MoE0wDWEznFMf817cM8R9jPiKO15EoRIMHANoE0xM8s3Kt0953cJw6DbUAJ7f5T6/Wxk0wt67NQ5rB0kwMdwy6aPfywkE5NqU4dWSybI0+TNC5QWSnLeC5/e4zTqadx1sS9T7tXbqQRJMUkmO6O3T6qzXAjqLpHqSiLklWSYzgf8miUTgcaQCnlyhUH54wacxiN6tWa27W9BZOIUAIUiCySvLF5fmi2t4MgFPTiQggU4vXxWLxW/evGnXrp0O54kQYlmSkRIx2CRzK4qzN81ImjVIAtCoqKho2rRp169fx7sQQ4BjRwAgSAIAtSAJQCMCgeDj44N3FQYCSQAaKZXKjIwMvKswEEgC0Aa7R7kxgCQAbXi8xnIZLCQBaEQgEBwdHfGuwkAgCUAjpVJZXFyMdxUGAkkA2jRt2hTvEgwEkgC0SU9Px7sEA4EkAIAgCeATrKwaS2cCkASgTWVlJd4lGAgkAWij9iEjXyVIAtBGU09eXx9IAgAIkgA+wcvLC+8SDASSALTJysrCuwQDgSQAgCAJ4BPgagsAEFxtAUCjA0kAGmFP4MS7CgOBJACNlEplWloa3lUYCCQBAARJANpALy8AIOjlBYDGCJIAtIH+jgBA0N8RALXgWlQAEFyLCkCjA0kA2tjb2+NdgoFAEoA2paWleJdgIJAEoA3cnwAAgvsTAKgFbQIACNoEAGo5OzvjXYKBwJPJQX3jxo2rrq4mEAgymayystLW1pZAIEgkkhs3buBdmh5BmwDqGz16NIfDKSgoKCkpkUgkhYWFBQUFROJX/lH5ylcPNEBYWJi7u3vdd5RKZadOnfCryBAgCUCNsWPHmpmZqX61t7efOHEirhXpHSQBqBEWFubq6oq9ViqVgYGBnp6eeBelX5AEoN748eOxZsHBwWHChAl4l6N3kASg3uDBg93d3bE9BG9vb7zL0Tsy3gU0RhKRklMkrqmWGfkR7PB+s69Lr4d2Hf/hJR/vWrQhEglsa4q1I5VIavhM4HyCoSVeKc9I4TPZZDqbDH97nWCYk0qyhVQaqWVn8+aBDbzxGpJgUDdjSsxtzFp1tcS7kK/Tg4slnq0YLTubN2Ba2E8wnITzpRZ2NIiB/vQc4ZD5ip/+T3UDpoUkGAinSFLNlbfobIF3IV+5rmEOKQ+r0Jdv6EASDIRTKKZQ4a+td1Q6kVcuFfDkXzoh/G8MhF8ls7Cj4l1Fo2DvTudxJF86FRxFNRC5TCmX4l1E4yDkyxpwFAjaBAAQJAGAWpAEABAkAYBakAQAECQBgFqQBAAQJAGAWpAEABAkAYBakAQAECQBGM7lK+f7hBpvp0mQBKBHV2IvbNm2Bnvdzj8gYv4yvCvSCK5FBXqUnv5W9drLy8fLywfXcrSBJBgvqVR6IvrIzVtxfH61r6/fjGnzWrVqixCSSCTHfz10997NysoKGxvbkD4DJk2cQSaTEUJDh4WOHzelpLQ44W68UFjTunW7RQtX0mj08OGhEydMHztmkmrO4cNDh4QNnzZ1Lpdbeejw7pcv/6mq4np7N5k2dW47/wCEUFZWxuSpozau33U0cj+dRv/l0MmUlOeRvx7Myvogl8t9fJpOnTynbdv2CKHKyopfjux59uzv6mqenZ1D+LejwsNHI4QiFk5/+fIZQig+/vrRI6dfvXpx8NDOO7f+1rIKOTlZkyaP2LXz8G+Xz7569YJIJAYHhc6Z/SOJ9B96rfg8sHVkvH45vDvuj9jZsxbu2X3MxcVtybK5hUUFCKE9e7fc+PPazBkRJ6IuTZk850rs+SNH92GTkMnks+ejPT29z57+/dfIC+/fvzsVE8lkMgM7dUtMuqua8z///MXn8/v07q9QKJYu+yE1NWXpkjVHfolp5tdi2fJ5mZkfEEIUCgUhFH3y6KiR4xcvWiUUClesjPD08D6wL+rQgWgf7ybLVszjVfMQQtt2rHuTmvLzT5sij54dO2bSwV92JT28hxDasG5X0ybNegf3jb1829vLt+6qaVoFEpmMEDp4aOeYUROvXrmz8qeNV2IvPEhMMMBfG9oEIyUQCOL+iJ0xfX5wUChC6McFPwlragoK8pgM5s1bcTNnzO8d3Bch5OLsmpubdem3M9On/YB9dj3cvQb0H4IQsrd36NSxa1raG4RQcHDfdeuXl5WV2tnZI4TuP7jj5eXj7e3799PH6e/f7dp5GGsH5s5ZlPzPX5evnFv040pEICCE/P0DsLnl5GQJBILQkIEeHl7YmEG9QqkUKkJozuwfiUSis5MLQsjNzePq1YvJyU+6dwtisVgkMplCpVpY/E8nBlVVXE2rgI3Qq2dIy5ZtEEId2ndydnJJS3uD/RH0CpJgpHJyMiUSSfNmLbFfKRTK2jXbEELPnj+Vy+UtmrdWjenn10IkEuXn52Jb4d7eTVSDzM3Z2Nd2l849aDRa0sN7Q78dKZPJHj1+MHLEdwiht29fUygU/7YdsPGJRGKb1u0+fEhTzaFFi9oFubq6u7l5bNy8ckjY8ICAzk18/fz9a6ei0+hnzp148SK5qoqrUCiqq3kuLm5aVi0j872mVaBQqQghnzqrwGKZ8/kN6aviS0ESjFQ1vxohZGZGq/d+TY0AIcRgMFXv0OkMhJBQWIP9WrePa4QQASGEEI1G69K5R2JiwtBvRz5/kczjVfXu3Q+bm1Qq7Tegq2p8uVxubW2j+pXJZGEvSCTSvj2RZ89Fx8VdORZ5wMHBcfKkWX37DpLJZEuWzZXL5XPnLHJ38ySRSCtX/ah91bSsApYE6v+ugmG65IIkGClsiwL70NSFfTTrvo+9Vn1kNQkO7rt23bIqXlViYkKLFq2dHJ2xqahU6rEjZ+qOqemhIZaWVrNmRsyaGZGdnXnhYszmras9PL0lYnFm5oe9u4+1adMOG62KW4nNXJMGr4JewR6zkXJ1cafRaC9TnmG/KhSK+Qumxcdf9/ZuQiKRXqe+VI2ZmprCYrG0b5AghDp17GpmZvb3348ePrrfp3d/7M1mzVpKJBK5XO7u7on9UKlmtrb2H09eWFSQlHQPe+3p6b1wwQoikZidlSGWiBFCbLaFqpii4sK63+Iff6M3eBX0CpJgpFgs1oD+Q06f+fXmzbi09Le7dm9KT3/bqrW/BdtiQP8hp89EJSXdKykpjo+/fvXaxWHhY7CjqFqYmZl17drr/IWTXG6lage0Q/tOTXz9Nm3++cWLf4qKC2/f+XP6jLFXr138ePLSkuLVa5dcuBiTm5udl5dzKiaSSCS2aNHa16cplUq9fOUch1P+NPnJvv3bOgZ0zsvPqaysQAiZs8w/fEh7/yGtqoqrmlWDV0GvYOvIeM2YPp9AJB4+ulcorPHy8t28ca+LsytCaN4PSxgM5p59W7jcSns7h+/GTVGdKNCud1DfFbdvdAzobGVljb1DIpG2btn/y5E9q9cuEYmEjo7O48dPHTF83MfT+vt3WLp49YVLMVEnDpNIJA8P7/Vrd7i5eSCElixeHRl54OatuKZNmy9dsqasvHT9huULF82MOn5h6NDRm7esmjd/yto12+vOrcGroD/QQ7CBPL1ZIRSgdr2t8S7k6/dnVH63ITbO3vQvmgq2jgBAkAQAakESAECQBABqQRIAQJAEAGpBEgBAkAQAakESAECQBABqQRIAQJAEAGpBEgBAkATDoTFIZAreRTQOLAsKmfLFH2xIgoFY2VOLsoR4V9EoZL6qtnMx+4wR/wckwUCcfekKuVLekCcFgy9Qki3yC2ATvvxzDUkwECIRdRlsczumEO9CvmZCvvzB5eLeo+waMC3cs2ZQJbnia0cK2ve2tbCjMtgkBH97XSCSCJWlEgFX+vJ+xfgVHlR6Q77fIQmGJhLI/7nDLc4W1vDlSnlD5qBUKKqqqiytrHRfHE4UCjmfL2Cz2Q2b3MKOQiAiVx9G+z6WnzG6epAE07N+/frx48d7enriXYguxcfHV1ZWjh49Gq8CIAmm5MSJE5Mm4dwHhP7IZDIymXz8+PEpU6YYfumwx2wygoKCOnfujHcVeoT1d8RgMHbs2GH4pUObYAKSkpK6d+8ul8sN8BgBY5Cfn+/q6oqttcEWCm2CsRs2bJiFhQXWSxfetRiIq6srQujdu3cnT5402EKhTTBehYWFzs7O2dnZX9nO8edLTk4OCAgoKSlxcHDQ97KgTTBSy5cv5/F4CKFGGwOEUEBAAELo6tWrFy+q6apVtyAJRkehUMTGxgYHBzdr1gzvWozC9OnTMzIyuFzuZ4zbcLB1ZFwuXLgwZMgQIpFIpVLxrsW4CIXCxMTEkJAQTY93+I+gTTAi8fHxWVlZNBoNYvAxOp3etWvXwMBAmUymj/lDm2AUeDwem81OT09v2rQp3rUYOw6Ho1Ao7OwacpmdFtAm4O/t27c//PADQghi8DlsbGzy8/OvXbum29lCEvAXHx8fHR2NdxWmpF27ds+fPy8vL9fhPGHrCE/nz58fNWoU3lWYquLiYkdHR13NDdoE3KxevdrDwwPvKkyYo6NjdHR0bGysTuYGbQJu3r17B2cM/rv79++z2ex27dr9x/lAEgxNLBavXbt206ZNeBcC/gdsHRnawoUL169fj3cVX5uZM2empKT8lzlAmwC+EgcOHJg8eTKDwWjY5JAEA6murl6wYEFkZCTehQD1YOvIEGQy2Y4dOyAG+vbkyZO9e/c2bFpoE8BXJSoqysfHp2fPnl86ISRB7xYsWDBnzhxfX1+8CwHawNaRfsXFxY0fPx5iYEh5eXlnz5790qmgTQBfoZ9//rlLly4DBw78/EmgTdAXhUKxcuVKvKtopNavX9+mTZsvmgSSoC9r1qzp0qUL3lU0XjY2NjU1NZ8/Pmwd6YVMJpNKpXQ6He9CGrUuXbrcv3//M28AhCToxZs3b9zd3VksFt6FNGoJCQlSqbRfv36fMzIkQfdSU1O3bt1qyF6rwH8H+wm69+7du8WLF+NdBUAIoUePHr1///5zxoQ2AXzNUlJSdu/eHRUV9ckxoU3QsYyMjKSkJLyrALXatGkzceJEPp//yTEhCToWHR1dVVWFdxXgX0FBQZ9z6AKSoGO9e/cOCQnBuwrwr7y8vO3bt39yNNhPAF+/0NDQ8+fPW1tbaxkH2gRdysvLM0CvzuBLRUVFEQgE7eNAEnTp5cuXqampeFcB6nN1dbX61KNKIQm61LJlywkTJuBdBaivvLx8wYIF2schG6qYRsHLywvvEoAatra2b968KS8vt7W11TQO7DHrUkJCgpWV1X/vhQroXGVlJZ1Op9FomkaAJOhASEgI9jjAmpoaEolkZmaGEKLRaFevXsW7NPC5YD9BB6ytrTkcDofDEQqFfD4few19nhqVe/furVmzRssIkAQdGDVqVL2L4O3s7MaPH49fRaA+d3d37Yf1YOtIBxQKxbhx41TXPCqVym7duu3btw/vusD/0P5od2gTdIBIJIaHh6uaBTs7OziWaoS0P9odkqAbw4cPd3Nzw143a9YMe5AwMCpz5sxJTk7WNBSSoBsEAmHEiBFUKtXW1nbs2LF4lwPUsLOzKy4u1jT0M/YTlEgiUgiq5bov7aszd+5cR0dH6Nzlc1jYkImkT1wLpFtSqZRAIJDJ6s8mfyIJqY95KYlVvAop3VzbNhYAX4TJIpfkCV18GO2CLd2bNbCfd93SdrXFX39WVpZKg0Y5sSzhogyge9WVskfXSsViZZO2TAMsLjEx8datW+vWrVM7VON+wuM4joAr7/aNPcQA6Im5FbnfROfUh1Xpzz59d+V/x2KxSktLNQ1Vv3VUWSp9fJ3TY5jOHvEJgCZKJbp1qiB8rsun7iDQL/VtQnmBGE64AcMgEJCQL68skeh7QUqlUsut/eqTUM2V2bpovGoPAN1y8qJzy6T6XopUKg0NDdU0VH0SZGKFRKTQZ1UA/EvIlyvket8IoVKpNjY2MplM7VDYGwaNyPXr1zUNgnPMoBEpKSmRy9WfI4YkgEZk1qxZ+fn5agdBEkAj4ubmpqlNgP0E0IhoeVoztAmgESkvL5dI1J+4gCSARmTlypUvX75UOwiSABoRW1tbTXeuwX4CaEQ2bNigaRC0CaAREQgEUqn6yzogCaARWb16taYnHjXGJGRmfgjuE/Dq1Qu8CzF2l6+c7xPaCe8qdIlOp2vqPr4x7ifY2tlHzF/m7OyKdyHG6ErshbT0N8uWrEEItfMPiJi/DO+KdGn9+vWaBjXGJLDN2d8MGY53FUYqPf2t6rWXl4+Xlw+u5eiYQqEgEAhqmwWdJSEl5Xnkrwezsj7I5XIfn6ZTJ89p27Y9QmjAoO6TJs4YNbK2a8TtO9Z/+JB25HAMQmjosNBxY7/Pzs5MTLqrkMsHDvx29KgJO3ZteJXynM5gfD9pZv9+YQihteuWIYRatfK/eCmGy6309w9YvnTtmbMn7iT8KZFIQvr0/2HuYmzdbt/588KFU/kFuRQKtWXLNnNm/+ji7Ip9z508dWzRwpU7dm3oGzqoX9/BU6aN3rcn0sen6aCwnvVWZNGPKwcN/BYhdCch/uLFmJzcLDqd0Tu439Qpc7T0tKwSH3/97PnooqICR0fn0aMmDOg/BHs/7o/YCxdjCgvz6XRGYKeus2YusLa2wf4I48dNKSktTrgbLxTWtG7dbtHClTY2tnPnTWbQGdu2HlDNeenyeXx+9cH9UTKZLOb08YS7N0tKiuzsHEYMH4cFOysrY/LUURvX7zoauZ9Oo/9y6GRJSfHhI3tevPynpkbg6Og8fNjYsMHhWG9wJ08du3Pnz7LyUjbbolvXXjOmz6fT6RELp798+Qxbi6NHTr969eLgoZ13bv2NEJJIJMd/PXT33s3KygobG9uQPgMmTZyB9ROhaRV09dHSoeXLl4eGhqp9EJ5u9hOEQuGKlRGeHt4H9kUdOhDt491k2Yp5vGqe9qnIZPKFizHduvaKvXx72rQfLlyMWbZ83tjRk67GJvTrO3jP3i3YHEhkcsqr51VVlTEnYw8diE5OfjJ77iQXF7fzZ+NW/bz5SuyFv58+Rgi9fZe6cdPKwMBuhw+d2rJ5n0goXL2m9vHgFApFJBJevnJu6ZI133wzQlUAnU4/dfKK6mfwoKEMBqNN63YIoaSkexs2/tShQ+Cxo2eXLF79IPHOzt0bP/l3uP/gzrYd6/r3C9u39/jgQUO3bV937/5thNDNm3E7dm7oGzro18jz69ZsT3//bvmK+dh9s2Qy+ez5aE9P77Onf/818sL79+9OxUQihIKD+j5/kay6x4rP5z979nfv4H4IocNH9p6/cGrcmO+PR54fMXzcgYM74v6IxVYTIRR98uiokeMXL1qFENq2fW05p2zTxj2/Hr8QPnT0nr1bniY/QQhd+u3MmbMnJk+effzYuSWLVz98dD/y14MIoQ3rdjVt0qx3cN/Yy7e9vXzrrtqevVtu/Hlt5oyIE1GXpkyecyX2/JGj+1T/R7WrYFp00yaUlhYLBILQkIEeHl4IoblzFgX1CqVSqJ+c0NfXr0uXHgih3sH9du/Z3KJF65Yt22C/noo5np+X06JFa4SQTCabMH4amUz29vb19vKVyqRDwoYhhAI6BFpYWGZkpAd26urm6nH4l1M+3k2wL6rhw8b+9PPCysoKKytrAoEgEomGDxvbObAbtseMLZ1AILi61HZc9/xF8h83rv68cpObmwdC6My5E23btp82dS5CyNXFbdrUHzZt/nnalLn29g5aVufipdPduwWNHjUBIeTXtHlFBYdTXoa9361br3Fjv0cIubl5/DB38eIlc16/ftm6tT9CyMPdC2s67O0dOnXsmpb2BiEU1Cvk4KGdT/5KCunTHyH08OE9hUIRHBTK5/OvXrs4buz3/foNxmp7//7dmbMnBg38FhEICCF//wBVQ5SZ9WHot6OaN2uJEHIZMrxpk2YODk4IoZA+AzoGdPH29kUIubq6Bwf1/evvh9g97yQymUKlWlhY1l2vqiruzVtxM2fM7x3cFyHk4uyam5t16bcz06f9gMVP7SoYoY0bNxKJ6r/9dZMEV1d3NzePjZtXDgkbHhDQuYmvn79/h8+Z0M21tmt17Im5bm6e2K8MBhMhxBfUfiM6OTqrOmxiMJkW7H//TywmSyDgY3MoKiqIjDxQUJAnEotkUilCqLqaZ2VV+8RFLFRqcTjl6zes+PbbkUG9QrCtyfT0t5MmzlCN4N+2A0IoM/O99iTUm2rG9HlYjDMy3wcH91W97+fXAiH0ISMdS4K3dxPVIHNzNtYS2tjYtm3TPinpLpaEB0kJHdp3sra2efnymUwmC+jQWTVJ27Yd4v6Iramp+Xg1u3bpefbcCT6/OjCwW5vW7Zo3b4W9b2FhefNW3I5dG8rLS2UymVBYQ6dr63QoI/O9XC5v0fzfOfv5tRCJRPn5udiOhNpVMEKauv3SWRJIJNK+PZFnz0XHxV05FnnAwcFx8qRZffsO+uSE9Tpbx57BoaLqd4Pyv6PV+xUbLeHuzfUbVoz/bsoPcxczmaxXr19gOxgqTKb6x1PLZLK165c5ObnMmhGBvSMSieRy+YnoIydPHas7JqeiXMu6iEQiqVRKo9HrvS8UCZVKJZZtDIPOQAgJhTVq11q1NxcUFHr4yB6xWCyTyZKTnyyMWIEQqqkRIIQW/DhDtduHrX5FJedqH4BWAAASfklEQVTj1VwQsdzby/fW7T8uXjrNZDKHhA2f/P0sMpm8/8D2W7f/WDB/ectWbc2oZmfPRSfcjdeyathC664C/fNWwdisWbMmODi4V69eHw/S2R6zpaXVrJkRs2ZGZGdnXrgYs3nrag9Pb7+mzevtp0skYl0tsZ64uCvt/AMmfz8L+1UsEn3mhMciD+TmZh89fFr1hUGj0chkcvjQ0dius4qllbYH+tJoNBqNhn1o6qLT6EQise77ghqBlmSq9OrZZ9/+bcnJT0RiEUKoW7cg1VQ/rdhQbzve3s6htKyk3hzIZPKwYWOGDRtTUcG5eSvu+K+HLC2thoWP+ePG1fHfTQ0NHVhbj+AT3Q1hC627CjWftwrGRigU6vccc2FRQVLSPey1p6f3wgUriERidlYG9kXC51erxszIfK+TJX5MIpXU3bq9k/Bn3VZFk6Ske5d+O/PTig11N3uIRGKTJs1KSorc3T2xHycnFxKZzDZna5+br69fSsoz1a/7D+7Yf3AHmUz29Wn66vW/J/LepKaotpG0sLS0at+u45O/kh4+vNc5sDu2Aent3YRCoVRWVqhqY7MtLCws67Wu2E72rds3sBvYra1tRo+a0KJF68zMDwqFQi6Xs9kW2GgCgeDR4wd1/1Af/9G8vZuQSKTXqf9exZmamsJisVz+fy/LVKxatUptg6CzJJSWFK9eu+TCxZjc3Oy8vJxTMZFEIhHbYG3atHnSw3tVVVypVHr6TBSPV6WTJX6sebNWyclP3r59XVxctHvPZmtrW4RQWtobkebGobCoYOu2Nf37hTk5ueQX5GE/HE45Qmj0qAkPEhPOnD2Rl5fz/kPaps0/z5s/RSCo/31fz/BhY58mP4k6cfhd2pvfLp+Ljb3QvFkrhNCIEd89eZJ04WJMcXHR8xfJ+w/uaNu2fbNPJQHbQHqa/Pjp08d9+vTH3mGxWIMHh5+IPpJw92ZhUcHzF8mLlszesk3Nc5MIBMK+/Vt37Nzw/kNaYVHB7Tt/pqe/9ffvQKFQmvj6xd+8XlCYn5HxfsXKiMDAbtXVvNzcbJlMZs4y//Ah7f2HtKoqrmpWFmyLAf2HnD4TlZR0r6SkOD7++tVrF4eFj9Gy2W2cmEwmtov/Md2sib9/h6WLV1+4FBN14jCJRPLw8F6/dgd2EGb2rIXbtq8dPXawuTl74IBv+/Ud/PTpY50stJ5x4yYXFuX/uHgWg8EcPCh8wvipHE7Zjl0biJqfH5H6+iVfwP/jxtU/bvz7aMCePXqvXbOtZ4/eK5avP3vuRNSJw0wmq1Wrtrt3HmEyP9F9Z6+efSLmL7twMebsuWgHB6d5PyzB9ndD+vQXi0UXLsYcizzAZLK6dwuaMWP+56xUjx699+zdQqPROgd2V705e+YCc5b50WP7OJxya2ubrl16Tpk85+NpmUzm1i0HIiMPLPxxhkQicXR0Vp2iWbxo1fYd6yZPGeno6Dz5+1nNm7VKff1y1pwJkcfODR06evOWVfPmT1m7Znvduc37YQmDwdyzbwuXW2lv5/DduCljx0z6nFUwKmvXru3du3ePHj0+HqS+N8i//6wQi5B/sLbNYgB05f7F4mYBLF9/ve91LFq0aNCgQcHBwR8PMrHWDYD/YvXq1fUOc6lAEr5M2DdBmgYtW7K2Wzf1e2PASJibm2saBEn4MkePnNE0yMoSNiaN3c8//xwWFtapk5pLzSEJX8bJ0RnvEkDDVVVVaTqfAEkAjcimTZs0XVAMSQCNCHZ2Uq3GePcmaLTmzp2bmpqqdhAkATQiXC5Xv1dlA2ASDhw4oGkDCZIAGhFLS0tNg2DrCDQi48eP53K5agdBEkAjkpWVpelqC0gCaEQuXrxIp9e/qRCjfj+BSifC45iBwTDZZBLFEF/KTk5OmgapXzzbilKSK9RnSQD8Ky9dYGWv/gYaHSouLp4+fbqmoeqTYO9upqH3SAB0TCZRsq0plnZ6T4KWB+povFMHIfTyQVVemrDXSEd91gYAijua1/1bW9cm6jffdUgikYjFYk0XZmtMAkIo7Z/qN4+r2wRZW9pRqTTYtwa6JOTLeRzp499L+k1wtHdTfzzHkLQlASGUm1bz4h63OFskl8Eu9Kdhf0xN/ZIDFXMrsliocPdjBIRaWznofbsIc+LECblcPmXKFLVDP3GO2d2P4e7HQAjJpZCET9u1a5e7u/vw4dAR9ycoESJTDP19kZ+f37JlS01DP/dqC5LB6zZFSoIcERXwtzJOERERWno7h+uOQGOh5eYEOMesY2w2W9PJfIC7fv36aRkKSdAlHo8nFuur41fwX2RnZ2vp2AKSoGNWVlaaLmsB+HJ2do6KitIyAuwn6JJYLK6oqMC7CqAGlUr9uBPluqBN0CU7OzuT6zS3kVi6dOnjx9o65IUk6JKZmVlWVhbeVQA13rx507x5cy0jwBeYLtnb26ueEQiMyu+//659BGgTdMnV1TU9PR3vKkB9IpGIx/vEo98gCbrk7u5eVFSkqb9BgJe1a9c+efJE+ziQBB0LDQ3NzMzEuwrwP3Jycrp37659HEiCjtnb22s/RgEM78yZMwyGtsfsQhJ0LzAw8K+//sK7CvCv/Pz88nJtjw/GQBJ0rFOnTtgTlPEuBNQaNWqU9mvvMJAE3fPz87t8+TLeVQCEEHr9+vW8efO0XIyt8ol71kADvHv3bv369adPn8a7EPAFoE3QvWbNmrm4uLx+/RrvQhq7srKyM2c0Pg2sHkiCXowePXrv3r14V9HYbd261dn5c58GBknQi/bt25uZmcHhVBwJhcLZs2cHBWl8Vmo9kAR9Wbx4cXR0NN5VNF5mZmZeXl6fPz4kQV88PDy6d+++Z88evAtpjG7cuLFq1aov6m4HkqBH3333XWpq6rNnz/AupNFJSEhYv379F00CR1H1Sy6Xjxw58rfffsO7EPAJ0CboF4lEOnjw4KBBg/AupLH4+++/G/a9A0nQO0dHx82bN3/33Xd4F/L1y8nJiYyMHDZsWAOmha0jAykvL1+6dOnx48fxLgSoB22Cgdja2s6cOXP8+PF4F/LVOn78OIfDafDkkATD6dix4/Lly8eNG4d3IV+hiIiIwMBAGxubBs8Bto4MLTc3d+TIkefPn/fw8MC7FvAvaBMMzd3dPTExccGCBXfu3MG7lq/BsmXLdDIfSAIOKBTK5cuX4+PjT506hXctpu3SpUu6eloFbB3h6ezZs7du3Tp69Cj0nPelxGKxmZlZcXGxo6NuHgUIbQKexowZM3/+/O7duz99+hTvWkxJWVlZcHAwdq5GV/OEJOCsbdu2T548uXLlyo4dO/CuxWQkJCQ8evRIt/OEJBiFTZs2ubq6Dh48GLrQ0+7QoUPYTfo6nzMkwViMHj362LFjq1ev/vwbDhubuXPntmnTRk8zhyQYEScnp7NnzyKEwsPD3717h3c5RgTrQmrdunWf7MquwSAJRmfs2LG7d+/esGFDZGQk3rUYhWnTpmE9kFtbW+tvKZAEY+Th4RETE2NhYREaGvrw4UO8y8ENl8vFrqTo06ePvpcFSTBeI0aMOH/+/Pnz5zdt2lRTU4N3OQZVXFw8YsQIuVyOENLyOHEdgiQYNWtr63379nXu3Llfv37YLkRdISEhONWlMzExMX379v34/Tt37mzduvW/XFH3pSAJJqB3796JiYkFBQUrVqxQdSgWHh7O5XInTZqEd3UNV1ZWdunSpbrd9z558mTJkiUIoXHjxnl7exuyGEiCyVi0aNGUKVN27Nixbt06hUKRn5+PEPrw4YPpXry0bdu2vLw8IpEYEhJSWVmJEEpPT9+0aRMuxUASTImPj8+JEyfatm3bsWNHhUKBPTfp4sWLJSUleJf2xW7fvv3s2TOsIxYul4s9B23ChAl4XYIFV+CZpA4dOtTtzCcoKMi0LtZQKBTh4eFYs4YhkUj4PncC2gTT06NHj3p9Wj1//vzGjRv4VfTFdu7cWVRUVPcduVweFhaGX0WQBFMzYsQIKpVKoVBUjblCoeByuQcPHjSVx5e8ePHi9u3bUqlUtQokEonFYkkkEhyrgq0j03P79m0ej8fn8zkcTkVFBbdczFB6sc08vd1aC6tlDDalokiEd43qmVtT5TIFw5ycnp3MFX2QmRVa21rY29uz2WwGg/Htt9/iWBskwYSlJFW9uF8lrlEwbRgsWzqZQiKbkShmZGSs/1OlEknFMplYLpcpeCV8XlmNux/LvyfbtSkd79IgCaYp/ZkgMbaMYUm3cmXTzKl4l9NwNVxxWWYFnUkIGmZr72aGYyWQBBMjl6NrR4tr+MjOx4pK/0ru+eRzhLySag8/erfBlnjVAEkwMac25po7WVg6ffphkianJJ1jaaXsN8EBl6VDEkzJ2e35Vh42Jr05pB0np8raFgUP1+PV15rAUVSTcXJjrrWn7VccA4SQjYdFBQfdOVdm+EVDEkzD9chiSxdLMxYF70L0zsbdorxE8fJBlYGXC0kwAe+eVguFRLYDE+9CDMShqe3LRB6vwqAnCiEJJiAxttzKHbeDKrhgO1kkxpZ/xog6A0kwds/uctn2TDKVhHchBmXpxCzJlXCKDHf9BSTB2L1+yLM24gZh+/4xl3/fro85W7tZPL9nuL0FSIJRKy+UyOWIQmtcDQLG3JaRmVJtsMVBEoxa5is+w5qBdxX4IFGJZkxKUZaBrib8Sk7Xf604RVJzW3M9zVwul92+H/Xi1a1KbpGlhUPPrmO6dqp9Vt+aLf379PqeW1XyPOWmRFLj5eE/4psVbLYtQigz58WV6ztKS7OsrZwHhMzSU20Ylh2zJEfk5EXT61Iw0CYYtbJ8EYmir//R9fj995NievecuGjumZ5dx1yN2/VX8lVsEJFIvpt4ysHe66cfYxf9cLagKO32/V8RQkIR/8TpxQw6e/6sE2NHrH309Lfqaj0e4SEQCJWlUv3Nvy5IglET8uV6OmokFPEf/XWpV/fvOrYbZGvj1rXTsIB2gxIST6pGcLD37NQ+jEQiW1o4+DXpklfwFiH0Nv1hjZA3dPAiZ8cmbi4tRoevrhHy9FEehmxGruYa6KwCJMF4ySSIbWOmpzahsChdrpA19emkesfHqz2nIl8sru1izMmhiWoQg87GPvElpVkUCs3Rvrb/FUsLewu2vT7Kw1DoZBLJQB9R2E8wXmQqqiwVOciVRBLhM0b/Mtgn/vCvs9G/t0QrEULVfI6ZGQMhRKGouVtALK6hUv5nqx0bWU9kIrlMJNff/OuCJBg1OpMkE8upDN3/m2g0JkJo7Ih1Tg4+dd+3sNB2UTSVQhOJ+HXfEQr1eKBTJpaxLA30EYUkGDWWFUUm0UsSnBybkEgUPr/CvlVt57t8QSVCBApZ27Wu9nYecoWsuDQT20AqKvlQzW/4w8A/SSZRsF0hCQAhe1ezslIRw1L3tzXSaawuHYfG3z3GZFq6ubSo5BZfvbHb0sJ+yne7tEzVrGk3Myoj9vqOgX3nyOXSP279wmLp8V4CEV/k4G6hv/nXBUkwar5tmTkXOMhTL5+GsP7z6TTzuJsHeNXl5iybFn49BoR+4vwAi2k5aey22D92HYycbmXpNDBk9oPH57AdDH2oKq7xbOmsp5nXA/esGbtflmT49fAgknW/02zkeKU1SiH/m5lOhlkcHEU1di27WHKL+J8x4teGXy5o24NtsMXB1pGx6zrY+ujyTGs3jddcHI2el5ufqnaQQi4jktT/i0eHr27VvKeuikx4EF33rFxdBERQath8Wjj7lLWV+o2fGq6YoJB5tjTczUmwdWQCHl3nFOQgO2/112bzeOUyufrr+CVSMVXdaQGEEItpTaXq7HoeobBaKFJ/OLVGWM2gq4+xBduepCGouc+L+oy0cfE1XI9gkATTcGZrnl1TewqtUbThvBI+ky7pM8rOkAuF/QTTMOpH1/SkPLyrMIQarlhQxjNwDKBNMCXcMsnvx8vc2jriXYgeiaollTmcMYtdDb9oaBNMhqUd9Ztp9m8TsqVC0+gd/kvxSgTF70rHLMIhBtAmmB6pWHFyY66Vi6W1u77u4DE8pUJZkVtFJUnCphvo7MHHIAkm6e6F8vcvqu19rC2dTbyDVCUqzawsz67qNsSubU/DnT34GCTBVAmq5Pd+K899J2DbMVh2DJYNnUQ2mW1dmVjBKxXwywUEpPBpw+w6GIeOUOuBJJg2iVCRmSpI+4dfXSnjlorN6CS2PUNUjedjmrSgmJGqK8RSkdzBk2FpS2nanunRjIGM4zoSSMLXQ6FANTxZTbVCIVfgXYt6ZCqRYU5imBtjpzWQBAAQHEUFoBYkAQAESQCgFiQBAARJAKAWJAEAhBD6PxFxsIrUSpvUAAAAAElFTkSuQmCC\n", | |
| "text/plain": [ | |
| "<IPython.core.display.Image object>" | |
| ] | |
| }, | |
| "metadata": {} | |
| } | |
| ], | |
| "source": [ | |
| "from IPython.display import Image, display\n", | |
| "\n", | |
| "from langchain_openai import ChatOpenAI\n", | |
| "from langchain_core.messages import SystemMessage, HumanMessage, RemoveMessage\n", | |
| "from langchain_core.runnables import RunnableConfig\n", | |
| "\n", | |
| "from langgraph.checkpoint.memory import MemorySaver\n", | |
| "from langgraph.graph import StateGraph, START, END\n", | |
| "from langgraph.graph import MessagesState\n", | |
| "\n", | |
| "# LLM\n", | |
| "model = ChatOpenAI(model=\"gpt-4o\", temperature=0)\n", | |
| "\n", | |
| "# State\n", | |
| "class State(MessagesState):\n", | |
| " summary: str\n", | |
| "\n", | |
| "# Define the logic to call the model\n", | |
| "def call_model(state: State, config: RunnableConfig):\n", | |
| "\n", | |
| " # Get summary if it exists\n", | |
| " summary = state.get(\"summary\", \"\")\n", | |
| "\n", | |
| " # If there is summary, then we add it\n", | |
| " if summary:\n", | |
| "\n", | |
| " # Add summary to system message\n", | |
| " system_message = f\"Summary of conversation earlier: {summary}\"\n", | |
| "\n", | |
| " # Append summary to any newer messages\n", | |
| " messages = [SystemMessage(content=system_message)] + state[\"messages\"]\n", | |
| "\n", | |
| " else:\n", | |
| " messages = state[\"messages\"]\n", | |
| "\n", | |
| " response = model.invoke(messages, config)\n", | |
| " return {\"messages\": response}\n", | |
| "\n", | |
| "def summarize_conversation(state: State):\n", | |
| "\n", | |
| " # First, we get any existing summary\n", | |
| " summary = state.get(\"summary\", \"\")\n", | |
| "\n", | |
| " # Create our summarization prompt\n", | |
| " if summary:\n", | |
| "\n", | |
| " # A summary already exists\n", | |
| " summary_message = (\n", | |
| " f\"This is summary of the conversation to date: {summary}\\n\\n\"\n", | |
| " \"Extend the summary by taking into account the new messages above:\"\n", | |
| " )\n", | |
| "\n", | |
| " else:\n", | |
| " summary_message = \"Create a summary of the conversation above:\"\n", | |
| "\n", | |
| " # Add prompt to our history\n", | |
| " messages = state[\"messages\"] + [HumanMessage(content=summary_message)]\n", | |
| " response = model.invoke(messages)\n", | |
| "\n", | |
| " # Delete all but the 2 most recent messages\n", | |
| " delete_messages = [RemoveMessage(id=m.id) for m in state[\"messages\"][:-2]]\n", | |
| " return {\"summary\": response.content, \"messages\": delete_messages}\n", | |
| "\n", | |
| "# Determine whether to end or summarize the conversation\n", | |
| "def should_continue(state: State):\n", | |
| "\n", | |
| " \"\"\"Return the next node to execute.\"\"\"\n", | |
| "\n", | |
| " messages = state[\"messages\"]\n", | |
| "\n", | |
| " # If there are more than six messages, then we summarize the conversation\n", | |
| " if len(messages) > 6:\n", | |
| " return \"summarize_conversation\"\n", | |
| "\n", | |
| " # Otherwise we can just end\n", | |
| " return END\n", | |
| "\n", | |
| "# Define a new graph\n", | |
| "workflow = StateGraph(State)\n", | |
| "workflow.add_node(\"conversation\", call_model)\n", | |
| "workflow.add_node(summarize_conversation)\n", | |
| "\n", | |
| "# Set the entrypoint as conversation\n", | |
| "workflow.add_edge(START, \"conversation\")\n", | |
| "workflow.add_conditional_edges(\"conversation\", should_continue)\n", | |
| "workflow.add_edge(\"summarize_conversation\", END)\n", | |
| "\n", | |
| "# Compile\n", | |
| "memory = MemorySaver()\n", | |
| "graph = workflow.compile(checkpointer=memory)\n", | |
| "display(Image(graph.get_graph().draw_mermaid_png()))" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "id": "f847a787-b301-488c-9b58-cba9f389f55d", | |
| "metadata": { | |
| "id": "f847a787-b301-488c-9b58-cba9f389f55d" | |
| }, | |
| "source": [ | |
| "### Streaming full state\n", | |
| "\n", | |
| "Now, let's talk about ways to [stream our graph state](https://langchain-ai.github.io/langgraph/concepts/low_level/#streaming).\n", | |
| "\n", | |
| "`.stream` and `.astream` are sync and async methods for streaming back results.\n", | |
| "\n", | |
| "LangGraph supports a few [different streaming modes](https://langchain-ai.github.io/langgraph/how-tos/stream-values/) for [graph state](https://langchain-ai.github.io/langgraph/how-tos/stream-values/):\n", | |
| "\n", | |
| "* `values`: This streams the full state of the graph after each node is called.\n", | |
| "* `updates`: This streams updates to the state of the graph after each node is called.\n", | |
| "\n", | |
| "\n", | |
| "\n", | |
| "Let's look at `stream_mode=\"updates\"`.\n", | |
| "\n", | |
| "Because we stream with `updates`, we only see updates to the state after node in the graph is run.\n", | |
| "\n", | |
| "Each `chunk` is a dict with `node_name` as the key and the updated state as the value." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 4, | |
| "id": "9a6f8ae9-f244-40c5-a2da-618b72631b22", | |
| "metadata": { | |
| "id": "9a6f8ae9-f244-40c5-a2da-618b72631b22", | |
| "outputId": "a729229c-9367-4ae5-8b92-c99842a60ec5", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "{'conversation': {'messages': AIMessage(content='Hello Lance! How can I assist you today?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 11, 'prompt_tokens': 11, 'total_tokens': 22, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-2024-08-06', 'system_fingerprint': 'fp_eb9dce56a8', 'finish_reason': 'stop', 'logprobs': None}, id='run-831c8413-e24f-4600-938a-2517fa27e161-0', usage_metadata={'input_tokens': 11, 'output_tokens': 11, 'total_tokens': 22, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})}}\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "# Create a thread\n", | |
| "config = {\"configurable\": {\"thread_id\": \"1\"}}\n", | |
| "\n", | |
| "# Start conversation\n", | |
| "for chunk in graph.stream({\"messages\": [HumanMessage(content=\"hi! I'm Lance\")]}, config, stream_mode=\"updates\"):\n", | |
| " print(chunk)" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "id": "0c4882e9-07dd-4d70-866b-dfc530418cad", | |
| "metadata": { | |
| "id": "0c4882e9-07dd-4d70-866b-dfc530418cad" | |
| }, | |
| "source": [ | |
| "Let's now just print the state update." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 5, | |
| "id": "c859c777-cb12-4682-9108-6b367e597b81", | |
| "metadata": { | |
| "id": "c859c777-cb12-4682-9108-6b367e597b81", | |
| "outputId": "2bd0117d-d3b4-4360-e4a5-20f710a6fed3", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "==================================\u001b[1m Ai Message \u001b[0m==================================\n", | |
| "\n", | |
| "Hi Lance! How's it going? What can I do for you today?\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "# Start conversation\n", | |
| "for chunk in graph.stream({\"messages\": [HumanMessage(content=\"hi! I'm Lance\")]}, config, stream_mode=\"updates\"):\n", | |
| " chunk['conversation'][\"messages\"].pretty_print()" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "id": "583bf219-6358-4d06-ae99-c40f43569fda", | |
| "metadata": { | |
| "id": "583bf219-6358-4d06-ae99-c40f43569fda" | |
| }, | |
| "source": [ | |
| "Now, we can see `stream_mode=\"values\"`.\n", | |
| "\n", | |
| "This is the `full state` of the graph after the `conversation` node is called." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 6, | |
| "id": "6ee763f8-6d1f-491e-8050-fb1439e116df", | |
| "metadata": { | |
| "id": "6ee763f8-6d1f-491e-8050-fb1439e116df", | |
| "outputId": "fcff48a6-7cfc-4575-bb9e-a0805b07dcbf", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "================================\u001b[1m Human Message \u001b[0m=================================\n", | |
| "\n", | |
| "hi! I'm Lance\n", | |
| "---------------------------------------------------------------------------\n", | |
| "================================\u001b[1m Human Message \u001b[0m=================================\n", | |
| "\n", | |
| "hi! I'm Lance\n", | |
| "==================================\u001b[1m Ai Message \u001b[0m==================================\n", | |
| "\n", | |
| "Hello Lance! How can I assist you today?\n", | |
| "---------------------------------------------------------------------------\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "# Start conversation, again\n", | |
| "config = {\"configurable\": {\"thread_id\": \"2\"}}\n", | |
| "\n", | |
| "# Start conversation\n", | |
| "input_message = HumanMessage(content=\"hi! I'm Lance\")\n", | |
| "for event in graph.stream({\"messages\": [input_message]}, config, stream_mode=\"values\"):\n", | |
| " for m in event['messages']:\n", | |
| " m.pretty_print()\n", | |
| " print(\"---\"*25)" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "id": "563c198a-d1a4-4700-b7a7-ff5b8e0b25d7", | |
| "metadata": { | |
| "id": "563c198a-d1a4-4700-b7a7-ff5b8e0b25d7" | |
| }, | |
| "source": [ | |
| "### Streaming tokens\n", | |
| "\n", | |
| "We often want to stream more than graph state.\n", | |
| "\n", | |
| "In particular, with chat model calls it is common to stream the tokens as they are generated.\n", | |
| "\n", | |
| "We can do this [using the `.astream_events` method](https://langchain-ai.github.io/langgraph/how-tos/streaming-from-final-node/#stream-outputs-from-the-final-node), which streams back events as they happen inside nodes!\n", | |
| "\n", | |
| "Each event is a dict with a few keys:\n", | |
| "\n", | |
| "* `event`: This is the type of event that is being emitted.\n", | |
| "* `name`: This is the name of event.\n", | |
| "* `data`: This is the data associated with the event.\n", | |
| "* `metadata`: Contains`langgraph_node`, the node emitting the event.\n", | |
| "\n", | |
| "Let's have a look." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 7, | |
| "id": "6ae8c7a6-c6e7-4cef-ac9f-190d2f4dd763", | |
| "metadata": { | |
| "id": "6ae8c7a6-c6e7-4cef-ac9f-190d2f4dd763", | |
| "outputId": "e0ea9383-b129-42e5-b2fe-174c6f4f1aa0", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "Node: . Type: on_chain_start. Name: LangGraph\n", | |
| "Node: __start__. Type: on_chain_start. Name: __start__\n", | |
| "Node: __start__. Type: on_chain_start. Name: _write\n", | |
| "Node: __start__. Type: on_chain_end. Name: _write\n", | |
| "Node: __start__. Type: on_chain_start. Name: _write\n", | |
| "Node: __start__. Type: on_chain_end. Name: _write\n", | |
| "Node: __start__. Type: on_chain_stream. Name: __start__\n", | |
| "Node: __start__. Type: on_chain_end. Name: __start__\n", | |
| "Node: conversation. Type: on_chain_start. Name: conversation\n", | |
| "Node: . Type: on_chain_stream. Name: LangGraph\n", | |
| "Node: . Type: on_chain_end. Name: LangGraph\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "config = {\"configurable\": {\"thread_id\": \"3\"}}\n", | |
| "input_message = HumanMessage(content=\"Tell me about the 49ers NFL team\")\n", | |
| "output_count = 0\n", | |
| "\n", | |
| "async for event in graph.astream_events({\"messages\": [input_message]}, config, version=\"v2\"):\n", | |
| " if event['metadata'].get('langgraph_node','') != 'conversation':\n", | |
| " print(f\"Node: {event['metadata'].get('langgraph_node','')}. Type: {event['event']}. Name: {event['name']}\")\n", | |
| " else:\n", | |
| " if output_count == 0:\n", | |
| " print(f\"Node: {event['metadata'].get('langgraph_node','')}. Type: {event['event']}. Name: {event['name']}\")\n", | |
| " output_count += 1" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "id": "0b63490f-3d24-4f68-95ca-5320ccb61d2d", | |
| "metadata": { | |
| "id": "0b63490f-3d24-4f68-95ca-5320ccb61d2d" | |
| }, | |
| "source": [ | |
| "The central point is that tokens from chat models within your graph have the `on_chat_model_stream` type.\n", | |
| "\n", | |
| "We can use `event['metadata']['langgraph_node']` to select the node to stream from.\n", | |
| "\n", | |
| "And we can use `event['data']` to get the actual data for each event, which in this case is an `AIMessageChunk`." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 9, | |
| "id": "cc3529f8-3960-4d41-9ed6-373f93183950", | |
| "metadata": { | |
| "id": "cc3529f8-3960-4d41-9ed6-373f93183950", | |
| "outputId": "fa870a5d-96b8-4a75-894c-31ed0f196741", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "{'chunk': AIMessageChunk(content='', additional_kwargs={}, response_metadata={}, id='run-6d5fda99-8641-4f6f-85e7-cdc119f3b4d2')}\n", | |
| "{'chunk': AIMessageChunk(content='The', additional_kwargs={}, response_metadata={}, id='run-6d5fda99-8641-4f6f-85e7-cdc119f3b4d2')}\n", | |
| "{'chunk': AIMessageChunk(content=' San', additional_kwargs={}, response_metadata={}, id='run-6d5fda99-8641-4f6f-85e7-cdc119f3b4d2')}\n", | |
| "{'chunk': AIMessageChunk(content=' Francisco', additional_kwargs={}, response_metadata={}, id='run-6d5fda99-8641-4f6f-85e7-cdc119f3b4d2')}\n", | |
| "{'chunk': AIMessageChunk(content=' ', additional_kwargs={}, response_metadata={}, id='run-6d5fda99-8641-4f6f-85e7-cdc119f3b4d2')}\n", | |
| "{'chunk': AIMessageChunk(content='49', additional_kwargs={}, response_metadata={}, id='run-6d5fda99-8641-4f6f-85e7-cdc119f3b4d2')}\n", | |
| "{'chunk': AIMessageChunk(content='ers', additional_kwargs={}, response_metadata={}, id='run-6d5fda99-8641-4f6f-85e7-cdc119f3b4d2')}\n", | |
| "{'chunk': AIMessageChunk(content=' are', additional_kwargs={}, response_metadata={}, id='run-6d5fda99-8641-4f6f-85e7-cdc119f3b4d2')}\n", | |
| "{'chunk': AIMessageChunk(content=' a', additional_kwargs={}, response_metadata={}, id='run-6d5fda99-8641-4f6f-85e7-cdc119f3b4d2')}\n", | |
| "{'chunk': AIMessageChunk(content=' stor', additional_kwargs={}, response_metadata={}, id='run-6d5fda99-8641-4f6f-85e7-cdc119f3b4d2')}\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "node_to_stream = 'conversation'\n", | |
| "config = {\"configurable\": {\"thread_id\": \"4\"}}\n", | |
| "input_message = HumanMessage(content=\"Tell me about the 49ers NFL team\")\n", | |
| "output_count = 0\n", | |
| "async for event in graph.astream_events({\"messages\": [input_message]}, config, version=\"v2\"):\n", | |
| " # Get chat model tokens from a particular node\n", | |
| " if event[\"event\"] == \"on_chat_model_stream\" and event['metadata'].get('langgraph_node','') == node_to_stream:\n", | |
| " if output_count < 10:\n", | |
| " print(event[\"data\"])\n", | |
| " output_count += 1" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "id": "226e569a-76c3-43d8-8f89-3ae687efde1c", | |
| "metadata": { | |
| "id": "226e569a-76c3-43d8-8f89-3ae687efde1c" | |
| }, | |
| "source": [ | |
| "As you see above, just use the `chunk` key to get the `AIMessageChunk`." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 10, | |
| "id": "3aeae53d-6dcf-40d0-a0c6-c40de492cc83", | |
| "metadata": { | |
| "id": "3aeae53d-6dcf-40d0-a0c6-c40de492cc83", | |
| "outputId": "248df5ec-4ead-44e9-b2d4-634882eca660", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| } | |
| }, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "name": "stdout", | |
| "text": [ | |
| "|The| San| Francisco| |49|ers| are| a| professional| American| football| team| based| in| the| San| Francisco| Bay| Area|.| They| compete| in| the| National| Football| League| (|NFL|)| as| a| member| of| the| league|'s| National| Football| Conference| (|N|FC|)| West| division|.| The| team| was| founded| in| |194|6| as| a| charter| member| of| the| All|-Amer|ica| Football| Conference| (|AA|FC|)| and| joined| the| NFL| in| |194|9| when| the| leagues| merged|.\n", | |
| "\n", | |
| "|###| Key| Points|:\n", | |
| "\n", | |
| "|-| **|Team| Name| and| Colors|**|:| The| team| is| named| after| the| prospect|ors| who| arrived| in| Northern| California| during| the| |184|9| Gold| Rush|.| The| team's| colors| are| red|,| gold|,| and| white|.\n", | |
| "\n", | |
| "|-| **|St|adium|**|:| The| |49|ers| play| their| home| games| at| Levi|'s| Stadium| in| Santa| Clara|,| California|,| which| they| moved| to| in| |201|4|.| Before| that|,| they| played| at| Cand|lestick| Park| in| San| Francisco|.\n", | |
| "\n", | |
| "|-| **|Champ|ionship|s| and| Success|**|:| The| |49|ers| are| one| of| the| most| successful| teams| in| NFL| history|.| They| have| won| five| Super| Bowl| championships| (|Super| Bowl| XVI|,| XIX|,| XX|III|,| XX|IV|,| and| XX|IX|)| and| have| appeared| in| the| Super| Bowl| multiple| times|.| The| team| has| also| won| numerous| division| titles| and| conference| championships|.\n", | |
| "\n", | |
| "|-| **|Not|able| Figures|**|:| The| |49|ers| have| had| several| Hall| of| Fame| players| and| coaches|,| including| Joe| Montana|,| Jerry| Rice|,| Steve| Young|,| Ronnie| L|ott|,| and| coach| Bill| Walsh|.| These| individuals| were| instrumental| in| the| team's| success|,| particularly| during| the| |198|0|s| and| |199|0|s|.\n", | |
| "\n", | |
| "|-| **|R|ival|ries|**|:| The| |49|ers| have| several| notable| rival|ries|,| including| with| the| Dallas| Cowboys|,| Los| Angeles| Rams|,| and| Seattle| Seahawks|.| These| rival|ries| have| often| been| highlighted| by| significant| and| competitive| games|.\n", | |
| "\n", | |
| "|-| **|Recent| Performance|**|:| In| recent| years|,| the| |49|ers| have| experienced| both| ups| and| downs|.| They| reached| the| Super| Bowl| in| the| |201|9| season| but| lost| to| the| Kansas| City| Chiefs|.| The| team| has| been| working| to| maintain| a| competitive| roster| and| return| to| championship| contention|.\n", | |
| "\n", | |
| "|-| **|Community| and| Culture|**|:| The| |49|ers| have| a| strong| fan| base| and| are| known| for| their| community| involvement| and| charitable| activities| in| the| Bay| Area|.| The| team| has| a| rich| history| and| a| tradition| of| excellence| that| continues| to| be| a| source| of| pride| for| its| fans|.\n", | |
| "\n", | |
| "|Overall|,| the| San| Francisco| |49|ers| are| a| stor|ied| franchise| with| a| legacy| of| success| and| a| bright| future| as| they| continue| to| build| on| their| historic| achievements|.||" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "config = {\"configurable\": {\"thread_id\": \"5\"}}\n", | |
| "input_message = HumanMessage(content=\"Tell me about the 49ers NFL team\")\n", | |
| "async for event in graph.astream_events({\"messages\": [input_message]}, config, version=\"v2\"):\n", | |
| " # Get chat model tokens from a particular node\n", | |
| " if event[\"event\"] == \"on_chat_model_stream\" and event['metadata'].get('langgraph_node','') == node_to_stream:\n", | |
| " data = event[\"data\"]\n", | |
| " print(data[\"chunk\"].content, end=\"|\")" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "id": "1ae885f8-102f-448a-9d68-8ded8d2bbd18", | |
| "metadata": { | |
| "id": "1ae885f8-102f-448a-9d68-8ded8d2bbd18" | |
| }, | |
| "outputs": [], | |
| "source": [] | |
| } | |
| ], | |
| "metadata": { | |
| "kernelspec": { | |
| "display_name": "Python 3 (ipykernel)", | |
| "language": "python", | |
| "name": "python3" | |
| }, | |
| "language_info": { | |
| "codemirror_mode": { | |
| "name": "ipython", | |
| "version": 3 | |
| }, | |
| "file_extension": ".py", | |
| "mimetype": "text/x-python", | |
| "name": "python", | |
| "nbconvert_exporter": "python", | |
| "pygments_lexer": "ipython3", | |
| "version": "3.11.8" | |
| }, | |
| "colab": { | |
| "provenance": [] | |
| } | |
| }, | |
| "nbformat": 4, | |
| "nbformat_minor": 5 | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment