Skip to content

Instantly share code, notes, and snippets.

@DahlitzFlorian

DahlitzFlorian/pegasus_abs_summarizer.ipynb

Created July 28, 2020 20:12
Show Gist options
  • Save DahlitzFlorian/2a37fb152005e6e3bf7aa5603e2d4b3a to your computer and use it in GitHub Desktop.
Save DahlitzFlorian/2a37fb152005e6e3bf7aa5603e2d4b3a to your computer and use it in GitHub Desktop.
Download ZIP
Abstractive-based Text Summarization Using PEGASUS
Display the source blob
Display the rendered blob
Raw
pegasus_abs_summarizer.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "Untitled3.ipynb",
"provenance": [],
"collapsed_sections": []
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
}
},
"cells": [
{
"cell_type": "code",
"metadata": {
"id": "Nrx4Y69PmtSy",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 850
},
"outputId": "f21da2a7-d716-46ed-d651-3e95939255de"
},
"source": [
"!curl https://sdk.cloud.google.com | bash\n",
"!gcloud init"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"text": [
" % Total % Received % Xferd Average Speed Time Time Time Current\n",
" Dload Upload Total Spent Left Speed\n",
"\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 443 100 443 0 0 17038 0 --:--:-- --:--:-- --:--:-- 17038\n",
"Downloading Google Cloud SDK install script: https://dl.google.com/dl/cloudsdk/channels/rapid/install_google_cloud_sdk.bash\n",
"\r######################################################################## 100.0%\n",
"Running install script from: /tmp/tmp.dIuKNT2snA/install_google_cloud_sdk.bash\n",
"which curl\n",
"curl -# -f https://dl.google.com/dl/cloudsdk/channels/rapid/google-cloud-sdk.tar.gz\n",
"\r######################################################## 79.1%\r######################################################################## 100.0%\n",
"\n",
"mkdir -p /root\n",
"\"/root/google-cloud-sdk\" already exists and may contain out of date files.\n",
"Remove /root/google-cloud-sdk or select a new installation directory, then run again.\n",
"Welcome! This command will take you through the configuration of gcloud.\n",
"\n",
"Settings from your current configuration [default] are:\n",
"component_manager:\n",
" disable_update_check: 'True'\n",
"compute:\n",
" gce_metadata_read_timeout_sec: '0'\n",
"core:\n",
" account: [email protected]\n",
"\n",
"Pick configuration to use:\n",
" [1] Re-initialize this configuration [default] with new settings \n",
" [2] Create a new configuration\n",
"Please enter your numeric choice: 1\n",
"\n",
"Your current configuration has been set to: [default]\n",
"\n",
"You can skip diagnostics next time by using the following flag:\n",
" gcloud init --skip-diagnostics\n",
"\n",
"Network diagnostic detects and fixes local network connection issues.\n",
"Reachability Check passed.\n",
"Network diagnostic passed (1/1 checks passed).\n",
"\n",
"Choose the account you would like to use to perform operations for \n",
"this configuration:\n",
" [1] [email protected]\n",
" [2] Log in with a new account\n",
"Please enter your numeric choice: 1\n",
"\n",
"You are logged in as: [[email protected]].\n",
"\n",
"This account has no projects.\n",
"\n",
"Would you like to create one? (Y/n)? n\n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "B4vM6-iznLP1",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "51e3d110-346f-41e9-afcf-a1135f12cf75"
},
"source": [
"!git clone https://github.com/google-research/pegasus\n",
"%cd pegasus\n",
"!export PYTHONPATH=.\n",
"!pip install -r requirements.txt"
],
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"text": [
"fatal: destination path 'pegasus' already exists and is not an empty directory.\n",
"/content/pegasus\n",
"Requirement already satisfied: absl-py in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 1)) (0.9.0)\n",
"Requirement already satisfied: mock in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 2)) (4.0.2)\n",
"Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 3)) (1.18.5)\n",
"Requirement already satisfied: rouge-score in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 4)) (0.0.4)\n",
"Requirement already satisfied: sacrebleu in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 5)) (1.4.12)\n",
"Requirement already satisfied: sentencepiece in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 6)) (0.1.91)\n",
"Requirement already satisfied: tensorflow-text==1.15.0rc0 in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 7)) (1.15.0rc0)\n",
"Requirement already satisfied: tensor2tensor==1.15.0 in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 8)) (1.15.0)\n",
"Requirement already satisfied: tfds-nightly in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 9)) (3.2.1.dev202007280106)\n",
"Requirement already satisfied: tensorflow-gpu==1.15.2 in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 10)) (1.15.2)\n",
"Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from absl-py->-r requirements.txt (line 1)) (1.15.0)\n",
"Requirement already satisfied: nltk in /usr/local/lib/python3.6/dist-packages (from rouge-score->-r requirements.txt (line 4)) (3.2.5)\n",
"Requirement already satisfied: mecab-python3==0.996.5 in /usr/local/lib/python3.6/dist-packages (from sacrebleu->-r requirements.txt (line 5)) (0.996.5)\n",
"Requirement already satisfied: portalocker in /usr/local/lib/python3.6/dist-packages (from sacrebleu->-r requirements.txt (line 5)) (1.7.1)\n",
"Requirement already satisfied: tensorflow<1.16,>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-text==1.15.0rc0->-r requirements.txt (line 7)) (1.15.3)\n",
"Requirement already satisfied: bz2file in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.98)\n",
"Requirement already satisfied: tensorflow-gan in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (2.0.0)\n",
"Requirement already satisfied: kfac in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.2.0)\n",
"Requirement already satisfied: flask in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.1.2)\n",
"Requirement already satisfied: gunicorn in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (20.0.4)\n",
"Requirement already satisfied: tensorflow-probability==0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.7.0)\n",
"Requirement already satisfied: gevent in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (20.6.2)\n",
"Requirement already satisfied: dopamine-rl in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.0.5)\n",
"Requirement already satisfied: pypng in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.0.20)\n",
"Requirement already satisfied: gin-config in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.3.0)\n",
"Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.4.1)\n",
"Requirement already satisfied: sympy in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.1.1)\n",
"Requirement already satisfied: opencv-python in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (4.1.2.30)\n",
"Requirement already satisfied: google-api-python-client in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.7.12)\n",
"Requirement already satisfied: mesh-tensorflow in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.1.16)\n",
"Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (4.41.1)\n",
"Requirement already satisfied: oauth2client in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (4.1.3)\n",
"Requirement already satisfied: Pillow in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (7.0.0)\n",
"Requirement already satisfied: gym in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.17.2)\n",
"Requirement already satisfied: tensorflow-datasets in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (2.1.0)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (2.23.0)\n",
"Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.16.0)\n",
"Requirement already satisfied: tf-slim in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.1.0)\n",
"Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->-r requirements.txt (line 8)) (2.10.0)\n",
"Requirement already satisfied: attrs>=18.1.0 in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->-r requirements.txt (line 9)) (19.3.0)\n",
"Requirement already satisfied: wrapt in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->-r requirements.txt (line 9)) (1.12.1)\n",
"Requirement already satisfied: dill in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->-r requirements.txt (line 9)) (0.3.2)\n",
"Requirement already satisfied: tensorflow-metadata in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->-r requirements.txt (line 9)) (0.22.2)\n",
"Requirement already satisfied: promise in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->-r requirements.txt (line 9)) (2.3)\n",
"Requirement already satisfied: termcolor in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->-r requirements.txt (line 9)) (1.1.0)\n",
"Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->-r requirements.txt (line 9)) (3.12.2)\n",
"Requirement already satisfied: gast==0.2.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (0.2.2)\n",
"Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (1.1.2)\n",
"Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (1.30.0)\n",
"Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (0.2.0)\n",
"Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (3.3.0)\n",
"Requirement already satisfied: tensorflow-estimator==1.15.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (1.15.1)\n",
"Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (1.0.8)\n",
"Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (1.15.0)\n",
"Requirement already satisfied: wheel>=0.26; python_version >= \"3\" in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (0.34.2)\n",
"Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (0.8.1)\n",
"Requirement already satisfied: tensorflow-hub>=0.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gan->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.8.0)\n",
"Requirement already satisfied: click>=5.1 in /usr/local/lib/python3.6/dist-packages (from flask->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (7.1.2)\n",
"Requirement already satisfied: Werkzeug>=0.15 in /usr/local/lib/python3.6/dist-packages (from flask->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.0.1)\n",
"Requirement already satisfied: Jinja2>=2.10.1 in /usr/local/lib/python3.6/dist-packages (from flask->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (2.11.2)\n",
"Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python3.6/dist-packages (from flask->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.1.0)\n",
"Requirement already satisfied: setuptools>=3.0 in /usr/local/lib/python3.6/dist-packages (from gunicorn->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (49.1.0)\n",
"Requirement already satisfied: cloudpickle>=0.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-probability==0.7.0->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.3.0)\n",
"Requirement already satisfied: decorator in /usr/local/lib/python3.6/dist-packages (from tensorflow-probability==0.7.0->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (4.4.2)\n",
"Requirement already satisfied: zope.interface in /usr/local/lib/python3.6/dist-packages (from gevent->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (5.1.0)\n",
"Requirement already satisfied: zope.event in /usr/local/lib/python3.6/dist-packages (from gevent->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (4.4)\n",
"Requirement already satisfied: greenlet>=0.4.16; platform_python_implementation == \"CPython\" in /usr/local/lib/python3.6/dist-packages (from gevent->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.4.16)\n",
"Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.1.0)\n",
"Requirement already satisfied: google-auth>=1.4.1 in /usr/local/lib/python3.6/dist-packages (from google-api-python-client->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.17.2)\n",
"Requirement already satisfied: google-auth-httplib2>=0.0.3 in /usr/local/lib/python3.6/dist-packages (from google-api-python-client->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.0.4)\n",
"Requirement already satisfied: uritemplate<4dev,>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from google-api-python-client->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (3.0.1)\n",
"Requirement already satisfied: httplib2<1dev,>=0.17.0 in /usr/local/lib/python3.6/dist-packages (from google-api-python-client->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.17.4)\n",
"Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python3.6/dist-packages (from oauth2client->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.4.8)\n",
"Requirement already satisfied: rsa>=3.1.4 in /usr/local/lib/python3.6/dist-packages (from oauth2client->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (4.6)\n",
"Requirement already satisfied: pyasn1-modules>=0.0.5 in /usr/local/lib/python3.6/dist-packages (from oauth2client->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (0.2.8)\n",
"Requirement already satisfied: pyglet<=1.5.0,>=1.4.0 in /usr/local/lib/python3.6/dist-packages (from gym->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.5.0)\n",
"Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (2.10)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (2020.6.20)\n",
"Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.24.3)\n",
"Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (3.0.4)\n",
"Requirement already satisfied: googleapis-common-protos in /usr/local/lib/python3.6/dist-packages (from tensorflow-metadata->tfds-nightly->-r requirements.txt (line 9)) (1.52.0)\n",
"Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (3.2.2)\n",
"Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from Jinja2>=2.10.1->flask->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (1.1.1)\n",
"Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.4.1->google-api-python-client->tensor2tensor==1.15.0->-r requirements.txt (line 8)) (4.1.1)\n",
"Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (1.7.0)\n",
"Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.2->-r requirements.txt (line 10)) (3.1.0)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "3ZyJooKZoFyw",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 986
},
"outputId": "6a13cd94-eeaa-4fc9-d26d-d660e7755204"
},
"source": [
"!mkdir ckpt\n",
"!gsutil cp -r gs://pegasus_ckpt/ ckpt/"
],
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"text": [
"mkdir: cannot create directory ‘ckpt’: File exists\n",
"Copying gs://pegasus_ckpt/c4.unigram.newline.10pct.96000.model...\n",
"Copying gs://pegasus_ckpt/c4.unigram.newline.10pct.96000.vocab...\n",
"Copying gs://pegasus_ckpt/checkpoint...\n",
"Copying gs://pegasus_ckpt/model.ckpt-1500000.data-00000-of-00001...\n",
"/ [4 files][ 2.1 GiB/ 2.1 GiB] 53.8 MiB/s \n",
"==> NOTE: You are performing a sequence of gsutil operations that may\n",
"run significantly faster if you instead use gsutil -m cp ... Please\n",
"see the -m section under \"gsutil help options\" for further information\n",
"about when gsutil -m can be advantageous.\n",
"\n",
"Copying gs://pegasus_ckpt/model.ckpt-1500000.index...\n",
"Copying gs://pegasus_ckpt/model.ckpt-1500000.meta...\n",
"Copying gs://pegasus_ckpt/aeslc/model.ckpt-32000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/aeslc/model.ckpt-32000.index...\n",
"Copying gs://pegasus_ckpt/aeslc/model.ckpt-32000.meta...\n",
"Copying gs://pegasus_ckpt/arxiv/model.ckpt-340000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/arxiv/model.ckpt-340000.index...\n",
"Copying gs://pegasus_ckpt/arxiv/model.ckpt-340000.meta...\n",
"Copying gs://pegasus_ckpt/big_patent/model.ckpt-480000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/big_patent/model.ckpt-480000.index...\n",
"Copying gs://pegasus_ckpt/big_patent/model.ckpt-480000.meta...\n",
"Copying gs://pegasus_ckpt/billsum/model.ckpt-180000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/billsum/model.ckpt-180000.index...\n",
"Copying gs://pegasus_ckpt/billsum/model.ckpt-180000.meta...\n",
"Copying gs://pegasus_ckpt/cnn_dailymail/model.ckpt-210000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/cnn_dailymail/model.ckpt-210000.index...\n",
"Copying gs://pegasus_ckpt/cnn_dailymail/model.ckpt-210000.meta...\n",
"Copying gs://pegasus_ckpt/gigaword/model.ckpt-300000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/gigaword/model.ckpt-300000.index...\n",
"Copying gs://pegasus_ckpt/gigaword/model.ckpt-300000.meta...\n",
"Copying gs://pegasus_ckpt/multi_news/model.ckpt-60000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/multi_news/model.ckpt-60000.index...\n",
"Copying gs://pegasus_ckpt/multi_news/model.ckpt-60000.meta...\n",
"Copying gs://pegasus_ckpt/newsroom/model.ckpt-190000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/newsroom/model.ckpt-190000.index...\n",
"Copying gs://pegasus_ckpt/newsroom/model.ckpt-190000.meta...\n",
"Copying gs://pegasus_ckpt/pubmed/model.ckpt-360000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/pubmed/model.ckpt-360000.index...\n",
"Copying gs://pegasus_ckpt/pubmed/model.ckpt-360000.meta...\n",
"Copying gs://pegasus_ckpt/reddit_tifu/model.ckpt-8000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/reddit_tifu/model.ckpt-8000.index...\n",
"Copying gs://pegasus_ckpt/reddit_tifu/model.ckpt-8000.meta...\n",
"Copying gs://pegasus_ckpt/wikihow/model.ckpt-180000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/wikihow/model.ckpt-180000.index...\n",
"Copying gs://pegasus_ckpt/wikihow/model.ckpt-180000.meta...\n",
"Copying gs://pegasus_ckpt/xsum/model.ckpt-30000.data-00000-of-00001...\n",
"Copying gs://pegasus_ckpt/xsum/model.ckpt-30000.index...\n",
"Copying gs://pegasus_ckpt/xsum/model.ckpt-30000.meta...\n",
"- [42 files][ 28.0 GiB/ 28.0 GiB] 52.4 MiB/s \n",
"==> NOTE: You are performing a sequence of gsutil operations that may\n",
"run significantly faster if you instead use gsutil -m cp ... Please\n",
"see the -m section under \"gsutil help options\" for further information\n",
"about when gsutil -m can be advantageous.\n",
"\n",
"\n",
"Operation completed over 42 objects/28.0 GiB. \n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "tNlFF86ltykf",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 238
},
"outputId": "22eb9fc6-861e-468b-8341-4f9994d7a3de"
},
"source": [
"!mkdir /content/data\n",
"%cd /content/data\n",
"!wget -O article.txt https://pastebin.com/raw/JumkZCTB"
],
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"text": [
"mkdir: cannot create directory ‘/content/data’: File exists\n",
"/content/data\n",
"--2020-07-28 16:42:16-- https://pastebin.com/raw/JumkZCTB\n",
"Resolving pastebin.com (pastebin.com)... 104.23.99.190, 104.23.98.190, 2606:4700:10::6817:63be, ...\n",
"Connecting to pastebin.com (pastebin.com)|104.23.99.190|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: unspecified [text/plain]\n",
"Saving to: ‘article.txt’\n",
"\n",
"article.txt [ <=> ] 2.80K --.-KB/s in 0s \n",
"\n",
"2020-07-28 16:42:17 (17.1 MB/s) - ‘article.txt’ saved [2869]\n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "cyuqQcVlt6_G",
"colab_type": "code",
"colab": {}
},
"source": [
"with open(\"/content/data/article.txt\") as f:\n",
" text = f.read()"
],
"execution_count": 5,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "CS9sSvcBuMF7",
"colab_type": "code",
"colab": {}
},
"source": [
"!mkdir -p /content/pegasus/data/testdata"
],
"execution_count": 6,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "SA-TQLxQopUX",
"colab_type": "code",
"colab": {}
},
"source": [
"import pandas as pd\n",
"import tensorflow as tf\n",
"\n",
"save_path = \"/content/pegasus/data/testdata/test_pattern_1.tfrecord\"\n",
"\n",
"input_dict = dict(\n",
" inputs=[\n",
" text\n",
" ],\n",
" targets=[\n",
" \"\"\n",
" ]\n",
")\n",
"\n",
"data = pd.DataFrame(input_dict)\n",
"\n",
"with tf.io.TFRecordWriter(save_path) as writer:\n",
" for row in data.values:\n",
" inputs, targets = row[:-1], row[-1]\n",
" example = tf.train.Example(\n",
" features=tf.train.Features(\n",
" feature={\n",
" \"inputs\": tf.train.Feature(bytes_list=tf.train.BytesList(value=[inputs[0].encode('utf-8')])),\n",
" \"targets\": tf.train.Feature(bytes_list=tf.train.BytesList(value=[targets.encode('utf-8')])),\n",
" }\n",
" )\n",
" )\n",
" writer.write(example.SerializeToString())"
],
"execution_count": 7,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "cHJvIc3Tpvgm",
"colab_type": "code",
"colab": {}
},
"source": [
"registry_entry = '''\n",
"@registry.register(\"test_transformer\")\n",
"def test_transformer(param_overrides):\n",
" return transformer_params(\n",
" {\n",
" \"train_pattern\": \"tfrecord:/content/pegasus/data/testdata/test_pattern_1.tfrecord\",\n",
" \"dev_pattern\": \"tfrecord:/content/pegasus/data/testdata/test_pattern_1.tfrecord\",\n",
" \"test_pattern\": \"tfrecord:/content/pegasus/data/testdata/test_pattern_1.tfrecord\",\n",
" \"max_input_len\": 1024,\n",
" \"max_output_len\": 256,\n",
" \"train_steps\": 180000,\n",
" \"learning_rate\": 0.0001,\n",
" \"batch_size\": 8,\n",
" }, param_overrides)'''\n"
],
"execution_count": 8,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "hewS-08dB_hC",
"colab_type": "code",
"colab": {}
},
"source": [
"public_params_file = \"/content/pegasus/pegasus/params/public_params.py\"\n",
"with open(public_params_file) as f:\n",
" content = f.read()\n",
"\n",
"if \"test_transformer\" not in content:\n",
" with open(public_params_file, \"a\") as f:\n",
" f.write(\"\\n\")\n",
" f.write(registry_entry)\n",
" f.write(\"\\n\")"
],
"execution_count": 9,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "aVZ6owHhvkcg",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "4bf5193b-a35c-4679-aa26-457df5ff7eef"
},
"source": [
"%cd /content/pegasus/\n",
"!pip install -e ."
],
"execution_count": 10,
"outputs": [
{
"output_type": "stream",
"text": [
"/content/pegasus\n",
"Obtaining file:///content/pegasus\n",
"Requirement already satisfied: absl-py in /usr/local/lib/python3.6/dist-packages (from pegasus==0.0.1) (0.9.0)\n",
"Requirement already satisfied: mock in /usr/local/lib/python3.6/dist-packages (from pegasus==0.0.1) (4.0.2)\n",
"Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from pegasus==0.0.1) (1.18.5)\n",
"Requirement already satisfied: rouge-score in /usr/local/lib/python3.6/dist-packages (from pegasus==0.0.1) (0.0.4)\n",
"Requirement already satisfied: sacrebleu in /usr/local/lib/python3.6/dist-packages (from pegasus==0.0.1) (1.4.12)\n",
"Requirement already satisfied: sentencepiece in /usr/local/lib/python3.6/dist-packages (from pegasus==0.0.1) (0.1.91)\n",
"Requirement already satisfied: tensorflow-text==1.15.0rc0 in /usr/local/lib/python3.6/dist-packages (from pegasus==0.0.1) (1.15.0rc0)\n",
"Requirement already satisfied: tfds-nightly in /usr/local/lib/python3.6/dist-packages (from pegasus==0.0.1) (3.2.1.dev202007280106)\n",
"Requirement already satisfied: tensor2tensor==1.15.0 in /usr/local/lib/python3.6/dist-packages (from pegasus==0.0.1) (1.15.0)\n",
"Requirement already satisfied: tensorflow-gpu==1.15.2 in /usr/local/lib/python3.6/dist-packages (from pegasus==0.0.1) (1.15.2)\n",
"Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from absl-py->pegasus==0.0.1) (1.15.0)\n",
"Requirement already satisfied: nltk in /usr/local/lib/python3.6/dist-packages (from rouge-score->pegasus==0.0.1) (3.2.5)\n",
"Requirement already satisfied: portalocker in /usr/local/lib/python3.6/dist-packages (from sacrebleu->pegasus==0.0.1) (1.7.1)\n",
"Requirement already satisfied: mecab-python3==0.996.5 in /usr/local/lib/python3.6/dist-packages (from sacrebleu->pegasus==0.0.1) (0.996.5)\n",
"Requirement already satisfied: tensorflow<1.16,>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-text==1.15.0rc0->pegasus==0.0.1) (1.15.3)\n",
"Requirement already satisfied: tensorflow-metadata in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->pegasus==0.0.1) (0.22.2)\n",
"Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->pegasus==0.0.1) (0.16.0)\n",
"Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->pegasus==0.0.1) (3.12.2)\n",
"Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->pegasus==0.0.1) (4.41.1)\n",
"Requirement already satisfied: wrapt in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->pegasus==0.0.1) (1.12.1)\n",
"Requirement already satisfied: attrs>=18.1.0 in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->pegasus==0.0.1) (19.3.0)\n",
"Requirement already satisfied: dill in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->pegasus==0.0.1) (0.3.2)\n",
"Requirement already satisfied: termcolor in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->pegasus==0.0.1) (1.1.0)\n",
"Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->pegasus==0.0.1) (2.23.0)\n",
"Requirement already satisfied: promise in /usr/local/lib/python3.6/dist-packages (from tfds-nightly->pegasus==0.0.1) (2.3)\n",
"Requirement already satisfied: dopamine-rl in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (1.0.5)\n",
"Requirement already satisfied: gym in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (0.17.2)\n",
"Requirement already satisfied: Pillow in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (7.0.0)\n",
"Requirement already satisfied: gevent in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (20.6.2)\n",
"Requirement already satisfied: tf-slim in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (1.1.0)\n",
"Requirement already satisfied: bz2file in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (0.98)\n",
"Requirement already satisfied: google-api-python-client in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (1.7.12)\n",
"Requirement already satisfied: oauth2client in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (4.1.3)\n",
"Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (1.4.1)\n",
"Requirement already satisfied: tensorflow-gan in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (2.0.0)\n",
"Requirement already satisfied: gin-config in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (0.3.0)\n",
"Requirement already satisfied: opencv-python in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (4.1.2.30)\n",
"Requirement already satisfied: pypng in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (0.0.20)\n",
"Requirement already satisfied: mesh-tensorflow in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (0.1.16)\n",
"Requirement already satisfied: kfac in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (0.2.0)\n",
"Requirement already satisfied: tensorflow-probability==0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (0.7.0)\n",
"Requirement already satisfied: gunicorn in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (20.0.4)\n",
"Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (2.10.0)\n",
"Requirement already satisfied: flask in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (1.1.2)\n",
"Requirement already satisfied: tensorflow-datasets in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (2.1.0)\n",
"Requirement already satisfied: sympy in /usr/local/lib/python3.6/dist-packages (from tensor2tensor==1.15.0->pegasus==0.0.1) (1.1.1)\n",
"Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->pegasus==0.0.1) (1.30.0)\n",
"Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->pegasus==0.0.1) (0.8.1)\n",
"Requirement already satisfied: wheel>=0.26; python_version >= \"3\" in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->pegasus==0.0.1) (0.34.2)\n",
"Requirement already satisfied: gast==0.2.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->pegasus==0.0.1) (0.2.2)\n",
"Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->pegasus==0.0.1) (1.1.2)\n",
"Requirement already satisfied: tensorflow-estimator==1.15.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->pegasus==0.0.1) (1.15.1)\n",
"Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->pegasus==0.0.1) (1.0.8)\n",
"Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->pegasus==0.0.1) (0.2.0)\n",
"Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->pegasus==0.0.1) (1.15.0)\n",
"Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==1.15.2->pegasus==0.0.1) (3.3.0)\n",
"Requirement already satisfied: googleapis-common-protos in /usr/local/lib/python3.6/dist-packages (from tensorflow-metadata->tfds-nightly->pegasus==0.0.1) (1.52.0)\n",
"Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.6.1->tfds-nightly->pegasus==0.0.1) (49.1.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19.0->tfds-nightly->pegasus==0.0.1) (2020.6.20)\n",
"Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19.0->tfds-nightly->pegasus==0.0.1) (3.0.4)\n",
"Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19.0->tfds-nightly->pegasus==0.0.1) (1.24.3)\n",
"Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19.0->tfds-nightly->pegasus==0.0.1) (2.10)\n",
"Requirement already satisfied: cloudpickle<1.4.0,>=1.2.0 in /usr/local/lib/python3.6/dist-packages (from gym->tensor2tensor==1.15.0->pegasus==0.0.1) (1.3.0)\n",
"Requirement already satisfied: pyglet<=1.5.0,>=1.4.0 in /usr/local/lib/python3.6/dist-packages (from gym->tensor2tensor==1.15.0->pegasus==0.0.1) (1.5.0)\n",
"Requirement already satisfied: zope.interface in /usr/local/lib/python3.6/dist-packages (from gevent->tensor2tensor==1.15.0->pegasus==0.0.1) (5.1.0)\n",
"Requirement already satisfied: greenlet>=0.4.16; platform_python_implementation == \"CPython\" in /usr/local/lib/python3.6/dist-packages (from gevent->tensor2tensor==1.15.0->pegasus==0.0.1) (0.4.16)\n",
"Requirement already satisfied: zope.event in /usr/local/lib/python3.6/dist-packages (from gevent->tensor2tensor==1.15.0->pegasus==0.0.1) (4.4)\n",
"Requirement already satisfied: google-auth-httplib2>=0.0.3 in /usr/local/lib/python3.6/dist-packages (from google-api-python-client->tensor2tensor==1.15.0->pegasus==0.0.1) (0.0.4)\n",
"Requirement already satisfied: httplib2<1dev,>=0.17.0 in /usr/local/lib/python3.6/dist-packages (from google-api-python-client->tensor2tensor==1.15.0->pegasus==0.0.1) (0.17.4)\n",
"Requirement already satisfied: google-auth>=1.4.1 in /usr/local/lib/python3.6/dist-packages (from google-api-python-client->tensor2tensor==1.15.0->pegasus==0.0.1) (1.17.2)\n",
"Requirement already satisfied: uritemplate<4dev,>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from google-api-python-client->tensor2tensor==1.15.0->pegasus==0.0.1) (3.0.1)\n",
"Requirement already satisfied: rsa>=3.1.4 in /usr/local/lib/python3.6/dist-packages (from oauth2client->tensor2tensor==1.15.0->pegasus==0.0.1) (4.6)\n",
"Requirement already satisfied: pyasn1-modules>=0.0.5 in /usr/local/lib/python3.6/dist-packages (from oauth2client->tensor2tensor==1.15.0->pegasus==0.0.1) (0.2.8)\n",
"Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python3.6/dist-packages (from oauth2client->tensor2tensor==1.15.0->pegasus==0.0.1) (0.4.8)\n",
"Requirement already satisfied: tensorflow-hub>=0.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gan->tensor2tensor==1.15.0->pegasus==0.0.1) (0.8.0)\n",
"Requirement already satisfied: decorator in /usr/local/lib/python3.6/dist-packages (from tensorflow-probability==0.7.0->tensor2tensor==1.15.0->pegasus==0.0.1) (4.4.2)\n",
"Requirement already satisfied: click>=5.1 in /usr/local/lib/python3.6/dist-packages (from flask->tensor2tensor==1.15.0->pegasus==0.0.1) (7.1.2)\n",
"Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python3.6/dist-packages (from flask->tensor2tensor==1.15.0->pegasus==0.0.1) (1.1.0)\n",
"Requirement already satisfied: Jinja2>=2.10.1 in /usr/local/lib/python3.6/dist-packages (from flask->tensor2tensor==1.15.0->pegasus==0.0.1) (2.11.2)\n",
"Requirement already satisfied: Werkzeug>=0.15 in /usr/local/lib/python3.6/dist-packages (from flask->tensor2tensor==1.15.0->pegasus==0.0.1) (1.0.1)\n",
"Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy->tensor2tensor==1.15.0->pegasus==0.0.1) (1.1.0)\n",
"Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.2->pegasus==0.0.1) (3.2.2)\n",
"Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.4.1->google-api-python-client->tensor2tensor==1.15.0->pegasus==0.0.1) (4.1.1)\n",
"Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from Jinja2>=2.10.1->flask->tensor2tensor==1.15.0->pegasus==0.0.1) (1.1.1)\n",
"Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.2->pegasus==0.0.1) (1.7.0)\n",
"Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.2->pegasus==0.0.1) (3.1.0)\n",
"Installing collected packages: pegasus\n",
" Found existing installation: pegasus 0.0.1\n",
" Can't uninstall 'pegasus'. No files were found to uninstall.\n",
" Running setup.py develop for pegasus\n",
"Successfully installed pegasus\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "8vBL78CPp2Gp",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "59505bcc-460a-49e4-800e-4b85e885a2cb"
},
"source": [
"!PYTHONPATH=. python pegasus/bin/evaluate.py --params=test_transformer \\\n",
"--param_overrides=vocab_filename=ckpt/pegasus_ckpt/c4.unigram.newline.10pct.96000.model,batch_size=1,beam_size=5,beam_alpha=0.6 \\\n",
"--model_dir=ckpt/pegasus_ckpt"
],
"execution_count": 11,
"outputs": [
{
"output_type": "stream",
"text": [
"WARNING:tensorflow:From pegasus/bin/evaluate.py:152: The name tf.enable_eager_execution is deprecated. Please use tf.compat.v1.enable_eager_execution instead.\n",
"\n",
"WARNING:tensorflow:From pegasus/bin/evaluate.py:153: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.\n",
"\n",
"WARNING:tensorflow:From pegasus/bin/evaluate.py:85: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use standard file APIs to check for files with this prefix.\n",
"W0728 16:42:38.932944 139681290311552 deprecation.py:323] From pegasus/bin/evaluate.py:85: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use standard file APIs to check for files with this prefix.\n",
"WARNING:tensorflow:From /content/pegasus/pegasus/ops/public_parsing_ops.py:93: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.\n",
"\n",
"W0728 16:42:38.950680 139681290311552 module_wrapper.py:139] From /content/pegasus/pegasus/ops/public_parsing_ops.py:93: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.\n",
"\n",
"WARNING:tensorflow:From /content/pegasus/pegasus/params/estimator_utils.py:49: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.\n",
"\n",
"W0728 16:42:39.218093 139681290311552 module_wrapper.py:139] From /content/pegasus/pegasus/params/estimator_utils.py:49: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.\n",
"\n",
"WARNING:tensorflow:Estimator's model_fn (<function _estimator_model_fn.<locals>.model_fn at 0x7f09979f8ae8>) includes params argument, but params are not passed to Estimator.\n",
"W0728 16:42:39.218894 139681290311552 estimator.py:1994] Estimator's model_fn (<function _estimator_model_fn.<locals>.model_fn at 0x7f09979f8ae8>) includes params argument, but params are not passed to Estimator.\n",
"INFO:tensorflow:Using config: {'_model_dir': 'ckpt/pegasus_ckpt', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true\n",
", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f0997a03208>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=1000, num_shards=None, num_cores_per_replica=None, per_host_input_for_training=2, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2, experimental_host_call_every_n_steps=1), '_cluster': None}\n",
"I0728 16:42:39.220199 139681290311552 estimator.py:212] Using config: {'_model_dir': 'ckpt/pegasus_ckpt', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true\n",
", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f0997a03208>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=1000, num_shards=None, num_cores_per_replica=None, per_host_input_for_training=2, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2, experimental_host_call_every_n_steps=1), '_cluster': None}\n",
"INFO:tensorflow:_TPUContext: eval_on_tpu True\n",
"I0728 16:42:39.221028 139681290311552 tpu_context.py:220] _TPUContext: eval_on_tpu True\n",
"WARNING:tensorflow:eval_on_tpu ignored because use_tpu is False.\n",
"W0728 16:42:39.221745 139681290311552 tpu_context.py:222] eval_on_tpu ignored because use_tpu is False.\n",
"WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"If using Keras pass *_constraint arguments to layers.\n",
"W0728 16:42:39.257469 139681290311552 deprecation.py:506] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"If using Keras pass *_constraint arguments to layers.\n",
"WARNING:tensorflow:From /content/pegasus/pegasus/data/datasets.py:117: The name tf.gfile.Glob is deprecated. Please use tf.io.gfile.glob instead.\n",
"\n",
"W0728 16:42:39.258476 139681290311552 module_wrapper.py:139] From /content/pegasus/pegasus/data/datasets.py:117: The name tf.gfile.Glob is deprecated. Please use tf.io.gfile.glob instead.\n",
"\n",
"2020-07-28 16:42:40.313757: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1\n",
"2020-07-28 16:42:40.362360: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected\n",
"2020-07-28 16:42:40.362465: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (07017d511ff6): /proc/driver/nvidia/version does not exist\n",
"WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/ragged/ragged_tensor.py:1586: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
"W0728 16:42:40.477251 139681290311552 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/ragged/ragged_tensor.py:1586: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
"INFO:tensorflow:Calling model_fn.\n",
"I0728 16:42:40.791538 139681290311552 estimator.py:1148] Calling model_fn.\n",
"INFO:tensorflow:Running infer on CPU\n",
"I0728 16:42:40.791894 139681290311552 tpu_estimator.py:3124] Running infer on CPU\n",
"WARNING:tensorflow:From /content/pegasus/pegasus/params/estimator_utils.py:78: The name tf.get_variable_scope is deprecated. Please use tf.compat.v1.get_variable_scope instead.\n",
"\n",
"W0728 16:42:40.792376 139681290311552 module_wrapper.py:139] From /content/pegasus/pegasus/params/estimator_utils.py:78: The name tf.get_variable_scope is deprecated. Please use tf.compat.v1.get_variable_scope instead.\n",
"\n",
"WARNING:tensorflow:From /content/pegasus/pegasus/layers/attention.py:41: The name tf.layers.Dense is deprecated. Please use tf.compat.v1.layers.Dense instead.\n",
"\n",
"W0728 16:42:40.792869 139681290311552 module_wrapper.py:139] From /content/pegasus/pegasus/layers/attention.py:41: The name tf.layers.Dense is deprecated. Please use tf.compat.v1.layers.Dense instead.\n",
"\n",
"WARNING:tensorflow:From /content/pegasus/pegasus/layers/embedding.py:57: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.\n",
"\n",
"W0728 16:42:40.843258 139681290311552 module_wrapper.py:139] From /content/pegasus/pegasus/layers/embedding.py:57: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.\n",
"\n",
"WARNING:tensorflow:From /content/pegasus/pegasus/layers/embedding.py:57: The name tf.AUTO_REUSE is deprecated. Please use tf.compat.v1.AUTO_REUSE instead.\n",
"\n",
"W0728 16:42:40.843591 139681290311552 module_wrapper.py:139] From /content/pegasus/pegasus/layers/embedding.py:57: The name tf.AUTO_REUSE is deprecated. Please use tf.compat.v1.AUTO_REUSE instead.\n",
"\n",
"WARNING:tensorflow:From /content/pegasus/pegasus/layers/embedding.py:61: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.\n",
"\n",
"W0728 16:42:40.843898 139681290311552 module_wrapper.py:139] From /content/pegasus/pegasus/layers/embedding.py:61: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.\n",
"\n",
"WARNING:tensorflow:From /content/pegasus/pegasus/layers/embedding.py:64: calling RandomNormal.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Call initializer instance with the dtype argument instead of passing it to the constructor\n",
"W0728 16:42:40.844166 139681290311552 deprecation.py:506] From /content/pegasus/pegasus/layers/embedding.py:64: calling RandomNormal.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Call initializer instance with the dtype argument instead of passing it to the constructor\n",
"WARNING:tensorflow:From /content/pegasus/pegasus/layers/attention.py:106: The name tf.matrix_band_part is deprecated. Please use tf.linalg.band_part instead.\n",
"\n",
"W0728 16:42:43.663957 139681290311552 module_wrapper.py:139] From /content/pegasus/pegasus/layers/attention.py:106: The name tf.matrix_band_part is deprecated. Please use tf.linalg.band_part instead.\n",
"\n",
"INFO:tensorflow:Done calling model_fn.\n",
"I0728 16:42:49.286858 139681290311552 estimator.py:1150] Done calling model_fn.\n",
"INFO:tensorflow:Graph was finalized.\n",
"I0728 16:42:50.440357 139681290311552 monitored_session.py:240] Graph was finalized.\n",
"2020-07-28 16:42:50.441106: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA\n",
"2020-07-28 16:42:50.486693: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz\n",
"2020-07-28 16:42:50.487148: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1e54bc0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:\n",
"2020-07-28 16:42:50.487192: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version\n",
"INFO:tensorflow:Restoring parameters from ckpt/pegasus_ckpt/model.ckpt-1500000\n",
"I0728 16:42:50.493927 139681290311552 saver.py:1284] Restoring parameters from ckpt/pegasus_ckpt/model.ckpt-1500000\n",
"2020-07-28 16:42:51.613891: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 393637888 exceeds 10% of system memory.\n",
"INFO:tensorflow:Running local_init_op.\n",
"I0728 16:43:34.249890 139681290311552 session_manager.py:500] Running local_init_op.\n",
"INFO:tensorflow:Done running local_init_op.\n",
"I0728 16:43:34.470002 139681290311552 session_manager.py:502] Done running local_init_op.\n",
"2020-07-28 16:43:40.147192: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 67108864 exceeds 10% of system memory.\n",
"2020-07-28 16:43:40.872855: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 67108864 exceeds 10% of system memory.\n",
"2020-07-28 16:43:41.564656: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 67108864 exceeds 10% of system memory.\n",
"2020-07-28 16:43:42.252804: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 67108864 exceeds 10% of system memory.\n",
"/content/pegasus/pegasus/eval/text_eval.py:61: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
" if np.issubdtype(matrix.dtype, float):\n",
"I0728 17:19:23.119514 139681290311552 text_eval.py:126] INPUTS: [0]:\n",
"A double for Aubameyang as Arsenal stuns Manchester City in FA Cup semifinal\n",
"PAUL GITTINGS JULY 18, 2020\n",
"\n",
"(CNN)Pierre-Emerick Aubameyang scored in each half as Arsenal stunned holder Manchester City 2-0 at Wembley on Saturday to reach a record 21st FA Cup final.\n",
"\n",
"The Gabonese striker struck in the 19th and 71st minutes as Arsenal coach Mikel Arteta gained bragging rights over Pepe Guardiola, his old mentor at City.\n",
"\n",
"Manchester City, aiming for a domestic cup double having won the English League Cup before the lockdown, could not make the most of the lion's share of possession as Arsenal defended stoutly and looked ever dangerous in counterattack.\n",
"\n",
"The opening goal came in this fashion as the Gunners launched a sweet passing move from inside their own half and Nicolas Pepe found Aubameyang, who finished neatly at the far post, all the more creditable having missed another chance moments before.\n",
"\n",
"The same pattern continued with City unable to capitalize on its pressure before Arsenal's Shkodran Mustafi forced a fine save from City keeper Ederson just before the break.\n",
"\n",
"Breakaway second goal\n",
"City, who had its European football ban lifted by a ruling earlier this week, had a second-half penalty appeal turned down by referee John Moss and the video assistant referee for a Mustafi challenge on Raheem Sterling, while Arsenal's rock-solid Emiliano Martinez saved well from City's Riyad Mahrez.\n",
"\n",
"The near misses came back to haunt the cupholder as Arsenal broke forward and Aubameyang raced onto Kieran Tierney's pass to shoot home his second past Ederson.\n",
"\n",
"David Luiz, who made a disastrous appearance for Arsenal in the first game of the restart against City, then epitomized his side's improvement under Arteta by brilliantly blocking a further Sterling effort, while Aymeric Laporte's thunderbolt effort went just wide.\n",
"\n",
"At the end it was a smiling Arteta, formerly an assistant manager to Guardiola at City, who was celebrating in a cavernous and empty Wembley, normally filled to capacity for such a big match.\n",
"\n",
"Luiz paid tribute to Arteta's influence since taking over at the North London club late last year, with a 2-1 victory over new Premier League champion Liverpool earlier this week further sign of progress.\n",
"\n",
"\"We have an amazing coach but we can't go from 0 to 100. We are improving. The spirit was great and I'm happy for the team because they deserve it,\" Luiz told BT Sport.\n",
"\n",
"\"We have a final to play this season and we will try and win a title for this club because this club deserves to win a trophy,\" the Brazilian added.\n",
"\n",
"The cup final is set for August 1 and Arsenal will face the winner of the second semifinal on Sunday between Chelsea and Manchester United.\n",
"\n",
"City will now focus on its Champions League dream with the second leg of its last 16 tie against new Spanish champion Real Madrid next month.\n",
"I0728 17:19:23.120754 139681290311552 text_eval.py:126] TARGETS: <1> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> <0> \n",
"I0728 17:19:23.121018 139681290311552 text_eval.py:126] PREDICTIONS: A double for Aubameyang as Arsenal stuns Manchester City in FA Cup semifinal.PAUL GITTINGS JULY 18, 2020\n",
"(CNN)Pierre-Emerick Aubameyang scored in each half as Arsenal stunned holder Manchester City 2-0 at Wembley on Saturday to reach a record 21st FA Cup final.\n",
"\n",
"INFO:tensorflow:prediction_loop marked as finished\n",
"I0728 17:19:24.507650 139681290311552 error_handling.py:101] prediction_loop marked as finished\n",
"INFO:tensorflow:prediction_loop marked as finished\n",
"I0728 17:19:24.507980 139681290311552 error_handling.py:101] prediction_loop marked as finished\n",
"WARNING:tensorflow:From /content/pegasus/pegasus/eval/text_eval.py:197: The name tf.gfile.Open is deprecated. Please use tf.io.gfile.GFile instead.\n",
"\n",
"W0728 17:19:24.941095 139681290311552 module_wrapper.py:139] From /content/pegasus/pegasus/eval/text_eval.py:197: The name tf.gfile.Open is deprecated. Please use tf.io.gfile.GFile instead.\n",
"\n"
],
"name": "stdout"
}
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment