Last active
June 4, 2020 22:48
-
-
Save hanneshapke/f0980b7422d367808dae409536fe9b46 to your computer and use it in GitHub Desktop.
TFX_Pipeline_for_Bert_Preprocessing_wo_tf.Example.ipynb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "nbformat": 4, | |
| "nbformat_minor": 0, | |
| "metadata": { | |
| "colab": { | |
| "name": "TFX_Pipeline_for_Bert_Preprocessing_wo_tf.Example.ipynb", | |
| "provenance": [], | |
| "collapsed_sections": [], | |
| "toc_visible": true, | |
| "machine_shape": "hm", | |
| "include_colab_link": true | |
| }, | |
| "kernelspec": { | |
| "name": "python3", | |
| "display_name": "Python 3" | |
| }, | |
| "accelerator": "GPU" | |
| }, | |
| "cells": [ | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "view-in-github", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "<a href=\"https://colab.research.google.com/gist/hanneshapke/f0980b7422d367808dae409536fe9b46/tfx_pipeline_for_bert_preprocessing_wo_tf-example.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "w8oETouR1D88", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "##### Copyright © 2020 The TensorFlow Authors." | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "uPq-BKBO1D89", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License.\n", | |
| "You may obtain a copy of the License at\n", | |
| "\n", | |
| "https://www.apache.org/licenses/LICENSE-2.0\n", | |
| "\n", | |
| "Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License." | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "Va0ssmD81D89", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "# Pipeline Example performing the Bert Preprocessing with TensorFlow Transform" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "LvwlIAWI1D89", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "<table class=\"tfo-notebook-buttons\" width=\"100%\">\n", | |
| " <td>\n", | |
| " <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/workshops/blob/master/blog/TFX_Pipeline_for_Bert_Preprocessing.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n", | |
| " </td>\n", | |
| " <td>\n", | |
| " <a target=\"_blank\" href=\"https://github.com/tensorflow/workshops/blob/master/blog/TFX_Pipeline_for_Bert_Preprocessing.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n", | |
| " </td>\n", | |
| "</table>" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "o18JmOsdRTEw", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## Motivation\n", | |
| "\n", | |
| "Instead of converting the input to a tranformer model into token ids on the client side, the model exported from this pipeline will allow the conversion on the server side.\n", | |
| "\n", | |
| "The pipeline takes advantage of the broad TensorFlow Eco system, including:\n", | |
| "* Loading the IMDB dataset via **TensorFlow Datasets**\n", | |
| "* Loading a pre-trained model via **tf.hub**\n", | |
| "* Manipulating the raw input data with **tf.text**\n", | |
| "* Building a simple model architecture with **Keras**\n", | |
| "* Composing the model pipeline with **TensorFlow Extended**, e.g. TensorFlow Transform, TensorFlow Data Validation and then consuming the tf.Keras model with the latest Trainer component from TFX\n", | |
| "\n", | |
| "The structure of the overall pipeline follows the [TFX Taxi Cab example](https://colab.research.google.com/github/tensorflow/tfx/blob/master/docs/tutorials/tfx/components.ipynb) \n", | |
| "\n", | |
| "### Outline\n", | |
| "\n", | |
| "* Install Required Packages\n", | |
| "* Load the training data set\n", | |
| "* Create the TFX Pipeline\n", | |
| "* Export the trained Model\n", | |
| "* Test the exported Model\n", | |
| "\n", | |
| "**Non-Colab users** \n", | |
| "\n", | |
| "This notebook was written to run in Google Colab environments. But they can run in any Jupyter environment. In that case, update the file and directory path and install TensorFlow>=2.2.0 manually.\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "Rinax0YJ_otk", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "# Project Setup\n", | |
| "\n", | |
| "## Install Required Packages" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "Sjjgiv0bM0hi", | |
| "colab_type": "code", | |
| "outputId": "936bb57b-8f43-461e-d061-eeeb4238d483", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 51 | |
| } | |
| }, | |
| "source": [ | |
| "try:\n", | |
| " # %tensorflow_version only exists in Colab.\n", | |
| " %tensorflow_version 2.x\n", | |
| "except Exception:\n", | |
| " pass\n", | |
| "\n", | |
| "!pip install -Uq tfx==0.22.0-rc0\n", | |
| "!pip install -Uq tensorflow-text # the tf-text version needs to match the tf version\n", | |
| "\n", | |
| "print(\"Restart your runtime enable after installing the packages\")" | |
| ], | |
| "execution_count": 0, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "\u001b[31mERROR: apache-beam 2.21.0 has requirement httplib2<=0.12.0,>=0.8, but you'll have httplib2 0.18.1 which is incompatible.\u001b[0m\n", | |
| "Restart your runtime enable after installing the packages\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "3rqjoMKeMxWq", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## Restart the Runtime\n", | |
| "\n", | |
| "**Note** \n", | |
| "After installing the required Python packages, you'll need to restart the Colab Runtime Engine (Menu > Runtime > Restart runtime...)\n", | |
| "\n", | |
| "\n", | |
| "\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "5NQSFUNALhFG", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## Import relevant packages" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "oThi-x8xLlv-", | |
| "colab_type": "code", | |
| "colab": {} | |
| }, | |
| "source": [ | |
| "import glob\n", | |
| "import os\n", | |
| "import pprint\n", | |
| "import re\n", | |
| "import tempfile\n", | |
| "from shutil import rmtree\n", | |
| "\n", | |
| "import numpy as np\n", | |
| "import pandas as pd\n", | |
| "import tensorflow as tf\n", | |
| "import tensorflow_data_validation as tfdv\n", | |
| "import tensorflow_hub as hub\n", | |
| "import tensorflow_model_analysis as tfma\n", | |
| "import tensorflow_transform as tft\n", | |
| "import tensorflow_transform.beam as tft_beam\n", | |
| "from tensorflow_transform.beam.tft_beam_io import transform_fn_io\n", | |
| "from tensorflow_transform.saved import saved_transform_io\n", | |
| "from tensorflow_transform.tf_metadata import (dataset_metadata, dataset_schema,\n", | |
| " metadata_io, schema_utils)\n", | |
| "from tfx.components import (Evaluator, ExampleValidator, ImportExampleGen,\n", | |
| " ModelValidator, Pusher, ResolverNode, SchemaGen,\n", | |
| " StatisticsGen, Trainer, Transform)\n", | |
| "from tfx.components.base import executor_spec\n", | |
| "from tfx.components.trainer.executor import GenericExecutor\n", | |
| "from tfx.dsl.experimental import latest_blessed_model_resolver\n", | |
| "from tfx.proto import evaluator_pb2, example_gen_pb2, pusher_pb2, trainer_pb2\n", | |
| "from tfx.types import Channel\n", | |
| "from tfx.types.standard_artifacts import Model, ModelBlessing\n", | |
| "from tfx.utils.dsl_utils import external_input\n", | |
| "\n", | |
| "import tensorflow_datasets as tfds\n", | |
| "import tensorflow_model_analysis as tfma\n", | |
| "import tensorflow_text as text\n", | |
| "\n", | |
| "from tfx.orchestration.experimental.interactive.interactive_context import \\\n", | |
| " InteractiveContext\n", | |
| "\n", | |
| "%load_ext tfx.orchestration.experimental.interactive.notebook_extensions.skip" | |
| ], | |
| "execution_count": 0, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "uuKziu0W9o7O", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## Check GPU Availability\n", | |
| "\n", | |
| "Check if your Colab notebook is configured to use Graphical Processing Units (GPUs). If zero GPUs are available, check if the Colab notebook is configured to use GPUs (Menu > Runtime > Change Runtime Type).\n", | |
| "\n", | |
| "\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "wlvqWtr7R8eu", | |
| "colab_type": "code", | |
| "outputId": "afc7493e-99a4-4339-ee6c-88fa61094b93", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 34 | |
| } | |
| }, | |
| "source": [ | |
| "num_gpus_available = len(tf.config.experimental.list_physical_devices('GPU'))\n", | |
| "print(\"Num GPUs Available: \", num_gpus_available)\n", | |
| "assert num_gpus_available > 0" | |
| ], | |
| "execution_count": 0, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "Num GPUs Available: 1\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "q4OMItPSLnDj", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## Download the IMDB Dataset from TensorFlow Datasets\n", | |
| "\n", | |
| "For our demo example, we are using the [IMDB data set](https://www.kaggle.com/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews) to train a sentiment model based on the pre-trained BERT model. The data set is provided through [TensorFlow Datasets](https://www.tensorflow.org/datasets). Our ML pipeline can read TFRecords, however it expects only TFRecord files in the data folder. That is the reason why we need to delete the additional files provided by TFDS." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "KjWjnzPGKjIk", | |
| "colab_type": "code", | |
| "outputId": "cfef6f89-9ac7-407a-cbce-2c7b6ddc157c", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 153 | |
| } | |
| }, | |
| "source": [ | |
| "!mkdir /content/tfds/\n", | |
| "\n", | |
| "def clean_before_download(base_data_dir):\n", | |
| " rmtree(base_data_dir)\n", | |
| " \n", | |
| "def delete_unnecessary_files(base_path):\n", | |
| " os.remove(base_path + \"dataset_info.json\")\n", | |
| " os.remove(base_path + \"label.labels.txt\")\n", | |
| " \n", | |
| " counter = 2\n", | |
| " for f in glob.glob(base_path + \"imdb_reviews-unsupervised.*\"):\n", | |
| " os.remove(f)\n", | |
| " counter += 1\n", | |
| " print(f\"Deleted {counter} files\")\n", | |
| "\n", | |
| "def get_dataset(name='imdb_reviews', version=\"0.1.0\"):\n", | |
| "\n", | |
| " base_data_dir = \"/content/tfds/\"\n", | |
| " config=\"plain_text\"\n", | |
| " version=\"0.1.0\"\n", | |
| "\n", | |
| " clean_before_download(base_data_dir)\n", | |
| " tfds.disable_progress_bar()\n", | |
| " builder = tfds.text.IMDBReviews(data_dir=base_data_dir, \n", | |
| " config=config, \n", | |
| " version=version)\n", | |
| " download_config = tfds.download.DownloadConfig(\n", | |
| " download_mode=tfds.GenerateMode.FORCE_REDOWNLOAD)\n", | |
| " builder.download_and_prepare(download_config=download_config)\n", | |
| "\n", | |
| " base_tfrecords_filename = os.path.join(base_data_dir, \"imdb_reviews\", config, version, \"\")\n", | |
| " train_tfrecords_filename = base_tfrecords_filename + \"imdb_reviews-train*\"\n", | |
| " test_tfrecords_filename = base_tfrecords_filename + \"imdb_reviews-test*\"\n", | |
| " label_filename = os.path.join(base_tfrecords_filename, \"label.labels.txt\")\n", | |
| " labels = [label.rstrip('\\n') for label in open(label_filename)]\n", | |
| " delete_unnecessary_files(base_tfrecords_filename)\n", | |
| " return (train_tfrecords_filename, test_tfrecords_filename), labels\n", | |
| "\n", | |
| "tfrecords_filenames, labels = get_dataset()" | |
| ], | |
| "execution_count": 0, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "mkdir: cannot create directory ‘/content/tfds/’: File exists\n", | |
| "\u001b[1mDownloading and preparing dataset imdb_reviews/plain_text/0.1.0 (download: 80.23 MiB, generated: Unknown size, total: 80.23 MiB) to /content/tfds/imdb_reviews/plain_text/0.1.0...\u001b[0m\n", | |
| "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_datasets/core/file_format_adapter.py:210: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version.\n", | |
| "Instructions for updating:\n", | |
| "Use eager execution and: \n", | |
| "`tf.data.TFRecordDataset(path)`\n", | |
| "\u001b[1mDataset imdb_reviews downloaded and prepared to /content/tfds/imdb_reviews/plain_text/0.1.0. Subsequent calls will reuse this data.\u001b[0m\n", | |
| "Deleted 22 files\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "RDtPNfOwriT8", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## Helper function to load the BERT model as Keras layer\n", | |
| "\n", | |
| "In our pipeline components, we are reusing the BERT Layer from tf.hub in two places\n", | |
| "* in the model architecture when we define our Keras model\n", | |
| "* in our preprocessing function when we extract the BERT settings (casing and vocab file path) to reuse the settings during the tokenization " | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "Tre_oQu0rlrU", | |
| "colab_type": "code", | |
| "outputId": "b1245259-316d-4787-a3e6-15832b519a54", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 51 | |
| } | |
| }, | |
| "source": [ | |
| "%%skip_for_export\n", | |
| "%%writefile bert.py\n", | |
| "\n", | |
| "import tensorflow_hub as hub\n", | |
| "\n", | |
| "BERT_TFHUB_URL = \"https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/2\"\n", | |
| "\n", | |
| "def load_bert_layer(model_url=BERT_TFHUB_URL):\n", | |
| " # Load the pre-trained BERT model as layer in Keras\n", | |
| " bert_layer = hub.KerasLayer(\n", | |
| " handle=model_url,\n", | |
| " trainable=True)\n", | |
| " return bert_layer\n" | |
| ], | |
| "execution_count": 0, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "Overwriting bert.py\n", | |
| "This cell will be skipped during export to pipeline.\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "k-5QGnm_lFJD", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "# TFX Pipeline\n", | |
| "\n", | |
| "The TensorFlow Extended Pipeline is more or less following the example setup shown here. We'll only note deviations from the original setup." | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "arPCEBYEFqEr", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## Initializing the Interactive TFX Pipeline" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "sBO0T3D5kkOt", | |
| "colab_type": "code", | |
| "outputId": "3f5af053-2b14-4296-ce57-16fa048a0277", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 51 | |
| } | |
| }, | |
| "source": [ | |
| "context = InteractiveContext()" | |
| ], | |
| "execution_count": 0, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "WARNING:absl:InteractiveContext pipeline_root argument not provided: using temporary directory /tmp/tfx-interactive-2020-06-04T20_59_02.746836-3cei_3jg as root for pipeline outputs.\n", | |
| "WARNING:absl:InteractiveContext metadata_connection_config not provided: using SQLite ML Metadata database at /tmp/tfx-interactive-2020-06-04T20_59_02.746836-3cei_3jg/metadata.sqlite.\n" | |
| ], | |
| "name": "stderr" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "qo2Q-c_ynL2x", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## Loading the dataset" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "W8GqUHwAKm6j", | |
| "colab_type": "code", | |
| "outputId": "eaf87a45-999e-47f7-915f-422f64c7b616", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 228 | |
| } | |
| }, | |
| "source": [ | |
| "output = example_gen_pb2.Output(\n", | |
| " split_config=example_gen_pb2.SplitConfig(splits=[\n", | |
| " example_gen_pb2.SplitConfig.Split(name='train', hash_buckets=45),\n", | |
| " example_gen_pb2.SplitConfig.Split(name='eval', hash_buckets=5)\n", | |
| " ]))\n", | |
| "# Load the data from our prepared TFDS folder\n", | |
| "examples = external_input(\"/content/tfds/imdb_reviews/plain_text/0.1.0\")\n", | |
| "example_gen = ImportExampleGen(input=examples, output_config=output)\n", | |
| "\n", | |
| "context.run(example_gen)" | |
| ], | |
| "execution_count": 0, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.\n" | |
| ], | |
| "name": "stderr" | |
| }, | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "application/javascript": [ | |
| "\n", | |
| " if (typeof window.interactive_beam_jquery == 'undefined') {\n", | |
| " var jqueryScript = document.createElement('script');\n", | |
| " jqueryScript.src = 'https://code.jquery.com/jquery-3.4.1.slim.min.js';\n", | |
| " jqueryScript.type = 'text/javascript';\n", | |
| " jqueryScript.onload = function() {\n", | |
| " var datatableScript = document.createElement('script');\n", | |
| " datatableScript.src = 'https://cdn.datatables.net/1.10.20/js/jquery.dataTables.min.js';\n", | |
| " datatableScript.type = 'text/javascript';\n", | |
| " datatableScript.onload = function() {\n", | |
| " window.interactive_beam_jquery = jQuery.noConflict(true);\n", | |
| " window.interactive_beam_jquery(document).ready(function($){\n", | |
| " \n", | |
| " });\n", | |
| " }\n", | |
| " document.head.appendChild(datatableScript);\n", | |
| " };\n", | |
| " document.head.appendChild(jqueryScript);\n", | |
| " } else {\n", | |
| " window.interactive_beam_jquery(document).ready(function($){\n", | |
| " \n", | |
| " });\n", | |
| " }" | |
| ] | |
| }, | |
| "metadata": { | |
| "tags": [] | |
| } | |
| }, | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "application/javascript": [ | |
| "\n", | |
| " var import_html = () => {\n", | |
| " ['https://raw.githubusercontent.com/PAIR-code/facets/1.0.0/facets-dist/facets-jupyter.html'].forEach(href => {\n", | |
| " var link = document.createElement('link');\n", | |
| " link.rel = 'import'\n", | |
| " link.href = href;\n", | |
| " document.head.appendChild(link);\n", | |
| " });\n", | |
| " }\n", | |
| " if ('import' in document.createElement('link')) {\n", | |
| " import_html();\n", | |
| " } else {\n", | |
| " var webcomponentScript = document.createElement('script');\n", | |
| " webcomponentScript.src = 'https://cdnjs.cloudflare.com/ajax/libs/webcomponentsjs/1.3.3/webcomponents-lite.js';\n", | |
| " webcomponentScript.type = 'text/javascript';\n", | |
| " webcomponentScript.onload = function(){\n", | |
| " import_html();\n", | |
| " };\n", | |
| " document.head.appendChild(webcomponentScript);\n", | |
| " }" | |
| ] | |
| }, | |
| "metadata": { | |
| "tags": [] | |
| } | |
| }, | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "WARNING:apache_beam.io.tfrecordio:Couldn't find python-snappy so the implementation of _TFRecordUtil._masked_crc32c is not as fast as it could be.\n" | |
| ], | |
| "name": "stderr" | |
| }, | |
| { | |
| "output_type": "execute_result", | |
| "data": { | |
| "text/html": [ | |
| "<style>\n", | |
| ".tfx-object.expanded {\n", | |
| " padding: 4px 8px 4px 8px;\n", | |
| " background: white;\n", | |
| " border: 1px solid #bbbbbb;\n", | |
| " box-shadow: 4px 4px 2px rgba(0,0,0,0.05);\n", | |
| "}\n", | |
| ".tfx-object, .tfx-object * {\n", | |
| " font-size: 11pt;\n", | |
| "}\n", | |
| ".tfx-object > .title {\n", | |
| " cursor: pointer;\n", | |
| "}\n", | |
| ".tfx-object .expansion-marker {\n", | |
| " color: #999999;\n", | |
| "}\n", | |
| ".tfx-object.expanded > .title > .expansion-marker:before {\n", | |
| " content: '▼';\n", | |
| "}\n", | |
| ".tfx-object.collapsed > .title > .expansion-marker:before {\n", | |
| " content: '▶';\n", | |
| "}\n", | |
| ".tfx-object .class-name {\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object .deemphasize {\n", | |
| " opacity: 0.5;\n", | |
| "}\n", | |
| ".tfx-object.collapsed > table.attr-table {\n", | |
| " display: none;\n", | |
| "}\n", | |
| ".tfx-object.expanded > table.attr-table {\n", | |
| " display: block;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table {\n", | |
| " border: 2px solid white;\n", | |
| " margin-top: 5px;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attr-name {\n", | |
| " vertical-align: top;\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attrvalue {\n", | |
| " text-align: left;\n", | |
| "}\n", | |
| "</style>\n", | |
| "<script>\n", | |
| "function toggleTfxObject(element) {\n", | |
| " var objElement = element.parentElement;\n", | |
| " if (objElement.classList.contains('collapsed')) {\n", | |
| " objElement.classList.remove('collapsed');\n", | |
| " objElement.classList.add('expanded');\n", | |
| " } else {\n", | |
| " objElement.classList.add('collapsed');\n", | |
| " objElement.classList.remove('expanded');\n", | |
| " }\n", | |
| "}\n", | |
| "</script>\n", | |
| "<div class=\"tfx-object expanded\"><div class = \"title\" onclick=\"toggleTfxObject(this)\"><span class=\"expansion-marker\"></span><span class=\"class-name\">ExecutionResult</span><span class=\"deemphasize\"> at 0x7f4be9ac6a90</span></div><table class=\"attr-table\"><tr><td class=\"attr-name\">.execution_id</td><td class = \"attrvalue\">1</td></tr><tr><td class=\"attr-name\">.component</td><td class = \"attrvalue\"><style>\n", | |
| ".tfx-object.expanded {\n", | |
| " padding: 4px 8px 4px 8px;\n", | |
| " background: white;\n", | |
| " border: 1px solid #bbbbbb;\n", | |
| " box-shadow: 4px 4px 2px rgba(0,0,0,0.05);\n", | |
| "}\n", | |
| ".tfx-object, .tfx-object * {\n", | |
| " font-size: 11pt;\n", | |
| "}\n", | |
| ".tfx-object > .title {\n", | |
| " cursor: pointer;\n", | |
| "}\n", | |
| ".tfx-object .expansion-marker {\n", | |
| " color: #999999;\n", | |
| "}\n", | |
| ".tfx-object.expanded > .title > .expansion-marker:before {\n", | |
| " content: '▼';\n", | |
| "}\n", | |
| ".tfx-object.collapsed > .title > .expansion-marker:before {\n", | |
| " content: '▶';\n", | |
| "}\n", | |
| ".tfx-object .class-name {\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object .deemphasize {\n", | |
| " opacity: 0.5;\n", | |
| "}\n", | |
| ".tfx-object.collapsed > table.attr-table {\n", | |
| " display: none;\n", | |
| "}\n", | |
| ".tfx-object.expanded > table.attr-table {\n", | |
| " display: block;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table {\n", | |
| " border: 2px solid white;\n", | |
| " margin-top: 5px;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attr-name {\n", | |
| " vertical-align: top;\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attrvalue {\n", | |
| " text-align: left;\n", | |
| "}\n", | |
| "</style>\n", | |
| "<script>\n", | |
| "function toggleTfxObject(element) {\n", | |
| " var objElement = element.parentElement;\n", | |
| " if (objElement.classList.contains('collapsed')) {\n", | |
| " objElement.classList.remove('collapsed');\n", | |
| " objElement.classList.add('expanded');\n", | |
| " } else {\n", | |
| " objElement.classList.add('collapsed');\n", | |
| " objElement.classList.remove('expanded');\n", | |
| " }\n", | |
| "}\n", | |
| "</script>\n", | |
| "<div class=\"tfx-object collapsed\"><div class = \"title\" onclick=\"toggleTfxObject(this)\"><span class=\"expansion-marker\"></span><span class=\"class-name\">ImportExampleGen</span><span class=\"deemphasize\"> at 0x7f4c33f67588</span></div><table class=\"attr-table\"><tr><td class=\"attr-name\">.inputs</td><td class = \"attrvalue\"><table class=\"attr-table\"><tr><td class=\"attr-name\">['input']</td><td class = \"attrvalue\"><style>\n", | |
| ".tfx-object.expanded {\n", | |
| " padding: 4px 8px 4px 8px;\n", | |
| " background: white;\n", | |
| " border: 1px solid #bbbbbb;\n", | |
| " box-shadow: 4px 4px 2px rgba(0,0,0,0.05);\n", | |
| "}\n", | |
| ".tfx-object, .tfx-object * {\n", | |
| " font-size: 11pt;\n", | |
| "}\n", | |
| ".tfx-object > .title {\n", | |
| " cursor: pointer;\n", | |
| "}\n", | |
| ".tfx-object .expansion-marker {\n", | |
| " color: #999999;\n", | |
| "}\n", | |
| ".tfx-object.expanded > .title > .expansion-marker:before {\n", | |
| " content: '▼';\n", | |
| "}\n", | |
| ".tfx-object.collapsed > .title > .expansion-marker:before {\n", | |
| " content: '▶';\n", | |
| "}\n", | |
| ".tfx-object .class-name {\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object .deemphasize {\n", | |
| " opacity: 0.5;\n", | |
| "}\n", | |
| ".tfx-object.collapsed > table.attr-table {\n", | |
| " display: none;\n", | |
| "}\n", | |
| ".tfx-object.expanded > table.attr-table {\n", | |
| " display: block;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table {\n", | |
| " border: 2px solid white;\n", | |
| " margin-top: 5px;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attr-name {\n", | |
| " vertical-align: top;\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attrvalue {\n", | |
| " text-align: left;\n", | |
| "}\n", | |
| "</style>\n", | |
| "<script>\n", | |
| "function toggleTfxObject(element) {\n", | |
| " var objElement = element.parentElement;\n", | |
| " if (objElement.classList.contains('collapsed')) {\n", | |
| " objElement.classList.remove('collapsed');\n", | |
| " objElement.classList.add('expanded');\n", | |
| " } else {\n", | |
| " objElement.classList.add('collapsed');\n", | |
| " objElement.classList.remove('expanded');\n", | |
| " }\n", | |
| "}\n", | |
| "</script>\n", | |
| "<div class=\"tfx-object collapsed\"><div class = \"title\" onclick=\"toggleTfxObject(this)\"><span class=\"expansion-marker\"></span><span class=\"class-name\">Channel</span> of type <span class=\"class-name\">'ExternalArtifact'</span> (1 artifact)<span class=\"deemphasize\"> at 0x7f4c33f675c0</span></div><table class=\"attr-table\"><tr><td class=\"attr-name\">.type_name</td><td class = \"attrvalue\">ExternalArtifact</td></tr><tr><td class=\"attr-name\">._artifacts</td><td class = \"attrvalue\"><table class=\"attr-table\"><tr><td class=\"attr-name\">[0]</td><td class = \"attrvalue\"><style>\n", | |
| ".tfx-object.expanded {\n", | |
| " padding: 4px 8px 4px 8px;\n", | |
| " background: white;\n", | |
| " border: 1px solid #bbbbbb;\n", | |
| " box-shadow: 4px 4px 2px rgba(0,0,0,0.05);\n", | |
| "}\n", | |
| ".tfx-object, .tfx-object * {\n", | |
| " font-size: 11pt;\n", | |
| "}\n", | |
| ".tfx-object > .title {\n", | |
| " cursor: pointer;\n", | |
| "}\n", | |
| ".tfx-object .expansion-marker {\n", | |
| " color: #999999;\n", | |
| "}\n", | |
| ".tfx-object.expanded > .title > .expansion-marker:before {\n", | |
| " content: '▼';\n", | |
| "}\n", | |
| ".tfx-object.collapsed > .title > .expansion-marker:before {\n", | |
| " content: '▶';\n", | |
| "}\n", | |
| ".tfx-object .class-name {\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object .deemphasize {\n", | |
| " opacity: 0.5;\n", | |
| "}\n", | |
| ".tfx-object.collapsed > table.attr-table {\n", | |
| " display: none;\n", | |
| "}\n", | |
| ".tfx-object.expanded > table.attr-table {\n", | |
| " display: block;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table {\n", | |
| " border: 2px solid white;\n", | |
| " margin-top: 5px;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attr-name {\n", | |
| " vertical-align: top;\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attrvalue {\n", | |
| " text-align: left;\n", | |
| "}\n", | |
| "</style>\n", | |
| "<script>\n", | |
| "function toggleTfxObject(element) {\n", | |
| " var objElement = element.parentElement;\n", | |
| " if (objElement.classList.contains('collapsed')) {\n", | |
| " objElement.classList.remove('collapsed');\n", | |
| " objElement.classList.add('expanded');\n", | |
| " } else {\n", | |
| " objElement.classList.add('collapsed');\n", | |
| " objElement.classList.remove('expanded');\n", | |
| " }\n", | |
| "}\n", | |
| "</script>\n", | |
| "<div class=\"tfx-object collapsed\"><div class = \"title\" onclick=\"toggleTfxObject(this)\"><span class=\"expansion-marker\"></span><span class=\"class-name\">Artifact</span> of type <span class=\"class-name\">'ExternalArtifact'</span> (uri: /content/tfds/imdb_reviews/plain_text/0.1.0)<span class=\"deemphasize\"> at 0x7f4c33f67630</span></div><table class=\"attr-table\"><tr><td class=\"attr-name\">.type</td><td class = \"attrvalue\"><class 'tfx.types.standard_artifacts.ExternalArtifact'></td></tr><tr><td class=\"attr-name\">.uri</td><td class = \"attrvalue\">/content/tfds/imdb_reviews/plain_text/0.1.0</td></tr></table></div></td></tr></table></td></tr></table></div></td></tr></table></td></tr><tr><td class=\"attr-name\">.outputs</td><td class = \"attrvalue\"><table class=\"attr-table\"><tr><td class=\"attr-name\">['examples']</td><td class = \"attrvalue\"><style>\n", | |
| ".tfx-object.expanded {\n", | |
| " padding: 4px 8px 4px 8px;\n", | |
| " background: white;\n", | |
| " border: 1px solid #bbbbbb;\n", | |
| " box-shadow: 4px 4px 2px rgba(0,0,0,0.05);\n", | |
| "}\n", | |
| ".tfx-object, .tfx-object * {\n", | |
| " font-size: 11pt;\n", | |
| "}\n", | |
| ".tfx-object > .title {\n", | |
| " cursor: pointer;\n", | |
| "}\n", | |
| ".tfx-object .expansion-marker {\n", | |
| " color: #999999;\n", | |
| "}\n", | |
| ".tfx-object.expanded > .title > .expansion-marker:before {\n", | |
| " content: '▼';\n", | |
| "}\n", | |
| ".tfx-object.collapsed > .title > .expansion-marker:before {\n", | |
| " content: '▶';\n", | |
| "}\n", | |
| ".tfx-object .class-name {\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object .deemphasize {\n", | |
| " opacity: 0.5;\n", | |
| "}\n", | |
| ".tfx-object.collapsed > table.attr-table {\n", | |
| " display: none;\n", | |
| "}\n", | |
| ".tfx-object.expanded > table.attr-table {\n", | |
| " display: block;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table {\n", | |
| " border: 2px solid white;\n", | |
| " margin-top: 5px;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attr-name {\n", | |
| " vertical-align: top;\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attrvalue {\n", | |
| " text-align: left;\n", | |
| "}\n", | |
| "</style>\n", | |
| "<script>\n", | |
| "function toggleTfxObject(element) {\n", | |
| " var objElement = element.parentElement;\n", | |
| " if (objElement.classList.contains('collapsed')) {\n", | |
| " objElement.classList.remove('collapsed');\n", | |
| " objElement.classList.add('expanded');\n", | |
| " } else {\n", | |
| " objElement.classList.add('collapsed');\n", | |
| " objElement.classList.remove('expanded');\n", | |
| " }\n", | |
| "}\n", | |
| "</script>\n", | |
| "<div class=\"tfx-object collapsed\"><div class = \"title\" onclick=\"toggleTfxObject(this)\"><span class=\"expansion-marker\"></span><span class=\"class-name\">Channel</span> of type <span class=\"class-name\">'Examples'</span> (1 artifact)<span class=\"deemphasize\"> at 0x7f4c33f67a58</span></div><table class=\"attr-table\"><tr><td class=\"attr-name\">.type_name</td><td class = \"attrvalue\">Examples</td></tr><tr><td class=\"attr-name\">._artifacts</td><td class = \"attrvalue\"><table class=\"attr-table\"><tr><td class=\"attr-name\">[0]</td><td class = \"attrvalue\"><style>\n", | |
| ".tfx-object.expanded {\n", | |
| " padding: 4px 8px 4px 8px;\n", | |
| " background: white;\n", | |
| " border: 1px solid #bbbbbb;\n", | |
| " box-shadow: 4px 4px 2px rgba(0,0,0,0.05);\n", | |
| "}\n", | |
| ".tfx-object, .tfx-object * {\n", | |
| " font-size: 11pt;\n", | |
| "}\n", | |
| ".tfx-object > .title {\n", | |
| " cursor: pointer;\n", | |
| "}\n", | |
| ".tfx-object .expansion-marker {\n", | |
| " color: #999999;\n", | |
| "}\n", | |
| ".tfx-object.expanded > .title > .expansion-marker:before {\n", | |
| " content: '▼';\n", | |
| "}\n", | |
| ".tfx-object.collapsed > .title > .expansion-marker:before {\n", | |
| " content: '▶';\n", | |
| "}\n", | |
| ".tfx-object .class-name {\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object .deemphasize {\n", | |
| " opacity: 0.5;\n", | |
| "}\n", | |
| ".tfx-object.collapsed > table.attr-table {\n", | |
| " display: none;\n", | |
| "}\n", | |
| ".tfx-object.expanded > table.attr-table {\n", | |
| " display: block;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table {\n", | |
| " border: 2px solid white;\n", | |
| " margin-top: 5px;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attr-name {\n", | |
| " vertical-align: top;\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attrvalue {\n", | |
| " text-align: left;\n", | |
| "}\n", | |
| "</style>\n", | |
| "<script>\n", | |
| "function toggleTfxObject(element) {\n", | |
| " var objElement = element.parentElement;\n", | |
| " if (objElement.classList.contains('collapsed')) {\n", | |
| " objElement.classList.remove('collapsed');\n", | |
| " objElement.classList.add('expanded');\n", | |
| " } else {\n", | |
| " objElement.classList.add('collapsed');\n", | |
| " objElement.classList.remove('expanded');\n", | |
| " }\n", | |
| "}\n", | |
| "</script>\n", | |
| "<div class=\"tfx-object collapsed\"><div class = \"title\" onclick=\"toggleTfxObject(this)\"><span class=\"expansion-marker\"></span><span class=\"class-name\">Artifact</span> of type <span class=\"class-name\">'Examples'</span> (uri: /tmp/tfx-interactive-2020-06-04T20_59_02.746836-3cei_3jg/ImportExampleGen/examples/1)<span class=\"deemphasize\"> at 0x7f4c33f676d8</span></div><table class=\"attr-table\"><tr><td class=\"attr-name\">.type</td><td class = \"attrvalue\"><class 'tfx.types.standard_artifacts.Examples'></td></tr><tr><td class=\"attr-name\">.uri</td><td class = \"attrvalue\">/tmp/tfx-interactive-2020-06-04T20_59_02.746836-3cei_3jg/ImportExampleGen/examples/1</td></tr><tr><td class=\"attr-name\">.span</td><td class = \"attrvalue\">0</td></tr><tr><td class=\"attr-name\">.split_names</td><td class = \"attrvalue\">[\"train\", \"eval\"]</td></tr></table></div></td></tr></table></td></tr></table></div></td></tr></table></td></tr><tr><td class=\"attr-name\">.exec_properties</td><td class = \"attrvalue\"><table class=\"attr-table\"><tr><td class=\"attr-name\">['input_config']</td><td class = \"attrvalue\">{\n", | |
| " \"splits\": [\n", | |
| " {\n", | |
| " \"name\": \"single_split\",\n", | |
| " \"pattern\": \"*\"\n", | |
| " }\n", | |
| " ]\n", | |
| "}</td></tr><tr><td class=\"attr-name\">['output_config']</td><td class = \"attrvalue\">{\n", | |
| " \"split_config\": {\n", | |
| " \"splits\": [\n", | |
| " {\n", | |
| " \"hash_buckets\": 45,\n", | |
| " \"name\": \"train\"\n", | |
| " },\n", | |
| " {\n", | |
| " \"hash_buckets\": 5,\n", | |
| " \"name\": \"eval\"\n", | |
| " }\n", | |
| " ]\n", | |
| " }\n", | |
| "}</td></tr><tr><td class=\"attr-name\">['custom_config']</td><td class = \"attrvalue\">None</td></tr><tr><td class=\"attr-name\">['_beam_pipeline_args']</td><td class = \"attrvalue\">[]</td></tr></table></td></tr></table></div></td></tr><tr><td class=\"attr-name\">.component.inputs</td><td class = \"attrvalue\"><table class=\"attr-table\"><tr><td class=\"attr-name\">['input']</td><td class = \"attrvalue\"><style>\n", | |
| ".tfx-object.expanded {\n", | |
| " padding: 4px 8px 4px 8px;\n", | |
| " background: white;\n", | |
| " border: 1px solid #bbbbbb;\n", | |
| " box-shadow: 4px 4px 2px rgba(0,0,0,0.05);\n", | |
| "}\n", | |
| ".tfx-object, .tfx-object * {\n", | |
| " font-size: 11pt;\n", | |
| "}\n", | |
| ".tfx-object > .title {\n", | |
| " cursor: pointer;\n", | |
| "}\n", | |
| ".tfx-object .expansion-marker {\n", | |
| " color: #999999;\n", | |
| "}\n", | |
| ".tfx-object.expanded > .title > .expansion-marker:before {\n", | |
| " content: '▼';\n", | |
| "}\n", | |
| ".tfx-object.collapsed > .title > .expansion-marker:before {\n", | |
| " content: '▶';\n", | |
| "}\n", | |
| ".tfx-object .class-name {\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object .deemphasize {\n", | |
| " opacity: 0.5;\n", | |
| "}\n", | |
| ".tfx-object.collapsed > table.attr-table {\n", | |
| " display: none;\n", | |
| "}\n", | |
| ".tfx-object.expanded > table.attr-table {\n", | |
| " display: block;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table {\n", | |
| " border: 2px solid white;\n", | |
| " margin-top: 5px;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attr-name {\n", | |
| " vertical-align: top;\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attrvalue {\n", | |
| " text-align: left;\n", | |
| "}\n", | |
| "</style>\n", | |
| "<script>\n", | |
| "function toggleTfxObject(element) {\n", | |
| " var objElement = element.parentElement;\n", | |
| " if (objElement.classList.contains('collapsed')) {\n", | |
| " objElement.classList.remove('collapsed');\n", | |
| " objElement.classList.add('expanded');\n", | |
| " } else {\n", | |
| " objElement.classList.add('collapsed');\n", | |
| " objElement.classList.remove('expanded');\n", | |
| " }\n", | |
| "}\n", | |
| "</script>\n", | |
| "<div class=\"tfx-object collapsed\"><div class = \"title\" onclick=\"toggleTfxObject(this)\"><span class=\"expansion-marker\"></span><span class=\"class-name\">Channel</span> of type <span class=\"class-name\">'ExternalArtifact'</span> (1 artifact)<span class=\"deemphasize\"> at 0x7f4c33f675c0</span></div><table class=\"attr-table\"><tr><td class=\"attr-name\">.type_name</td><td class = \"attrvalue\">ExternalArtifact</td></tr><tr><td class=\"attr-name\">._artifacts</td><td class = \"attrvalue\"><table class=\"attr-table\"><tr><td class=\"attr-name\">[0]</td><td class = \"attrvalue\"><style>\n", | |
| ".tfx-object.expanded {\n", | |
| " padding: 4px 8px 4px 8px;\n", | |
| " background: white;\n", | |
| " border: 1px solid #bbbbbb;\n", | |
| " box-shadow: 4px 4px 2px rgba(0,0,0,0.05);\n", | |
| "}\n", | |
| ".tfx-object, .tfx-object * {\n", | |
| " font-size: 11pt;\n", | |
| "}\n", | |
| ".tfx-object > .title {\n", | |
| " cursor: pointer;\n", | |
| "}\n", | |
| ".tfx-object .expansion-marker {\n", | |
| " color: #999999;\n", | |
| "}\n", | |
| ".tfx-object.expanded > .title > .expansion-marker:before {\n", | |
| " content: '▼';\n", | |
| "}\n", | |
| ".tfx-object.collapsed > .title > .expansion-marker:before {\n", | |
| " content: '▶';\n", | |
| "}\n", | |
| ".tfx-object .class-name {\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object .deemphasize {\n", | |
| " opacity: 0.5;\n", | |
| "}\n", | |
| ".tfx-object.collapsed > table.attr-table {\n", | |
| " display: none;\n", | |
| "}\n", | |
| ".tfx-object.expanded > table.attr-table {\n", | |
| " display: block;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table {\n", | |
| " border: 2px solid white;\n", | |
| " margin-top: 5px;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attr-name {\n", | |
| " vertical-align: top;\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attrvalue {\n", | |
| " text-align: left;\n", | |
| "}\n", | |
| "</style>\n", | |
| "<script>\n", | |
| "function toggleTfxObject(element) {\n", | |
| " var objElement = element.parentElement;\n", | |
| " if (objElement.classList.contains('collapsed')) {\n", | |
| " objElement.classList.remove('collapsed');\n", | |
| " objElement.classList.add('expanded');\n", | |
| " } else {\n", | |
| " objElement.classList.add('collapsed');\n", | |
| " objElement.classList.remove('expanded');\n", | |
| " }\n", | |
| "}\n", | |
| "</script>\n", | |
| "<div class=\"tfx-object collapsed\"><div class = \"title\" onclick=\"toggleTfxObject(this)\"><span class=\"expansion-marker\"></span><span class=\"class-name\">Artifact</span> of type <span class=\"class-name\">'ExternalArtifact'</span> (uri: /content/tfds/imdb_reviews/plain_text/0.1.0)<span class=\"deemphasize\"> at 0x7f4c33f67630</span></div><table class=\"attr-table\"><tr><td class=\"attr-name\">.type</td><td class = \"attrvalue\"><class 'tfx.types.standard_artifacts.ExternalArtifact'></td></tr><tr><td class=\"attr-name\">.uri</td><td class = \"attrvalue\">/content/tfds/imdb_reviews/plain_text/0.1.0</td></tr></table></div></td></tr></table></td></tr></table></div></td></tr></table></td></tr><tr><td class=\"attr-name\">.component.outputs</td><td class = \"attrvalue\"><table class=\"attr-table\"><tr><td class=\"attr-name\">['examples']</td><td class = \"attrvalue\"><style>\n", | |
| ".tfx-object.expanded {\n", | |
| " padding: 4px 8px 4px 8px;\n", | |
| " background: white;\n", | |
| " border: 1px solid #bbbbbb;\n", | |
| " box-shadow: 4px 4px 2px rgba(0,0,0,0.05);\n", | |
| "}\n", | |
| ".tfx-object, .tfx-object * {\n", | |
| " font-size: 11pt;\n", | |
| "}\n", | |
| ".tfx-object > .title {\n", | |
| " cursor: pointer;\n", | |
| "}\n", | |
| ".tfx-object .expansion-marker {\n", | |
| " color: #999999;\n", | |
| "}\n", | |
| ".tfx-object.expanded > .title > .expansion-marker:before {\n", | |
| " content: '▼';\n", | |
| "}\n", | |
| ".tfx-object.collapsed > .title > .expansion-marker:before {\n", | |
| " content: '▶';\n", | |
| "}\n", | |
| ".tfx-object .class-name {\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object .deemphasize {\n", | |
| " opacity: 0.5;\n", | |
| "}\n", | |
| ".tfx-object.collapsed > table.attr-table {\n", | |
| " display: none;\n", | |
| "}\n", | |
| ".tfx-object.expanded > table.attr-table {\n", | |
| " display: block;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table {\n", | |
| " border: 2px solid white;\n", | |
| " margin-top: 5px;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attr-name {\n", | |
| " vertical-align: top;\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attrvalue {\n", | |
| " text-align: left;\n", | |
| "}\n", | |
| "</style>\n", | |
| "<script>\n", | |
| "function toggleTfxObject(element) {\n", | |
| " var objElement = element.parentElement;\n", | |
| " if (objElement.classList.contains('collapsed')) {\n", | |
| " objElement.classList.remove('collapsed');\n", | |
| " objElement.classList.add('expanded');\n", | |
| " } else {\n", | |
| " objElement.classList.add('collapsed');\n", | |
| " objElement.classList.remove('expanded');\n", | |
| " }\n", | |
| "}\n", | |
| "</script>\n", | |
| "<div class=\"tfx-object collapsed\"><div class = \"title\" onclick=\"toggleTfxObject(this)\"><span class=\"expansion-marker\"></span><span class=\"class-name\">Channel</span> of type <span class=\"class-name\">'Examples'</span> (1 artifact)<span class=\"deemphasize\"> at 0x7f4c33f67a58</span></div><table class=\"attr-table\"><tr><td class=\"attr-name\">.type_name</td><td class = \"attrvalue\">Examples</td></tr><tr><td class=\"attr-name\">._artifacts</td><td class = \"attrvalue\"><table class=\"attr-table\"><tr><td class=\"attr-name\">[0]</td><td class = \"attrvalue\"><style>\n", | |
| ".tfx-object.expanded {\n", | |
| " padding: 4px 8px 4px 8px;\n", | |
| " background: white;\n", | |
| " border: 1px solid #bbbbbb;\n", | |
| " box-shadow: 4px 4px 2px rgba(0,0,0,0.05);\n", | |
| "}\n", | |
| ".tfx-object, .tfx-object * {\n", | |
| " font-size: 11pt;\n", | |
| "}\n", | |
| ".tfx-object > .title {\n", | |
| " cursor: pointer;\n", | |
| "}\n", | |
| ".tfx-object .expansion-marker {\n", | |
| " color: #999999;\n", | |
| "}\n", | |
| ".tfx-object.expanded > .title > .expansion-marker:before {\n", | |
| " content: '▼';\n", | |
| "}\n", | |
| ".tfx-object.collapsed > .title > .expansion-marker:before {\n", | |
| " content: '▶';\n", | |
| "}\n", | |
| ".tfx-object .class-name {\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object .deemphasize {\n", | |
| " opacity: 0.5;\n", | |
| "}\n", | |
| ".tfx-object.collapsed > table.attr-table {\n", | |
| " display: none;\n", | |
| "}\n", | |
| ".tfx-object.expanded > table.attr-table {\n", | |
| " display: block;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table {\n", | |
| " border: 2px solid white;\n", | |
| " margin-top: 5px;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attr-name {\n", | |
| " vertical-align: top;\n", | |
| " font-weight: bold;\n", | |
| "}\n", | |
| ".tfx-object table.attr-table td.attrvalue {\n", | |
| " text-align: left;\n", | |
| "}\n", | |
| "</style>\n", | |
| "<script>\n", | |
| "function toggleTfxObject(element) {\n", | |
| " var objElement = element.parentElement;\n", | |
| " if (objElement.classList.contains('collapsed')) {\n", | |
| " objElement.classList.remove('collapsed');\n", | |
| " objElement.classList.add('expanded');\n", | |
| " } else {\n", | |
| " objElement.classList.add('collapsed');\n", | |
| " objElement.classList.remove('expanded');\n", | |
| " }\n", | |
| "}\n", | |
| "</script>\n", | |
| "<div class=\"tfx-object collapsed\"><div class = \"title\" onclick=\"toggleTfxObject(this)\"><span class=\"expansion-marker\"></span><span class=\"class-name\">Artifact</span> of type <span class=\"class-name\">'Examples'</span> (uri: /tmp/tfx-interactive-2020-06-04T20_59_02.746836-3cei_3jg/ImportExampleGen/examples/1)<span class=\"deemphasize\"> at 0x7f4c33f676d8</span></div><table class=\"attr-table\"><tr><td class=\"attr-name\">.type</td><td class = \"attrvalue\"><class 'tfx.types.standard_artifacts.Examples'></td></tr><tr><td class=\"attr-name\">.uri</td><td class = \"attrvalue\">/tmp/tfx-interactive-2020-06-04T20_59_02.746836-3cei_3jg/ImportExampleGen/examples/1</td></tr><tr><td class=\"attr-name\">.span</td><td class = \"attrvalue\">0</td></tr><tr><td class=\"attr-name\">.split_names</td><td class = \"attrvalue\">[\"train\", \"eval\"]</td></tr></table></div></td></tr></table></td></tr></table></div></td></tr></table></td></tr></table></div>" | |
| ], | |
| "text/plain": [ | |
| "ExecutionResult(\n", | |
| " component_id: ImportExampleGen\n", | |
| " execution_id: 1\n", | |
| " outputs:\n", | |
| " examples: Channel(\n", | |
| " type_name: Examples\n", | |
| " artifacts: [Artifact(type_name: Examples, uri: /tmp/tfx-interactive-2020-06-04T20_59_02.746836-3cei_3jg/ImportExampleGen/examples/1, id: 2)]\n", | |
| " ))" | |
| ] | |
| }, | |
| "metadata": { | |
| "tags": [] | |
| }, | |
| "execution_count": 8 | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "iu2ejZTWK-E5", | |
| "colab_type": "code", | |
| "outputId": "ca0112fa-f52d-431f-d052-0e0897f49c94", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 51 | |
| } | |
| }, | |
| "source": [ | |
| "%%skip_for_export\n", | |
| "\n", | |
| "for artifact in example_gen.outputs['examples'].get():\n", | |
| " print(artifact.uri)" | |
| ], | |
| "execution_count": 0, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "/tmp/tfx-interactive-2020-06-04T20_59_02.746836-3cei_3jg/ImportExampleGen/examples/1\n", | |
| "This cell will be skipped during export to pipeline.\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "nE9VL-pmF6L_", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "## TensorFlow Data Validation" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "EglytaKVLQzr", | |
| "colab_type": "code", | |
| "outputId": "384f0961-dcbd-44e7-a310-8bfb002624c7", | |
| "colab": { | |
| "base_uri": "https://localhost:8080/", | |
| "height": 993 | |
| } | |
| }, | |
| "source": [ | |
| "%%skip_for_export\n", | |
| "\n", | |
| "statistics_gen = StatisticsGen(\n", | |
| " examples=example_gen.outputs['examples'])\n", | |
| "context.run(statistics_gen)\n", | |
| "\n", | |
| "context.show(statistics_gen.outputs['statistics'])" | |
| ], | |
| "execution_count": 0, | |
| "outputs": [ | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "text/html": [ | |
| "<b>Artifact at /tmp/tfx-interactive-2020-06-04T20_59_02.746836-3cei_3jg/StatisticsGen/statistics/2</b><br/><br/>" | |
| ], | |
| "text/plain": [ | |
| "<IPython.core.display.HTML object>" | |
| ] | |
| }, | |
| "metadata": { | |
| "tags": [] | |
| } | |
| }, | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "text/html": [ | |
| "<div><b>'train' split:</b></div><br/><br/>" | |
| ], | |
| "text/plain": [ | |
| "<IPython.core.display.HTML object>" | |
| ] | |
| }, | |
| "metadata": { | |
| "tags": [] | |
| } | |
| }, | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "text/html": [ | |
| "<iframe id='facets-iframe' width=\"100%\" height=\"500px\"></iframe>\n", | |
| " <script>\n", | |
| " facets_iframe = document.getElementById('facets-iframe');\n", |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment