data-transformation.md

Examples

TL;DR

The schema will be stable in a single production deployment. Schema changes will occur between production deployments when:

• A new property to an existing event type is added, a new column is created in VAC.
• A new event type is added, a new table is created in VAC.

Changes can be approved by legal before they land in production VAC. Schema changes are not uncontrolled and therefore are stable.

Request Example

{
  "envelope": {
    "product_name": "pks"
  },
  "payload": {
    "type": "create_cluster"
    "data": {
      "hashed_user_id": "b6edf1531bf95b5a12fa415403b39ce8"
    }
  }
}

Constraints

• Must contain the key envelope which is an object.
• Must contain the key payload which contains keys type (string) and data (object).

1. Adding a new envelope property

• First Request

{
  "envelope": {
    "product_name": "pks"
  },
  "payload": {
    "type": "create_cluster"
    "data": {
      "hashed_user_id": "b6edf1531bf95b5a12fa415403b39ce8"
    }
  }
}

• Second Request

{
  "envelope": {
    "product_name": "pks",
    "product_version": "v1.0.1" # <------- New property
  },
  "payload": {
    "type": "create_cluster"
    "data": {
      "hashed_user_id": "b6edf1531bf95b5a12fa415403b39ce8"
    }
  }
}

We expect the following in VAC:

• A new column product_version is added to an existing table.
• No new tables are added.

2. Adding a new payload property

• First Request

{
  "envelope": {
    "product_name": "pks"
  },
  "payload": {
    "type": "create_cluster"
    "data": {
      "hashed_user_id": "b6edf1531bf95b5a12fa415403b39ce8"
    }
  }
}

• Second Request

{
  "envelope": {
    "product_name": "pks"
  },
  "payload": {
    "type": "create_cluster"
    "data": {
      "plan_id": "small", # <------- New property
      "hashed_user_id": "b6edf1531bf95b5a12fa415403b39ce8"
    }
  }
}

We expect the following in VAC:

• Add new column plan_id is added to the create_cluster table.
• No new tables are added.

3. Adding a new event type

• First Request

{
  "envelope": {
    "product_name": "pks"
  },
  "payload": {
    "type": "create_cluster"
    "data": {
      "hashed_user_id": "b6edf1531bf95b5a12fa415403b39ce8"
    }
  }
}

• Second Request

{
  "envelope": {
    "product_name": "pks"
  },
  "payload": {
    "type": "delete_cluster" # <------- New type
    "data": {
      "hashed_user_id": "b6edf1531bf95b5a12fa415403b39ce8"
    }
  }
}

We expect the following in VAC:

• New table called delete_cluster gets created

Swagger spec

paths:
  /api/send:
    post:
      description: "## send a generic payload to VAC via TM Server\n"
      operationId: "sendPayload"
      consumes:
      - "application/json"
      produces:
      - "text/plain"
      parameters:
      - in: "body"
        name: "body"
        required: true
        schema:
          $ref: "#/definitions/Event"
        x-exportParamName: "Body"
      responses:
        201:
          description: "entry created"
        default:
          description: "Error sending payload"
definitions:
  Event:
    type: "object"
    required:
    - "envelope"
    - "payload"
    properties:
      envelope:
        type: "object"
        additionalProperties:
          type: string
      payload:
        type: "object"
        properties:
          type:
            type: "string"
          data:
            type: "object"
            additionalProperties:
              type: string
    example:
      envelope:
        product_name: "PKS"
        product_version: "v1.0.1"
      payload:
        type: "create_cluster"
        data:
          hashed_user_id: "b6edf1531bf95b5a12fa415403b39ce8"
          plan_id: "small"
          timestamp: "2018-03-07T13:13:20+00:00"