In order to have a flexible schema that will be easy to maintain, the proposed approach is to save a Dashboard as a String that stores a JSON document that can be parsed by GraphQL and at the end by our frontend.
The index representation is the following one:
curl -k -XPUT --silent "http://localhost:9200/dashboards_v1?pretty" -H 'Content-Type: application/json' -d '
{
"settings": {
"number_of_shards": 2,
"number_of_replicas": 1
},
"mappings": {
"dashboard": {
"properties": {
"name": {
"type": "keyword"
},
"owner_id": {
"type": "keyword"
},
"elements": {
"type": "text"
},
"created_at": {
"type": "date",
"format" : "yyyy-MM-dd HH:mm:ss"
},
"updated_at": {
"type": "date",
"format" : "yyyy-MM-dd HH:mm:ss"
}
}
}
}
}
'
curl -k -XPOST --silent "http://localhost:5001/_aliases?pretty" -H 'Content-Type: application/json' -d '
{
"actions" : [
{"add": {"index": "dashboards_v1", "alias": "dashboards"}}
}'The full representation of a Dashboard will be stored in the elements field.
A GraphQL layer is proposed in order to return dashboard's data and to parse a portion of the JSON document in order to extract the visual elements and allow the client to query them as necessary.
The GraphQL schema is the following one:
DASHBOARD {
ownerId: ID
name: String
createdAt: DateTime
updatedAt: DateTime
elements: [DASHBOARD_ELEMENT]
}
DASHBOARD_ELEMENT {
category: String
type: String
distribution: ElementDistribution
schema: String
elements: [DASHBOARD_ELEMENT]
}
ELEMENT_DISTRIBUTION {
width: Int
position: Int
}A Dashboard is a type the representation of a dashboard and it can contain any number of Dashboard Elements.
A Dashboard Element is a type that represents a visual element attached to the a Dashboard, each one has a category and a type that can help to organize the types of dashboard elements that we have, a first approach to categorize the elements can be the following one:
- Categories
- Insight
- Types
- Pie
- Line
- Bar
- Table
- Treemap
- Types
- Text
- Types
- Paragraph
- Title
- Subtitle
- Types
- Separator
- Types
- Horizontal
- Types
- Insight
The distribution field contains information about the current position of the element over a Dashboard along with the width which can be stored as a number that represents the number of columns of a grid that the element occupies (this is just a proposal).
The schema field is a String that stores a JSON document that represents what is needed in order to display the visual element. In the case of a Chart it contains the configuration and filters used to generate the chart. In the case of a text it contains the content of the text and in the future it can contain more values to represent things like styles. This schema is not meant to be parsed by GraphQL because each Dashboard Element may need different information in order to be rendered and we cannot know ahead of time this information to make a fixed GraphqQL schema.
The elements field is an array of Dashboard Element because in the future we may want to associate elements with other elements like in the case of a Chart that has an associated text and they should be rendered as one element on a Dashboard.
An Element Distribution is a type that represents how a Dashboard Elementìs distributed over a Dashboard in terms of position and width.
A GraphQL schema like this one, will allow the client to query the following data:
query {
dashboard {
id
ownerId
name
elements {
category
type
distribution {
width
position
}
elements {
category
type
distribution {
width
position
}
schema
}
schema
}
}
}The nested selection of the elements field allows us to retrieve visual elements that contains inner elements such as a Chart that has an associated text and they should be rendered together as one Dashboard Element. This logic can be applied to more elements in the future.
Example output of the previous query:
{
"data": {
"dashboard": {
"elements": [
{
"category": "text",
"distribution": {
"position": 1,
"width": 6
},
"elements": [],
"schema": "{\"content\": \"Incremento de asaltos\"}",
"type": "subtitle"
},
{
"category": "text",
"distribution": {
"position": 2,
"width": 24
},
"elements": [],
"schema": "{\"content\": \"Los asaltos han ido incrementando en los estados de ...\"}",
"type": "paragraph"
},
{
"category": "chart",
"distribution": {
"position": 3,
"width": 12
},
"elements": [
{
"category": "text",
"distribution": {
"position": null,
"width": null
},
"schema": "{\"content\": \"Gr\\u00e1fica de incremento en asaltos\"}",
"type": "paragraph"
}
],
"schema": "{\"configuration\": {\"sdlocs\": \"country:c68c4d2a-b34e-4a22-a2da-d71bceaa1944,state:b161863c-39fa-4b04-8e11-137f8d9443e8\", \"yop\": \"count\", \"yaxis\": \"ccat\", \"interval\": \"year\", \"xaxis\": \"idt\"}, \"filters\": {\"sdt\": \"2018-01-01\", \"edt\": \"2018-05-01\", \"source\": \"internal\"}}",
"type": "line"
},
{
"category": "chart",
"distribution": {
"position": 4,
"width": 12
},
"elements": [
{
"category": "text",
"distribution": {
"position": null,
"width": null
},
"schema": "{\"content\": \"Gr\\u00e1fica de incremento en asaltos\"}",
"type": "paragraph"
}
],
"schema": "{\"configuration\": {\"sdlocs\": \"country:c68c4d2a-b34e-4a22-a2da-d71bceaa1944,state:b161863c-39fa-4b04-8e11-137f8d9443e8\", \"yop\": \"count\", \"yaxis\": \"ccat\", \"interval\": \"year\", \"xaxis\": \"idt\"}, \"filters\": {\"sdt\": \"2018-01-01\", \"edt\": \"2018-05-01\", \"source\": \"internal\"}}",
"type": "bar"
},
{
"category": "separator",
"distribution": {
"position": 5,
"width": 24
},
"elements": [],
"schema": null,
"type": "horizontal"
},
{
"category": "chart",
"distribution": {
"position": 6,
"width": 24
},
"elements": [],
"schema": "{\"configuration\": {\"xdcrimes\": \"category:kidnapping,category:homicide,category:extortion,category:car_theft\", \"yop\": \"count\", \"yaxis\": \"ccat\", \"chart\": \"pie\"}, \"filters\": {\"source\": \"secretariat\"}}",
"type": "pie"
}
],
"id": "uSqgjmYB1C2l0t2yPeIS",
"name": "Asaltos 2018",
"ownerId": "ksj824ba1j1330"
}
}
}In order to store a Dashboard that looks like this:
The following request can be made to Elasticsearch (http://localhost:9200/dashboards/dashboard/):
{
"name": "Asaltos 2018",
"owner_id": "ksj824ba1j1330",
"created_at": "2018-01-01 00:00:00",
"updated_at": "2018-01-05 00:00:00",
"elements": "[{\"category\":\"text\",\"type\":\"subtitle\",\"distribution\":{\"position\":1,\"width\":6},\"schema\":{\"content\":\"Incremento de asaltos\"}},{\"category\":\"text\",\"type\":\"paragraph\",\"distribution\":{\"position\":2,\"width\":24},\"schema\":{\"content\":\"Los asaltos han ido incrementando en los estados de ...\"}},{\"category\":\"chart\",\"type\":\"line\",\"distribution\":{\"position\":3,\"width\":12},\"schema\":{\"configuration\":{\"xaxis\":\"idt\",\"interval\":\"year\",\"sdlocs\":\"country:c68c4d2a-b34e-4a22-a2da-d71bceaa1944,state:b161863c-39fa-4b04-8e11-137f8d9443e8\",\"yaxis\":\"ccat\",\"yop\":\"count\"},\"filters\":{\"source\":\"internal\",\"sdt\":\"2018-01-01\",\"edt\":\"2018-05-01\"}},\"elements\":[{\"category\":\"text\",\"type\":\"paragraph\",\"schema\":{\"content\":\"Gráfica de incremento en asaltos\"}}]},{\"category\":\"chart\",\"type\":\"bar\",\"distribution\":{\"position\":4,\"width\":12},\"schema\":{\"configuration\":{\"xaxis\":\"idt\",\"interval\":\"year\",\"sdlocs\":\"country:c68c4d2a-b34e-4a22-a2da-d71bceaa1944,state:b161863c-39fa-4b04-8e11-137f8d9443e8\",\"yaxis\":\"ccat\",\"yop\":\"count\"},\"filters\":{\"source\":\"internal\",\"sdt\":\"2018-01-01\",\"edt\":\"2018-05-01\"}},\"elements\":[{\"category\":\"text\",\"type\":\"paragraph\",\"schema\":{\"content\":\"Gráfica de incremento en asaltos\"}}]},{\"category\":\"separator\",\"type\":\"horizontal\",\"distribution\":{\"position\":5,\"width\":24}},{\"category\":\"chart\",\"type\":\"pie\",\"distribution\":{\"position\":6,\"width\":24},\"schema\":{\"configuration\":{\"chart\":\"pie\",\"xdcrimes\":\"category:kidnapping,category:homicide,category:extortion,category:car_theft\",\"yaxis\":\"ccat\",\"yop\":\"count\"},\"filters\":{\"source\":\"secretariat\"}}}]"
}This Dashboard corresponds to the following JS Object:
{
"name": "Asaltos 2018",
"ownerId": "ksj824ba1j1330",
"createdAt": "2018-01-01 00:00:00",
"updatedAt": "2018-01-05 00:00:00",
"elements": [
{
"category": "text",
"type": "subtitle",
"distribution": {
"position": 1,
"width": 6
},
"schema": {
"content": "Incremento de asaltos"
}
},
{
"category": "text",
"type": "paragraph",
"distribution": {
"position": 2,
"width": 24
},
"schema": {
"content": "Los asaltos han ido incrementando en los estados de ..."
}
},
{
"category": "chart",
"type": "line",
"distribution": {
"position": 3,
"width": 12
},
"schema": {
"configuration": {
"xaxis": "idt",
"interval": "year",
"sdlocs": "country:c68c4d2a-b34e-4a22-a2da-d71bceaa1944,state:b161863c-39fa-4b04-8e11-137f8d9443e8",
"yaxis": "ccat",
"yop": "count"
},
"filters": {
"source": "internal",
"startDate": "2018-01-01",
"endDate": "2018-05-01"
}
},
"elements": [
{
"category": "text",
"type": "paragraph",
"schema": {
"content": "Gráfica de incremento en asaltos"
}
}
]
},
{
"category": "chart",
"type": "bar",
"distribution": {
"position": 4,
"width": 12
},
"schema": {
"configuration": {
"xaxis": "idt",
"interval": "year",
"sdlocs": "country:c68c4d2a-b34e-4a22-a2da-d71bceaa1944,state:b161863c-39fa-4b04-8e11-137f8d9443e8",
"yaxis": "ccat",
"yop": "count"
},
"filters": {
"source": "internal",
"startDate": "2018-01-01",
"endDate": "2018-05-01"
}
},
"elements": [
{
"category": "text",
"type": "paragraph",
"schema": {
"content": "Gráfica de incremento en asaltos"
}
}
]
},
{
"category": "separator",
"type": "horizontal",
"distribution": {
"position": 5,
"width": 24
}
},
{
"category": "chart",
"type": "pie",
"distribution": {
"position": 6,
"width": 24
},
"schema": {
"configuration": {
"chart": "pie",
"xdcrimes": "category:kidnapping,category:homicide,category:extortion,category:car_theft",
"yaxis": "ccat",
"yop": "count"
},
"filters": {
"source": "secretariat"
}
}
}
]
}This show us a how a Dashboard can look like while is being use in the frontend, fully parsed and ready to use.
An implementation of the GraphQL schema in python along with an example of how to retrieve a stored Dashboard in ES that follows the index described above can be found here, this should be considered only a demonstration of the capabilities of the described spec and it's a WIP.
A very basic implementation of how to display a stored Dashboard returned by the API with the query shown above can be found here, this should considered only a demonstration of the capabilities of the described spec and it's a WIP.
Thanks! I like the idea, lets do it this way, as you say there will be only a limited set of nested elements, whatever works best for us is good for me 👍