diff --git a/README.md b/README.md index 6a87e1e..73bf1db 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,105 @@ # trace-server-protocol -Specification of the Trace Server Protocol - -This protocol is built to decouple the backend and frontend of trace analysers, allowing traces to reside and be analysed on the backend, and visual models to be exchanged with a variety of clients. - -The protocol is meant to be RESTful, over HTTP. +Specification of the Trace Server Protocol (TSP). The specification is currently written in **OpenAPI 3.0** and can be pretty-visualized in the [github pages][tspGhPages]. **👋 Want to help?** Read our [contributor guide][contributing]. +## About, why and goals of the TSP: + +Similarly to the philosophy behind the Language Server Protocol (LSP), +the TSP is an open, HTTP-based protocol for use between *analytics or +interactive visualization web applications* and *servers that provide +trace analysis specific features* such as: + +- generation of data that can interpeted as charts from trace analysis +- selection, filtering and correlation of elements in charts using data + contained in the traces +- statistics calculation from data contained in the traces + +The goal of the protocol is to allow trace analysis support to be +implemented and distributed independently of any given analytics or +interactive visualization web application. + +## What the TSP is: + +The TSP is a RESTful API on top of the HTTP protocol that enables to: + +- analyze time series data (e.g. traces and logs); +- return the result of the analysis in form of data that represents + *charts* or *charts components*; +- *navigate (or explore)* through the returned data; + +Summarizing: + +> the TSP is an API to explore data and navigate charts that has been +> generated from the interpretation of time series data. + +With the term *chart* we mean different type of visualizations such as +XY charts, timegraph charts, tables, pie charts, etc. + +With the term *chart components* we mean different elements that +can be used to build or enrich charts such as arrows, annotations, +bookmarks, etc. + +With the terms *exploration and navigation* we mean interactions with +the charts, such as: + +- **filter** specific components of the charts. For example: + - remove from the chart all components that do not match a condition; + - return components of the chart that do match a condition; +- **select** a subset of all data points of a chart. For example: + - given a Timegraph chart, return only the data between a specific + time interval +- **select** specific components of the charts. For example: + - when analyzing the trace of a SW execution, select only some + components of the chart that represent specific SW functions; + - when analyzing the trace of a SW execution, select only some + components of the chart executed on specific CPUs; + - given a Timegraph chart, automatically select areas of the chart + where the components match a specific condition (**zoom-on-crash-area**) +- **correlate** components between charts. For example: + - given 2 charts generated by the analysis of the same time series data, + it enables to correlate the chart components that represent the same + information. For example, given a table containing the max length + of an interval in a time chart, it enables to identify the corresponding + interval in the time chart (**go-to-min/max**) +- **correlate** chart components and their information. For example: + - given a interval in a time chart, extrapolate all information related + to that interval (e.g. **source location**) +- **order** specific parts of the charts. For example: + - return the **N longest** intervals of a timegraph chart; +- **customize** charts and charts components. For example: + - given a table containing statistics from analysis of all input data, + generate a new table containing statistics from analysis of a + subset of the input data; + - given a time chart where intervals represent thread execution over + time, and they are connected using arrows, generate a new time chart + showing the longest path of connected intervals (e.g. **critical path**) + +## Guidelines for the TSP implementation + +The main resources (endpoints) of the API should be: + +- **data sources** (e.g. time series data such as traces and logs). These + are analyzed and used to create the data that can be retrieved; +- **charts** (e.g. tables, XY charts), used to retrieve data representing + a chart; +- **chart components** (e.g. annotations, styles), used to retrieve data + representing specific information for a chart; + +The *exploration and navigation* should be achieved by: + +- parameters in the HTTP message body data (usually when POST-ing), or + as query strings (usually when GET-ing). These parameters should be: + - **data source related**. For example, considering that data sources + have a strong relation to time, a classic parameter can be "time" + related (e.g. *requested_timerange*); + - **chart related**. For example, when "talking" to a resource representing + an XY chart, a parameter can be related to "XY dimensions" (e.g. + *requested_Y_elements*); + ## Current version The current version of the specification is currently implemented and supported in the [Trace Compass trace-server][tcServer] (reference implementation) and what is currently supported by the [tsp-typescript-client][tspClient]. diff --git a/TSP-examples.md b/TSP-examples.md new file mode 100755 index 0000000..1ff179f --- /dev/null +++ b/TSP-examples.md @@ -0,0 +1,120 @@ +# How to follow TSP guidelines + +This file/document will be removed. It is meant to give some examples +of how to follow the TSP guidelines when extending the TSP. + +NOTE: + +- the examples below are just pseudocode to try to give a concrete + feeling on how to implement the exploration/navigation features and + follow the TSP guidelines. They are not meant to be seen as concrete + proposals to extend/modify the existing TSP; +- the GET / POST methods are chosen just to be more RESTful (i.e. try + to not use a POST when we do not create new resources) +- the parameters in the requests are always written as body params, + just to make it easier to write. But they do not have to be body params. + +## Timegraph states (a.k.a. intervals) + +``` +GET tsp/api/experiments/{expUUID}/outputs/timeGraph/{outputId}/states +{"parameters": + {"requested_timerange": + {"start": 111111111, + "end": 222222222, + "nbTimes": 1920}, // to follow "TSP guidelines", would it be better "precision"? or "samples"? + "requested_items": [1,2] // to follow "TSP guidelines", would it be better "requested_row_ids"? + } +} +``` + +## Filter rows of a timegraph + +``` +GET tsp/api/experiments/{expUUID}/outputs/timeGraph/{outputId}/states +{"parameters": + {"requested_timerange": + {"start": 111111111, + "end": 222222222, + "nbTimes": 1920}, // to follow "TSP guidelines", would it be better "precision"? or "samples"? + "requested_row_labels": + [CPU0,CPU2] // this is the label of the rows. How does the cli know about name of rows? + // The cli "asks" to the server about the labels of rows in the chart (e.g. + // from the timegraph/tree endpoint) + } +} +``` + +## Filter states (a.k.a. intervals) of a timegraph chart + +``` +GET tsp/api/experiments/{expUUID}/outputs/timeGraph/{outputId}/states +{"parameters": + {"requested_timerange": + {"start": 111111111, + "end": 222222222, + "nbTimes": 1982}, + "requested_intervals": + [ThreadA*,FunctionB*,BankTransactionC*] + } +} +``` + +## Filter states (a.k.a. intervals) of the chart with fullsearch + +``` +GET tsp/api/experiments/{expUUID}/outputs/timeGraph/{outputId}/states +{"parameters": + {"requested_timerange": + {"start": 111111111, + "end": 222222222, + "nbTimes": inf}, // or "max", however give the idea that we are trying to get all possible samples + "requested_intervals": + [ThreadA*,FunctionB*] + } +} +``` + +## Correlate components between charts (e.g. go-to-max) + +An example of how to jump from a value in a table to the interval/state +that it represents in another chart. + +First, ask some info to the table: + +``` +cli (ask): GET tsp/api/experiments/{expUUID}/outputs///tree + {"parameters":{"requested_times":[0,100000000]}} // Side question: why is "requested_times" needed? +srv (ret): {"headers":[{Min},{Max},...], "entries" :[{1 sec},{2 min},...]} +``` + +In order to implement the "go-to-max" functionality, the client asks to the server +to return the "time series data" (in this case an interval) that was used to calculate +the value "2 min" at the table index 1,1: + +``` +cli (ask): GET tsp/api/experiments/{expUUID}/outputs///tree + {"parameters":{"table_row": 1, "table_col":1]}} +srv (ret): {"data":[{"start": 1234, "end": 2345, "label": "ThreadA"}]} +``` + +The client uses than that info to "go to max" i.e. zoom in, filter or whaterver. +In theory it could also be possible to enable the server to "auto-align" other charts, +not sure if it is a good idea. + +## Customize (e.g. select a subset of the input data to use to create a new chart) + +Cli asks to the "time series input data" (e.g. the event table) to return some info on the input data: + +``` +cli (ask): GET tsp/api/experiments/{expUUID}/outputs/table//columns +srv (ret): {"model":[{"name": "timestamp", "type":"number", ...}, {"name": "Device", "type":"string", ...}, ...]} +``` + +Cli asks to create a new statistics table using a subset of the input data. +The subset is selected using some of the previously returned params. + +``` +cli (ask): POST tsp/api/experiments/{expUUID}/outputs/ + {"parameters":{"outputId/chart-id":"my.custom.chart", "include":[{"Device": "CPU0"},...]}} +```