Merge pull request #22 from qlik-oss/table-support

Return table from plugin

CHANGELOG.md

@@ -0,0 +1,36 @@
+# Previous versions
+The table maps all released SSE versions with the appropriate Qlik versions. See the Changelog below for more details of each release.
+
+| __SSE Version__ | __Qlik Sense Version__ | __QlikView Version__ |
+| ----- | ----- | ----- |
+| [v1.1.0](https://github.com/qlik-oss/server-side-extension/releases/tag/v1.1.0) | Qlik Sense February 2018 (or later) | No supported version |
+| [v1.0.0](https://github.com/qlik-oss/server-side-extension/releases/tag/v1.0.0) | Qlik Sense June 2017 (or later) | QlikView November 2017 (or later) |
+
+# Changelog
+
+## v1.1.0
+Release date: 2017-01-16
+
+v1.1.0 is _backwards compatible_ with version v1.0.0, meaning a client and server can have different SSE versions and still use all functionality supported in v1.0.0.
+
+Qlikview does not support the functionality of table load using SSE (`Load ... Extension ...`, see the [Qlik Sense Help](http://help.qlik.com/en-US/sense/February2018/Subsystems/Hub/Content/Scripting/ScriptRegularStatements/Load.htm)) and there is not yet a plan to support it. But because of the backwards compatability, Qlikview can connect to SSEs of version v1.1.0 without changing anything in the QlikView deployment, given that the new features in v1.1.0 are not used.
+
+### New Features:  
+[#22](https://github.com/qlik-oss/server-side-extension/pull/22) : `TableDescription` (and `FieldDescription`), send metadata of the table and fields returned to Qlik during data load. See [documentation](docs/writing_a_plugin.md#tabledescription). Both script examples written in python are updated with possibility to return multiple columns.
+
+### Bugfixes:  
+[#23](https://github.com/qlik-oss/server-side-extension/issues/23) : Docker files had wrong Python version specified
+
+### Notable Changes:
+[#17](https://github.com/qlik-oss/server-side-extension/pull/17) : Run Python examples in docker  
+[#22](https://github.com/qlik-oss/server-side-extension/pull/22) : Python script example added, now using _pandas_ and _exec_ libraries. See commit [here](https://github.com/qlik-oss/server-side-extension/commit/f27f0e33270f6d9ec96cf11c7530e6b281c83306)  
+[#22](https://github.com/qlik-oss/server-side-extension/pull/22) : C# example added. See commit [here](https://github.com/qlik-oss/server-side-extension/commit/cf1b8d28a1431fd192587aac13ebe73dad388c4c)  
+[#13](https://github.com/qlik-oss/server-side-extension/pull/13) : Java example added  
+[#14](https://github.com/qlik-oss/server-side-extension/pull/14) : Go example added  
+[#18](https://github.com/qlik-oss/server-side-extension/pull/18) : Documentation and example for QlikView added  
+
+
+## v1.0.0
+Release date: 2017-06-26
+
+Initial release.

README.md

@@ -1,18 +1,20 @@
 # Server Side Extension
 
-This repository provides a server-side extension (SSE) protocol based on gRPC that allows you to extend the Qlik built-in expression library with functionality from external calculation engines. You can use external calculations engines from both load script expressions and chart expressions. These SSE plugins are referred to as analytic connections by Qlik.  
-SSE is part of the Advanced Analytics Integration (AAI) concept at Qlik.  
+This repository provides a server-side extension (SSE) protocol based on gRPC that allows you to extend the Qlik built-in expression library with functionality from external calculation engines. You can use external calculation engines in both load scripts and charts. In Qlik Sense and QlikView, you connect to these server-side extensions (SSEs) by defining analytic connections.  
 This repository includes documentation that describes the SSE protocol, how to use it and how to build your own plugins. Currently we have examples written in a few different languages only but examples in more languages will come later.  
 
 ## Status
 [![CircleCI](https://circleci.com/gh/qlik-oss/server-side-extension.svg?style=shield)](https://circleci.com/gh/qlik-oss/server-side-extension)  
 
-**Current Plugin Version and State:** v1.0.0  
-**Matching Qlik Sense Version:** Qlik Sense June 2017 release (or later). Both desktop and enterprise.  
-**Matching QlikView Version:** QlikView November 2017 (or later). Both desktop and server.  
-**Examples Disclaimer:** The examples provided are just examples, therefore **use them at your own risk**.  
+__Latest SSE Version:__ [v1.1.0](https://github.com/qlik-oss/server-side-extension/releases/latest)  
+__Examples Disclaimer:__ The examples provided are just examples, therefore __use them at your own risk__.   
 
-[Previous Versions](docs/versions.md)
+| __Latest Product Version__ | __SSE Supported__ |
+| ----- | ----- |
+| Qlik Sense February 2018 | v1.1.0 |
+| QlikView November 2017| v1.0.0 |
+
+[Previous Versions](CHANGELOG.md)
 
 ## Documentation
 

docs/SSE_Protocol.md

@@ -8,11 +8,13 @@
  * [CommonRequestHeader](#qlik.sse.CommonRequestHeader)
  * [Dual](#qlik.sse.Dual)
  * [Empty](#qlik.sse.Empty)
+ * [FieldDescription](#qlik.sse.FieldDescription)
  * [FunctionDefinition](#qlik.sse.FunctionDefinition)
  * [FunctionRequestHeader](#qlik.sse.FunctionRequestHeader)
  * [Parameter](#qlik.sse.Parameter)
  * [Row](#qlik.sse.Row)
  * [ScriptRequestHeader](#qlik.sse.ScriptRequestHeader)
+ * [TableDescription](#qlik.sse.TableDescription)
  * [DataType](#qlik.sse.DataType)
  * [FunctionType](#qlik.sse.FunctionType)
  * [Connector](#qlik.sse.Connector)
@@ -51,7 +53,7 @@ A full description of the plugin, sent to the Qlik engine, listing all functions
 <a name="qlik.sse.CommonRequestHeader"/>
 
 ### CommonRequestHeader
-A header sent at the start of both an EvaluateScript request and an ExecuteFunction request under the key &quot;qlik-commonrequestheader-bin&quot;.
+A header sent at the start of both an EvaluateScript request and an ExecuteFunction request under the key "qlik-commonrequestheader-bin".
 
 | Field | Type | Label | Description |
 | ----- | ---- | ----- | ----------- |
@@ -80,6 +82,18 @@ An empty message used when nothing is to be passed in a call.
 | ----- | ---- | ----- | ----------- |
 
 
+<a name="qlik.sse.FieldDescription"/>
+
+### FieldDescription
+Field definition for function and script calls.
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| dataType | [DataType](#qlik.sse.DataType) | optional | The data type of the field. |
+| name | [string](#string) | optional | The name of the field. |
+| tags | [string](#string) | repeated | The tags of the field. |
+
+
 <a name="qlik.sse.FunctionDefinition"/>
 
 ### FunctionDefinition
@@ -97,7 +111,7 @@ The definition of a function, which informs the Qlik engine how to use it.
 <a name="qlik.sse.FunctionRequestHeader"/>
 
 ### FunctionRequestHeader
-A header sent at the start of an ExecuteFunction request under the key &quot;qlik-functionrequestheader-bin&quot;.
+A header sent at the start of an ExecuteFunction request under the key "qlik-functionrequestheader-bin".
 
 | Field | Type | Label | Description |
 | ----- | ---- | ----- | ----------- |
@@ -129,7 +143,7 @@ A row of duals.
 <a name="qlik.sse.ScriptRequestHeader"/>
 
 ### ScriptRequestHeader
-A header sent at the start of an EvaluateScript request under the key &quot;qlik-scriptrequestheader-bin&quot;.
+A header sent at the start of an EvaluateScript request under the key "qlik-scriptrequestheader-bin".
 
 | Field | Type | Label | Description |
 | ----- | ---- | ----- | ----------- |
@@ -139,6 +153,18 @@ A header sent at the start of an EvaluateScript request under the key &quot;qlik
 | params | [Parameter](#qlik.sse.Parameter) | repeated | The parameters names and types passed to the script. |
 
 
+<a name="qlik.sse.TableDescription"/>
+
+### TableDescription
+A header sent before returning data to Qlik, under the key "qlik-tabledescription-bin".
+
+| Field | Type | Label | Description |
+| ----- | ---- | ----- | ----------- |
+| fields | [FieldDescription](#qlik.sse.FieldDescription) | repeated | The fields of the table. |
+| name | [string](#string) | optional | The name of the table. |
+| numberOfRows | [int64](#int64) | optional | Number of rows in table. |
+
+
 
 <a name="qlik.sse.DataType"/>
 

docs/configuration.md

@@ -1,23 +1,19 @@
 # Configuring SSE plugins in Qlik
 
 ## Qlik Sense
-You must configure SSE plugins, referred to by Qlik as analytic connections, in the Qlik Management Console (QMC) or using the Qlik Repository Service API. After you add a new analytic connection, you must restart the Qlik engine. 
+You must configure SSE plugins, referred to by Qlik as analytic connections, in the Qlik Management Console (QMC) or using the Qlik Repository Service API. After you add a new analytic connection, you must restart the Qlik engine.
 
-For an overview of analytic connections, see __Analytic connections__ in the Qlik Sense Developers help:
-[https://help.qlik.com/en-US/sense-developer/June2017/Content/AnalyticConnections.htm](https://help.qlik.com/en-US/sense-developer/June2017/Content/AnalyticConnections.htm)
+For an overview of analytic connections, see [Analytic connections](http://help.qlik.com/en-US/sense-developer/November2017/Content/AnalyticConnections.htm) in the Qlik Sense Developers help.
 
-For instructions for adding a new SSE plugin in QMC, see __Creating an analytic connection__ in the Qlik Sense help: 
-[https://help.qlik.com/en-US/sense/June2017/Subsystems/ManagementConsole/Content/create-analytic-connection.htm](https://help.qlik.com/en-US/sense/June2017/Subsystems/ManagementConsole/Content/create-analytic-connection.htm)
+For instructions of adding a new SSE plugin in QMC, see [Creating an analytic connection](https://help.qlik.com/en-US/sense/November2017/Subsystems/ManagementConsole/Content/create-analytic-connection.htm) in the Qlik Sense help.
+
+For information about SSE plugins for Qlik Sense Desktop, see [Configuring analytic connections in Qlik Sense Desktop](http://help.qlik.com/en-US/sense/November2017/Subsystems/Hub/Content/Introduction/configure-analytic-connection-desktop.htm) in the Qlik Sense help.
 
-For information about SSE plugins for Qlik Sense Desktop, see __Configuring analytic connections in Qlik Sense Desktop__ in the Qlik Sense help:
-[http://help.qlik.com/en-US/sense/June2017/Subsystems/Hub/Content/Introduction/configure-analytic-connection-desktop.htm](http://help.qlik.com/en-US/sense/June2017/Subsystems/Hub/Content/Introduction/configure-analytic-connection-desktop.htm)
 
 ## QlikView
 
 You must configure SSE plugins, referred to by Qlik as analytic connections, by editing the settings.ini file (either for QlikView Server or QlikView Desktop). After you add a new analytic connection, you must restart QlikView.
 
-For an overview of analytic connections, including how to add a new SSE plugin, see __Analytic connections__ in the QlikView help:
-[https://help.qlik.com/en-US/qlikview/November2017/Subsystems/Client/Content/Analytic_connections.htm](https://help.qlik.com/en-US/qlikview/November2017/Subsystems/Client/Content/Analytic_connections.htm)
+For an overview of analytic connections, including how to add a new SSE plugin, see [Analytic connections](https://help.qlik.com/en-US/qlikview/November2017/Subsystems/Client/Content/Analytic_connections.htm) in the QlikView help.
 
-For step-by-step instructions on how to get started with SSE in QlikView, see __Get started with analytic connections__ in the QlikView help:
-[https://help.qlik.com/en-US/qlikview/November2017/Subsystems/Client/Content/Getting-started-with-analytic-connections.htm](https://help.qlik.com/en-US/qlikview/November2017/Subsystems/Client/Content/Getting-started-with-analytic-connections.htm)
+For step-by-step instructions on how to get started with SSE in QlikView, see [Get started with analytic connections](https://help.qlik.com/en-US/qlikview/November2017/Subsystems/Client/Content/Getting-started-with-analytic-connections.htm) in the QlikView help.

docs/limitations.md

@@ -1,14 +1,14 @@
-# Limitations in this version of SSE plugins
+# Limitations
 
-#### Returning data to Qlik
-There is no support for returning several columns or higher-order data, to Qlik than was sent to the plugin. The cardinality of the response from the plugin must be the same as sent from Qlik.
-- If less rows, or lower cardinality, is sent back from the plugin, Qlik will add null values to match the number of rows sent from Qlik to the plugin originally. Note that the mapping may not work as intended if less data is sent back.
-- If higher-order data is sent back, either too many rows or more than one column, Qlik will neglect the additional data and only take the first _n_ rows in the first column, assuming _n_ is the number of rows sent to the plugin. Note that the mapping may not work as intended if more data is sent back.
+#### Expressions using SSE must persist the cardinality
+When you use SSE in a chart expression or in the Qlik load script (the `LOAD ... EXTENSION ...` statement excluded), you should preserve the cardinality and return a single column. In the case of aggregations, the response column should contain a single value (one row). In the case of tensor functions, the response column should contain the same number of rows as the request.
+- If too little data is returned from the plugin, Qlik will add null values to match the number of rows expected.
+- If higher-order data is returned, either too many rows or too many columns, Qlik will discard any additional data.
 
-#### Load script data cardinality (Qlik Limitation)
-There is no support for tensor calls from the load script. Only scalar and aggregation calls are supported.
+#### Chart expressions cannot consume a table returned from SSE plugin
+It is only possible to consume a returned table from an SSE call in the Qlik load script when using the `LOAD ... EXTENSION` statement. In all other cases, including chart expressions, only the first column returned from the plugin will be used by Qlik.
 
 #### Changes to plugins require engine restart (Qlik Limitation)
-If you add, remove, or change a plugin, you must restart the Qlik engine. For Qlik Sense this means either the engine service (for Qlik Sense Enterprise) or Qlik Sense Desktop. For QlikView, you must restart the QlikView Server service or QlikView Desktop.
+If you add, remove or change the capabilities of a plugin, you must restart the Qlik engine. For Qlik Sense this means either the engine service (for Qlik Sense Enterprise) or Qlik Sense Desktop. For QlikView, you must restart the QlikView Server service or QlikView Desktop.
 
-It is only during engine startup that the `GetCapability` plugin method is called.
+It is only during engine startup that Qlik tries to contact the SSE plugin by calling the `GetCapability` plugin method.

docs/planned_improvements.md

@@ -1,3 +1 @@
 # Planned improvements
-
--Currently we only return one column of values (or one value) back to Qlik since Qlik is not capable of treating more columns in a single response. We are working on implementing support for returning a complete table (or matrix) in a response to Qlik. In Qlik this will be used from the load script to start with.