# Endpoints

## How to get data from our API

You have the possibility to get results from predefined elements (datasets or scripts) using our API which returns JSON data. In addition, you can also set custom filters on the requests.\
Right now our API has a limit of returning 1 million rows of data at a time when querying data.

### Prerequisites

1. **SF user account to generate an API access token**
2. **SF platform backend URL of your environment**
3. **Id of the predefined element (dataset/script)**

First, you need an API access token, which can be generated by each user. This token is equivalent to your credentials so you will have the same permissions as the user who created this API access token. For information on how to generate such an API access token see [Get your access token](https://manual.senseforce.io/manual/sf-platform/get-your-access-token#getting-your-access-token).

The second information you need is the backend URL of your environment. You can see this URL when your login screen is displayed. Here you require the first part of the URL which usually looks like ***exampleapi.senseforce.io***

![SF platform backend URL](https://1533098984-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-Lj6tqYfpNT_59dZ_Z3Z%2F-MLCPuQK2gSwTYNPb_5O%2F-MLCbZoh-cJ1ugKsAlgY%2Fimage.png?alt=media\&token=06033ad0-ede4-4ef8-b369-e14c9eb3d3c2)

The third and last thing you need for your API request is the ID of the element you want to request. For example, for a dataset, you can see this information when you open the desired dataset in the SF platform. The ID is the last part which is shown in the address bar (see example below).

!\[Dataset ID

]\(<https://1533098984-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-Lj6tqYfpNT_59dZ_Z3Z%2F-MLCPuQK2gSwTYNPb_5O%2F-MLCdSTpnIJVLD__qA5s%2Fimage.png?alt=media\\&token=4178c660-64c6-45e2-89ca-f312462b9a99>)

{% hint style="info" %}
Double-check the headers before making a request. When a request library (like 'requests' in Python) or Postman is used, some headers are auto-generated so we don't have to manually set them, but this is not always true especially for 'Host' or 'Content-Type'. \
![](https://1533098984-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lj6tqYfpNT_59dZ_Z3Z%2Fuploads%2FWyT9akPgCwR2dyWakIeD%2Fimage.png?alt=media\&token=c1e3eb26-2e36-4214-b29c-3b900cf2aa3d)
{% endhint %}

### How to construct the API request

Below you can find information about endpoints structure and required parameters.

## Execute a dataset through Senseforce API

<mark style="color:green;">`POST`</mark> `https://<your senseforce backend platform url>/api/dataset/execute/<id>`

The URL you have to send your POST request to has to have the structure as shown above. This API endpoint can contain extra two optional parameters: **limit** and **offset**. After inserting your parameters the request looks like *\*\*<https://exampleapi.senseforce.io/api/dataset/execute/d9d6gDg7fGmd3mI**?**offset**=0&**limit**=100>*

\
For how to get the backend URL and/or dataset id see the section above.\
Within the body of your request, you can add filter clauses which are additional filters applied to the original dataset. For more information about additional filters see the sections below. The body itself is optional. So it can be an empty array or an array of filter objects, but for each filter object the parameters mentioned in "Body parameters" are mandatory.

#### Query Parameters

| Name   | Type   | Description              |
| ------ | ------ | ------------------------ |
| id     | string | The dataset id has to    |
| offset | string | *Number of rows to skip* |
| limit  | string | Number of rows to return |

#### Headers

| Name          | Type   | Description                              |
| ------------- | ------ | ---------------------------------------- |
| Content-Type  | string | Set to "application/json"                |
| Authorization | string | Set to "Bearer \<your API access token>" |

#### Request Body

| Name       | Type    | Description                                                                       |
| ---------- | ------- | --------------------------------------------------------------------------------- |
| clause     | object  | Filter clause object. Each additional filter is defined by such a *clause* object |
| type       | string  | Can be "timestamp" or "string" (for all other column datatypes)                   |
| operator   | integer | Id of the filter operator used to define the filter condition                     |
| parameters | string  | Filter arguments                                                                  |
| columnName | string  | Name of the dataset column the filter should be applied to                        |

{% tabs %}
{% tab title="200 Cake successfully retrieved." %}

```
{    "name": "Cake's name",    "recipe": "Cake's recipe name",    "cake": "Binary cake"}
```

{% endtab %}

{% tab title="404 Could not find a cake matching this query." %}

```
{    "message": "Ain't no cake like that."}
```

{% endtab %}
{% endtabs %}

#### Requesting original dataset

When you want to get all data from the dataset you can send requests with an empty array as a body. Below you can find a sample Python script to do so.

```python
import requests
import json
from pandas import DataFrame

url = "https://<your senseforce backend platform url>/api/dataset/execute/<id>"
headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer <your API access token>'}
filters = []

response = requests.post(url, headers=headers, json=filters)
data = response.text
parsed_data = json.loads(data)
df = DataFrame(parsed_data)
```

**Requesting dataset with original labels**

By default, all response element property names (dataset column labels) are converted to first letter lower case.

To avoid this behavior, and to keep the original label letter case, use an additional parameter in the request `useOriginalLabels` and set it to `true`. \
`(/api/dataset/execute/<id>?useOriginalLabels=true)`

To use converted column labels, you can explicitly set `useOriginalLabels` to `false`, or simply omit the parameter.

![Example of useOriginalLabels parameter usage.](https://1533098984-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lj6tqYfpNT_59dZ_Z3Z%2Fuploads%2F9hr9v8CiaokhQ2wVSIXB%2Fimage.png?alt=media\&token=da9f3de3-8fe9-4977-9775-eb827d5bd510)

#### Requesting dataset with additional filters

All filters defined in the dataset will be applied anyway, but you can also apply additional filters to it. To define a filter you have to create a structure like the one shown in the example below.&#x20;

In the example below a filter for the column named "device" is applied so that it only contains values equal to "vienna-prater-ferris-wheel-motor1".

```
[{
		"clause": {
			"type": "string",
			"operator": 7,
			"parameters": [{
					"value": "vienna-prater-ferris-wheel-motor1"
				}
			]
		},
		"columnName": "device"
	}
]
```

Within these clause objects (filter definitions) all operators which are also available on the SF Platform can be used. But some restrictions have to be considered. Not each operator can be applied to each column, because of their datatype. And not all of these filters have the same number of parameters. Most of them have only one, but some of them have none (e.g. IsEmpty, IsNotEmpty) and some have two (e.g. Between).

A summary of all available filter "operator" and their restrictions are given in the table below:

#### Filter Operators Table

<table data-header-hidden><thead><tr><th width="238">Operator</th><th width="120">Operator id</th><th width="104"># param</th><th>Required parameter datatype</th><th>Required column datatype</th></tr></thead><tbody><tr><td>Operator</td><td>Operator id</td><td># param</td><td>Required parameter datatype</td><td>Required column datatype</td></tr><tr><td>LessThan</td><td>1</td><td>1</td><td>same as column</td><td>integer | long | double </td></tr><tr><td>GreaterThan</td><td>2</td><td>1</td><td>same as column</td><td>integer | long | double </td></tr><tr><td>LessThanOrEqualTo</td><td>3</td><td>1</td><td>same as column</td><td>integer | long | double </td></tr><tr><td>GreaterThanOrEqualTo</td><td>4</td><td>1</td><td>same as column</td><td>integer | long | double </td></tr><tr><td>Equal</td><td>5</td><td>1</td><td>same as column</td><td>integer | long | double | string</td></tr><tr><td>NotEqual</td><td>6</td><td>1</td><td>same as column</td><td>integer | long | double | string</td></tr><tr><td>Like</td><td>7</td><td>1</td><td>string</td><td>string</td></tr><tr><td>RegExpMatch</td><td>8</td><td>1</td><td>string</td><td>string</td></tr><tr><td>NotRegExpMatch</td><td>9</td><td>1</td><td>string</td><td>string</td></tr><tr><td>Between</td><td>10</td><td>2</td><td>same as column</td><td>integer | long | double | timestamp</td></tr><tr><td>In</td><td>11</td><td>1...n</td><td>same as column</td><td>each datatype allowed</td></tr><tr><td>IsEmpty</td><td>12</td><td>0</td><td>-</td><td>each datatype allowed</td></tr><tr><td>IsNotEmpty</td><td>13</td><td>0</td><td>-</td><td>each datatype allowed</td></tr><tr><td>CustomToday</td><td>14</td><td>0</td><td>-</td><td>timestamp</td></tr><tr><td>CustomThisWeek</td><td>15</td><td>0</td><td>-</td><td>timestamp</td></tr><tr><td>CustomThisMonth</td><td>16</td><td>0</td><td>-</td><td>timestamp</td></tr><tr><td>CustomLastThreeMonths</td><td>17</td><td>0</td><td>-</td><td>timestamp</td></tr><tr><td>CustomLastXMinutes</td><td>18</td><td>1</td><td>integer</td><td>timestamp</td></tr><tr><td>CustomDay</td><td>19</td><td>1</td><td>timestamp</td><td>timestamp</td></tr><tr><td>NotLike</td><td>20</td><td>1</td><td>string</td><td>string</td></tr><tr><td>CustomYesterday</td><td>21</td><td>0</td><td>-</td><td>timestamp</td></tr><tr><td>CustomLastXDays</td><td>22</td><td>1</td><td>integer</td><td>timestamp</td></tr><tr><td>CustomLastXWeeks</td><td>23</td><td>1</td><td>integer</td><td>timestamp</td></tr><tr><td>CustomLastXMonths</td><td>24</td><td>1</td><td>integer</td><td>timestamp</td></tr><tr><td>CustomLastWeek</td><td>25</td><td>0</td><td>-</td><td>timestamp</td></tr><tr><td>CustomLastMonth</td><td>26</td><td>0</td><td>-</td><td>timestamp</td></tr><tr><td>CustomRelativeBetween</td><td>27</td><td>2</td><td>integer</td><td>timestamp</td></tr><tr><td>NotIn</td><td>28</td><td>1...n</td><td>same as column</td><td>each datatype allowed</td></tr></tbody></table>

{% hint style="info" %}
NOTE: "timestamps" are integer values representing the Unix timestamp in milliseconds (!!!).
{% endhint %}

Below you can find an extended version of the example Python script where also additional filters are applied.

<pre class="language-python"><code class="lang-python">import requests
import json
from pandas import DataFrame

class operator:
    LessThan = 1
    GreaterThan = 2
    LessThanOrEqualTo = 3
    GreaterThanOrEqualTo = 4
    Equal = 5
    NotEqual = 6
    Like = 7
    RegExpMatch = 8
    NotRegExprMatch = 9
    Between = 10
    In = 11
    IsEmpty = 12
    IsNotEmpty = 13
    CustomToday = 14
    CustomThisWeek = 15
    CustomThisMonth = 16
    CustomLastThreeMonths = 17
    CustomLastXMinutes = 18
    CustomDay = 19
    NotLike = 20
    CustomYesterday = 21
    CustomLastXDays = 22
    CustomLastXWeeks = 23
    CustomLastXMonths = 24
    CustomLastWeek = 25
<strong>    CustomLastMonth = 26
</strong><strong>    CustomRelativeBetween = 27
</strong><strong>    NotIn = 28
</strong>
class datatype:
    timestamp = "timestamp"
    string = "string"

url = "https://&#x3C;your senseforce backend platform url>/api/dataset/execute/&#x3C;id>"
headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer &#x3C;your API access token>'}
filters = [{
        		"clause": {
        			"type": datatype.string,
        			"operator": operator.GreaterThan,
        			"parameters": [{		
                      "value": 2000
        				}
        			]
        		},
        		"columnName": "compressed size"
        	},
         {
        		"clause": {
        			"type": datatype.string,
        			"operator": operator.NotEqual,
        			"parameters": [{
                        "value": "Thing XY"
        				}
        			]
        		},
        		"columnName": "(sf-messages) Thing"
        	},
         {
            "clause": {
        			"type": datatype.timestamp,
        			"operator": operator.Between,
        			"parameters": [{		
                        "value": 1604043829000
        				},
                   {		
                        "value": 1604053829000
        				}
        			]
        		},
        		"columnName": "(sf-messages) Timestamp"
         }]

response = requests.post(url, headers=headers, json=filters)
data = response.text
parsed_data = json.loads(data)
df = DataFrame(parsed_data)
</code></pre>

## Execute a script through Senseforce API

<mark style="color:green;">`POST`</mark> `https://<your senseforce backend platform url>/api/script/execute/{scriptId}`

The URL request has to have the structure as shown above. This API endpoint can contain extra two optional parameters: **limit** and **offset**. After inserting the parameters, the request may look like this:\
\&#xNAN;*<https://exampleapi.senseforce.io/api/script/execute/>**807a0c12-35e9-4b7f-b695-837c2cf5fb41**?**offset**=0&**limit**=100*\
\
Within the body of the request, you can add script filters and dataset filters. Dataset filters are additional filters applied to the original dataset. For more information about the script filters and dataset, filters see the sections below. \
The body can be an empty object or can contain filter objects.<br>

#### Path Parameters

| Name     | Type   | Description |
| -------- | ------ | ----------- |
| scriptId | string |             |

#### Query Parameters

| Name   | Type   | Description              |
| ------ | ------ | ------------------------ |
| offset | string | Number of rows to skip   |
| limit  | string | Number of rows to return |

#### Headers

| Name          | Type   | Description                              |
| ------------- | ------ | ---------------------------------------- |
| Authorization | string | Set to "Bearer \<your API access token>" |
| Content-Type  | string | Set to "application/json"                |

#### Request Body

| Name           | Type  | Description             |
| -------------- | ----- | ----------------------- |
| ScriptFilters  | array | List of script filters  |
| DatasetFilters | array | List of dataset filters |

{% tabs %}
{% tab title="200 " %}

```
```

{% endtab %}
{% endtabs %}

The Body Parameters ("ScriptFilters" and "DatasetFilters") contain the same filter object structure in their list. See the filter object structure in the section below.\
The Body Parameter "DatasetFilters" can contain filter objects associated with a column of a dataset. These filters can target columns from multiple datasets as well (a script can work with one or multiple datasets).

![Filter object structure ](https://1533098984-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-Lj6tqYfpNT_59dZ_Z3Z%2F-Ma2DabGWmRJoHrajKf2%2F-Ma2EPoQ65K5LtvavitX%2Fimage.png?alt=media\&token=6ada6afb-5ae3-45c8-8bcb-d44c43c26ea8)

#### Executing a script

Let's use a demo script and see how this script can be executed via Senseforce API.\
The following script has a "value" variable and it outputs this variable as result.&#x20;

![](https://1533098984-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-Lj6tqYfpNT_59dZ_Z3Z%2F-Ma2PFd5BML9tzdgsR_z%2F-Ma2QAjDK69tWn5HHVEJ%2Fimage.png?alt=media\&token=18c85b10-f9df-44d5-9cf6-63b0fa207196)

Let's execute this script via Senseforce API. When you want to execute a script without any filters you can send requests with an empty object as a body. Below you can find a C# request sample:

```python
var client = new RestClient("https://<your senseforce backend platform url>/api/script/execute/7157b7c7-dd56-46ee-93e2-2955cd91cfdc");

var request = new RestRequest(Method.POST);
request.AddHeader("Authorization", "Bearer <your API access token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}",  ParameterType.RequestBody);
IRestResponse response = client.Execute(request);

Console.WriteLine(response.Content);
```

#### Executing a script with script filters

In the example below a script filter for the column named "value" is applied so that it only contains values equal to "hello from script". The Request Body will be like:

```python
request.AddParameter("application/json", 
"{
    "scriptFilters": [
      {
		    "clause": {
			    "type": "string",
			    "operator": 7,
			    "parameters": [{ "value": "hello from script" }]
		    },
		    "columnName": "value"
	  	}
		]
}",  ParameterType.RequestBody);
```

A summary of all available filter "operator" and their restrictions are given in the above "Filter Operators" table

#### Executing a script with dataset filters

Let's use a demo script that uses two datasets so you can see how the dataset filters can be applied.

![](https://1533098984-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-Lj6tqYfpNT_59dZ_Z3Z%2F-Ma2mQ4b502okjZLDETp%2F-Ma2mUo5tp8pGeW-hKB6%2Fimage.png?alt=media\&token=b5e615cf-8b37-4994-83ab-919793f5df95)

In the example below,  two dataset filters are set, which target different datasets. The Request Body will be like:

```python
request.AddParameter("application/json", 
"{
    "datasetFilters": [
        {
            "clause": {
                "type": "string",
                "operator": 7,
                "parameters": [{ "value": "virtual_event" }]
            },
            "columnName": "(another_test) Thing"
	      },
        {
            "clause": {
                "type": "string",
                "operator": 7,
                "parameters": [{ "value": "virtual_event" }]
            },
            "columnName": "(test) Thing"
	      }
    ]
}",  ParameterType.RequestBody);

```

#### Executing a script with script and dataset filters

Let's use a demo script which uses two datasets so you can see how the dataset filters and script filters can be applied.

![](https://1533098984-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-Lj6tqYfpNT_59dZ_Z3Z%2F-Ma2oRLxCTEbuCQfaAx-%2F-Ma2yy-ffqctymvUuE4o%2Fimage.png?alt=media\&token=ab4304f1-5511-488c-8da4-06b4747303be)

In the example below,  one script filter and two dataset filters are set.  The Request Body will be like:

```python
request.AddParameter("application/json", 
"{
    "scriptFilters": [
        {
            "clause": {
                "type": "number",
                "operator": 5,
                "parameters": [{ "value": "285" }]
            },
            "columnName": "size"
	      }
    ],
    "datasetFilters": [
        {
            "clause": {
                "type": "string",
                "operator": 7,
                "parameters": [{ "value": "virtual_event" }]
            },
            "columnName": "(another_test) Thing"
	      },
        {
            "clause": {
                "type": "string",
                "operator": 7,
                "parameters": [{ "value": "virtual_event" }]
            },
            "columnName": "(test) Thing"
	      }
    ]
}",  ParameterType.RequestBody);

```

Dataset filters are first considered and then at the end, the script filter is applied.