Create reports on REST API / HTTP request on Helical Insight

NOTE:     From version 5.2.2 of Open Source BI Helical Insight, we have provided DIRECT inbuilt support to REST API driver. This method is much more simpler to use and recommended. Please refer to this link to learn more :

https://www.helicalinsight.com/connect-and-use-an-api-as-a-data-source-in-helical-insight-5-0/

If you are using older version or want to use Apache drill, then continue to read the below.

In this blog we are going to cover how to connect to an API / HTTP requests from Open Source BI product Helical Insight and use that to create reports.

Introduction : Apache drill has introduced HTTP storage plugin from drill version 1.18 and onwards. The HTTP storage plugin lets you query APIs over HTTP/REST. The plugin expects JSON or CSV responses. In this document we will cover how to create HTTP plugin and how to connect it with Helical Insight application.

You may refer to our other blogs to learn about how to download and install Apache Drill.

To configure the plugin, create a new storage plugin in Apache drill with any name (we have given name as http) and add configurations within this storage plugin. Below we have provided sample configuration of one of the API, which we are using as an example for this document :

{
  "type": "http",
  "cacheResults": false,
  "connections": {
    "population": {
      "url": "https://datausa.io/api/data?drilldowns=State&measures=Population",
      "method": "GET",
      "headers": null,
      "authType": "none",
      "userName": null,
      "password": null,
      "postBody": null,
      "params": [
        "year"
      ],
      "dataPath": "data",
      "requireTail": false,
      "inputType": "json"
    }
  },
  "timeout": 0,
  "proxyHost": null,
  "proxyPort": 0,
  "proxyType": "direct",
  "proxyUsername": null,
  "proxyPassword": null,
  "enabled": true
}

As you can see above, within connections we can provide our API url, method (GET OR POST), parameters, headers, authentication, datapath etc. Using above configurations we have created http storage plugin within Apache drill 1.18.

NOTE : It is mandatory to define the key name of output response from API into dataPath.
Learn about creating the plugins on apache drill here.

For example we have an API:

API : https://datausa.io/api/data?drilldowns=State&measures=Population

Output of this API : {"data":[{"ID State":"04000US01","State":"Alabama","ID Year":2019,"Year":"2019","Population":4903185,"Slug State":"alabama"}]}

So within dataPath we need to define the JSON object key within which we see the response. In our case it is data and hence we defined dataPath as data. Once we have created http storage we can then connect apache drill with Helical Insight application from Datasources page.

Now go to metadata creation page and add any random table to metadata, this needs to be done in order to create a metadata view.

Create metadata Query view, in our case we have created this using SQL : select * from http.population

Save the metadata and go to report creation page for creating a report. Select the metadata which you have just now created to use.

Save the report. You can create N number of reports and dashboards by following same method.

Reference : https://drill.apache.org/docs/http-storage-plugin/

You May Also Like

Helical Insight’s self-service capabilities is one to reckon with. It allows you to simply drag and drop columns, add filters, apply aggregate functions if required, and create reports and dashboards on the fly. For advanced users, the self-service component has ability to add javascript, HTML, HTML5, CSS, CSS3 and AJAX. These customizations allow you to create dynamic reports and dashboards. You can also add new charts inside the self-service component, add new kind of aggregate functions and customize it using our APIs.
Helical Insight’s self-service capabilities is one to reckon with. It allows you to simply drag and drop columns, add filters, apply aggregate functions if required, and create reports and dashboards on the fly. For advanced users, the self-service component has ability to add javascript, HTML, HTML5, CSS, CSS3 and AJAX. These customizations allow you to create dynamic reports and dashboards. You can also add new charts inside the self-service component, add new kind of aggregate functions and customize it using our APIs.
Helical Insight, via simple browser based interface of Canned Reporting module, also allows to create pixel perfect printer friendly document kind of reports also like Invoice, P&L Statement, Balance sheet etc.
Helical Insight, via simple browser based interface of Canned Reporting module, also allows to create pixel perfect printer friendly document kind of reports also like Invoice, P&L Statement, Balance sheet etc.
If you have a product, built on any platform like Dot Net or Java or PHP or Ruby, you can easily embed Helical Insight within it using iFrames or webservices, for quick value add through instant visualization of data.
If you have a product, built on any platform like Dot Net or Java or PHP or Ruby, you can easily embed Helical Insight within it using iFrames or webservices, for quick value add through instant visualization of data.
Being a 100% browser-based BI tool, you can connect with your database and analyse across any location and device. There is no need to download or install heavy memory-consuming developer tools – All you need is a Browser application! We are battle-tested on most of the commonly used browsers.
Being a 100% browser-based BI tool, you can connect with your database and analyse across any location and device. There is no need to download or install heavy memory-consuming developer tools – All you need is a Browser application! We are battle-tested on most of the commonly used browsers.
We have organization level security where the Superadmin can create, delete and modify roles. Dashboards and reports can be added to that organization. This ensures multitenancy.
We have organization level security where the Superadmin can create, delete and modify roles. Dashboards and reports can be added to that organization. This ensures multitenancy.
We have organization level security where the Superadmin can create, delete and modify roles. Dashboards and reports can be added to that organization. This ensures multitenancy.
We have organization level security where the Superadmin can create, delete and modify roles. Dashboards and reports can be added to that organization. This ensures multitenancy.
A first-of-its-kind Open-Source BI framework, Helical Insight is completely API-driven. This allows you to add functionalities, including but not limited to adding a new exporting type, new datasource type, core functionality expansion, new charting in adhoc etc., at any place whenever you wish, using your own in-house developers.
A first-of-its-kind Open-Source BI framework, Helical Insight is completely API-driven. This allows you to add functionalities, including but not limited to adding a new exporting type, new datasource type, core functionality expansion, new charting in adhoc etc., at any place whenever you wish, using your own in-house developers.
It handles huge volumes of data effectively. Caching, Pagination, Load-Balancing and In-Memory not only provides you with amazing experience, but also and does not burden the database server more than required. Further effective use of computing power gives best performance and complex calculations even on the big data even with smaller machines for your personal use. Filtering, Sorting, Cube Analysis, Inter Panel Communication on the dashboards all at lightning speed. Thereby, making best open-source Business Intelligence solution in the market.
It handles huge volumes of data effectively. Caching, Pagination, Load-Balancing and In-Memory not only provides you with amazing experience, but also and does not burden the database server more than required. Further effective use of computing power gives best performance and complex calculations even on the big data even with smaller machines for your personal use. Filtering, Sorting, Cube Analysis, Inter Panel Communication on the dashboards all at lightning speed. Thereby, making best open-source Business Intelligence solution in the market.
With advance NLP algorithm, business users simply ask questions like, “show me sales of last quarter”, “average monthly sales of my products”. Let the application give the power to users without knowledge of query language or underlying data architecture
With advance NLP algorithm, business users simply ask questions like, “show me sales of last quarter”, “average monthly sales of my products”. Let the application give the power to users without knowledge of query language or underlying data architecture
Our application is compatible with almost all databases, be it RDBMS, or columnar database, or even flat files like spreadsheets or csv files. You can even connect to your own custom database via JDBC connection. Further, our database connection can be switched dynamically based on logged in users or its organization or other parameters. So, all your clients can use the same reports and dashboards without worrying about any data security breech.
Our application is compatible with almost all databases, be it RDBMS, or columnar database, or even flat files like spreadsheets or csv files. You can even connect to your own custom database via JDBC connection. Further, our database connection can be switched dynamically based on logged in users or its organization or other parameters. So, all your clients can use the same reports and dashboards without worrying about any data security breech.
Our application can be installed on an in-house server where you have full control of your data and its security. Or on cloud where it is accessible to larger audience without overheads and maintenance of the servers. One solution that works for all.
Our application can be installed on an in-house server where you have full control of your data and its security. Or on cloud where it is accessible to larger audience without overheads and maintenance of the servers. One solution that works for all.
Different companies have different business processes that the existing BI tools do not encompass. Helical Insight permits you to design your own workflows and specify what functional module of BI gets triggered
Different companies have different business processes that the existing BI tools do not encompass. Helical Insight permits you to design your own workflows and specify what functional module of BI gets triggered