Extend EFWD

In this article you will learn about why and how to extend EFWD file. Mainly, the role of EFWD is to connect the Helical Insight Application with different datasources with the help of plugins available for the datasources. The various databases it supports like mysql, oracle, postgres, SQLite and so on. Now, for a business user it plays an important role as it supports multiple databases for report creation. As you are aware of that the technology doesn’t last for a longer period hence EFWD helps your business in getting connected with these latest technologies. This will save your time and cost.

Why to Extend EFWD

The EFWD file enables the dashboard to access the data required for generating reports. Originally, the EFWD file is configured to pool data from Mondrian and all RDBMS that support MySQL queries. However, if the user requires acquiring data from other data sources, HDI allows the user to develop his own plugin for his chosen data source and use it to get data to make his reports.

Architecture

archi

How to Extend EFWD

EFWD can be extended by adding a plugin and making changes in some settings files. In this section the contents of the plugin and the required configuration file modifications will be explained.

The example provided here is for creating a plugin for importing the data from a CSV file.

The Concept:
To maintain uniformity, the data from any data source that does not support MySQL queries must be imported into a database of your choice. (The database used for explanation is SQLite).

Prerequisite:

⦁ The CSV file should be a proper CSV with a well defined delimiter.

⦁ The first row of the CSV file should contain the column headers.

⦁ The Driver Should be placed in the “System -> Drivers” Folder.

⦁ The Package name SHOULD NOT have a similar name as the Application (com.helical.xxx.xxx).

Creating a PROPERTIES File :

  • The PROPERTIES file contains the database details where the data is to be dumped.
  • This PROPERTIES file is to be placed in the solution directory, in “System -> Admin”.

The contents of this file (here – csvDataSource.properties) should be in the following manner:

efwd1



Note : Since SQLite does not need a “username” and “password”, these still need to be mentioned and equated to dummy values.

Configuring setting.XML :
The setting.xml needs to be modified in order for the report to be able to use the user defined data source.
“setting.xml” is located in the file repository in folder “System -> Admin”.
In this file, a new “<DataSource/>” tag needs to be added inside “<DataSources></DataSources>”.





The new “<DataSource/>” tag, the following fields must be present:

  1. class : The class which holds the actual functioning of the plugin.
  2. classifier : Unique identifier for the datasource.
  3. name : The name given to identify the datasource.
  4. type : This is the type used in the EFWD file of the report for type recognition of the data map.

Change in the .EFWD File:
The connection specified in the .efwd file should be in the following format:

efwd3


  • The Connection “type” should be the same as what is specified in the “<DataSource/>” in “setting.xml”.
  • The tags inside the “<Connection></Connection>” tag should be noted carefully (here – “<dir></dir>” and “<file></file>”) since the data in these tags will be used in the data dumping code of the plugin.
  • The “<dir></dir>” tag holds the path of the directory where the source file is stored.
  • While, “<file></file>” tag holds the source file name with its extension.

Driver Creation:
To create a driver to handle CSV as data source, the driver must implement the “IDriver” class belonging to the Application. The Classes that are required to be imported, are mentioned in the sample code.

Sample Code

import java.io.File;
import java.sql.Connection;
import java.util.Map;
import net.sf.json.JSONObject;
import com.helical.efw.drivers.EfwdQueryProcessor;
import com.helical.efw.drivers.IDriver;
import com.helical.efw.drivers.JDBCDriver;
import com.helical.efw.singleton.ApplicationProperties;
import com.helical.efw.utility.PropertiesFileReader;
import com.helical.efwd.jdbc.IJdbcDao;
import com.helical.framework.utils.ApplicationContextAccessor;

public class CsvDataSourceDriver implements IDriver {

@Override
public JSONObject getJSONData(JSONObject requestParameterJson,
JSONObject connectionDetails, JSONObject dataMapTagContent,
ApplicationProperties applicationProperties) {

    String dir = connectionDetails.getString("dir");
    String file = connectionDetails.getString("file");

    // Create the complete path of csv file
    String csvFilePath = applicationProperties.getSolutionDirectory() + File.separator + dir + File.separator + file;
    // Create table name and database name having the same name as the //csv file
    String tableName = file.substring(0, file.length() - 4);
    String dbName = tableName + ".db";

    String dbFilePath = applicationProperties.getSolutionDirectory() + File.separator + dir + File.separator + dbName;

    PropertiesFileReader map = new PropertiesFileReader();
    Map<String, String> read = map.read("Admin", "csvDataSource.properties");

    String csvDriverUrl = read.get("csvDriverUrl");
    String driver = read.get("Driver");
    String username = read.get("username");
    String password = read.get("password");

    File fileToCheck = new File(dbFilePath);

    if(!fileToCheck.exists()) {
        CsvToDatabaseDumpHandler.dump(csvFilePath, tableName, dbFilePath, csvDriverUrl, driver, username, password);
    }

    String query = getQuery(dataMapTagContent, requestParameterJson);
    IJdbcDao bean = ApplicationContextAccessor.getBean(IJdbcDao.class);
    Connection connection = JDBCDriver.getConnection (csvDriverUrl+dbFilePath, username, password, driver);
    return JSONObject.fromObject(bean.query(connection, query));
}

@Override
public String getQuery(JSONObject dataMapTagContent, JSONObject requestParameterJson) {
    EfwdQueryProcessor queryProcessor = new EfwdQueryProcessor();
    return queryProcessor.getQuery(dataMapTagContent, requestParameterJson);
}
}

Sample Explanation:
public JSONObject getJSONData(JSONObject requestParameterJson,
JSONObject connectionDetails, JSONObject dataMapTagContent,
ApplicationProperties applicationProperties)


To get any data, we must first get 4 JSONObjects (method : getJSONObjects) – Parameters:

  1. requestParameterJSON – Contains the directory of the .efwd file and the map id to be used from the same file.
  2. connectionDetails – Contains the details of the connection tag in the .efwd file.
  3. dataMapTagContents – Contains the SQL query in the map tag of the .efwd file.
  4. applicationProperties – Contains the repository details.

String dir = connectionDetails.getString(“dir”);
String file = connectionDetails.getString(“file”);

From the connectionDetails JSON the source file (here – .csv file) and its containing directory location are taken.

  • // Create the complete path of csv file
    String csvFilePath = applicationProperties.getSolutionDirectory()
    + File.separator + dir + File.separator + file;
  • // Create table name and database name having the same name as the //csv file
    String tableName = file.substring(0, file.length() – 4);
    String dbName = tableName + “.db”;
  • // Create database at the same location as the csv file.
    String dbFilePath = applicationProperties.getSolutionDirectory()
    + File.separator + dir + File.separator + dbName;
  • PropertiesFileReader map = new PropertiesFileReader();
    Map<String, String> read = map.read(“Admin”,
    “csvDataSource.properties”);

String csvDriverUrl = read.get(“csvDriverUrl”);
String driver = read.get(“Driver”);
String username = read.get(“username”);
String password = read.get(“password”);

To be able to connect to the database, we need the details that are specified in the PROPERTIES file (as mentioned above). Using the method “PropertiesFileReader”, a HashMap is created and the values of the driverURL, Driver name, username and password are obtained.

 

  • File fileToCheck = new File(dbFilePath);if(!fileToCheck.exists()) {
    CsvToDatabaseDumpHandler.dump(csvFilePath, tableName, dbFilePath, csvDriverUrl, driver, username, password);
    }

First check if the database containing the CSV exists. If it doesn’t exist, create it, otherwise use the existing database.
“CsvToDatabaseDumpHandler” is the method which contains the logic for converting the CSV to a Database table.

 

  • String query = getQuery(dataMapTagContent, requestParameterJson);
    IJdbcDao bean = ApplicationContextAccessor.getBean(IJdbcDao.class);
    Connection connection = JDBCDriver.getConnection
    (csvDriverUrl+dbFilePath, username, password, driver);
    return JSONObject.fromObject(bean.query(connection, query));

The “query” is obtained from the “getQuery” method in the form of a String. This IJdbcDao can help you perform most trivial JDBC tasks like inserting parameters into PreparedStatement’s, iterating ResultSet’s etc.
This returns a JSONObject containing the result of the Query provided.

  • public String getQuery(JSONObject dataMapTagContent,
    JSONObject requestParameterJson) {
    EfwdQueryProcessor queryProcessor = new EfwdQueryProcessor();
    return queryProcessor.getQuery(dataMapTagContent, requestParameterJson);
    }

The “getQuery” method gets the query from the .efwd file by using the directory path and map id provided in the requestParameter JSONObject and getMapTagContent JSONObject respectively.


For More Info,
Contact us at demo@helicalinsight.com

You May Also Like

Leave a Reply

Helical Insight’s self-service capabilities is one to reckon with. It allows you to simply drag and drop columns, add filters, apply aggregate functions if required, and create reports and dashboards on the fly. For advanced users, the self-service component has ability to add javascript, HTML, HTML5, CSS, CSS3 and AJAX. These customizations allow you to create dynamic reports and dashboards. You can also add new charts inside the self-service component, add new kind of aggregate functions and customize it using our APIs.
Helical Insight’s self-service capabilities is one to reckon with. It allows you to simply drag and drop columns, add filters, apply aggregate functions if required, and create reports and dashboards on the fly. For advanced users, the self-service component has ability to add javascript, HTML, HTML5, CSS, CSS3 and AJAX. These customizations allow you to create dynamic reports and dashboards. You can also add new charts inside the self-service component, add new kind of aggregate functions and customize it using our APIs.
Helical Insight, via simple browser based interface of Canned Reporting module, also allows to create pixel perfect printer friendly document kind of reports also like Invoice, P&L Statement, Balance sheet etc.
Helical Insight, via simple browser based interface of Canned Reporting module, also allows to create pixel perfect printer friendly document kind of reports also like Invoice, P&L Statement, Balance sheet etc.
If you have a product, built on any platform like Dot Net or Java or PHP or Ruby, you can easily embed Helical Insight within it using iFrames or webservices, for quick value add through instant visualization of data.
If you have a product, built on any platform like Dot Net or Java or PHP or Ruby, you can easily embed Helical Insight within it using iFrames or webservices, for quick value add through instant visualization of data.
Being a 100% browser-based BI tool, you can connect with your database and analyse across any location and device. There is no need to download or install heavy memory-consuming developer tools – All you need is a Browser application! We are battle-tested on most of the commonly used browsers.
Being a 100% browser-based BI tool, you can connect with your database and analyse across any location and device. There is no need to download or install heavy memory-consuming developer tools – All you need is a Browser application! We are battle-tested on most of the commonly used browsers.
We have organization level security where the Superadmin can create, delete and modify roles. Dashboards and reports can be added to that organization. This ensures multitenancy.
We have organization level security where the Superadmin can create, delete and modify roles. Dashboards and reports can be added to that organization. This ensures multitenancy.
We have organization level security where the Superadmin can create, delete and modify roles. Dashboards and reports can be added to that organization. This ensures multitenancy.
We have organization level security where the Superadmin can create, delete and modify roles. Dashboards and reports can be added to that organization. This ensures multitenancy.
A first-of-its-kind Open-Source BI framework, Helical Insight is completely API-driven. This allows you to add functionalities, including but not limited to adding a new exporting type, new datasource type, core functionality expansion, new charting in adhoc etc., at any place whenever you wish, using your own in-house developers.
A first-of-its-kind Open-Source BI framework, Helical Insight is completely API-driven. This allows you to add functionalities, including but not limited to adding a new exporting type, new datasource type, core functionality expansion, new charting in adhoc etc., at any place whenever you wish, using your own in-house developers.
It handles huge volumes of data effectively. Caching, Pagination, Load-Balancing and In-Memory not only provides you with amazing experience, but also and does not burden the database server more than required. Further effective use of computing power gives best performance and complex calculations even on the big data even with smaller machines for your personal use. Filtering, Sorting, Cube Analysis, Inter Panel Communication on the dashboards all at lightning speed. Thereby, making best open-source Business Intelligence solution in the market.
It handles huge volumes of data effectively. Caching, Pagination, Load-Balancing and In-Memory not only provides you with amazing experience, but also and does not burden the database server more than required. Further effective use of computing power gives best performance and complex calculations even on the big data even with smaller machines for your personal use. Filtering, Sorting, Cube Analysis, Inter Panel Communication on the dashboards all at lightning speed. Thereby, making best open-source Business Intelligence solution in the market.
With advance NLP algorithm, business users simply ask questions like, “show me sales of last quarter”, “average monthly sales of my products”. Let the application give the power to users without knowledge of query language or underlying data architecture
With advance NLP algorithm, business users simply ask questions like, “show me sales of last quarter”, “average monthly sales of my products”. Let the application give the power to users without knowledge of query language or underlying data architecture
Our application is compatible with almost all databases, be it RDBMS, or columnar database, or even flat files like spreadsheets or csv files. You can even connect to your own custom database via JDBC connection. Further, our database connection can be switched dynamically based on logged in users or its organization or other parameters. So, all your clients can use the same reports and dashboards without worrying about any data security breech.
Our application is compatible with almost all databases, be it RDBMS, or columnar database, or even flat files like spreadsheets or csv files. You can even connect to your own custom database via JDBC connection. Further, our database connection can be switched dynamically based on logged in users or its organization or other parameters. So, all your clients can use the same reports and dashboards without worrying about any data security breech.
Our application can be installed on an in-house server where you have full control of your data and its security. Or on cloud where it is accessible to larger audience without overheads and maintenance of the servers. One solution that works for all.
Our application can be installed on an in-house server where you have full control of your data and its security. Or on cloud where it is accessible to larger audience without overheads and maintenance of the servers. One solution that works for all.
Different companies have different business processes that the existing BI tools do not encompass. Helical Insight permits you to design your own workflows and specify what functional module of BI gets triggered
Different companies have different business processes that the existing BI tools do not encompass. Helical Insight permits you to design your own workflows and specify what functional module of BI gets triggered