Skip to content

Latest commit

 

History

History
119 lines (81 loc) · 6 KB

datasource.md

File metadata and controls

119 lines (81 loc) · 6 KB

Configure Data Source

Supported data source list in CBoard as below:

  • JDBC
  • Native ElasticSearch 1.x, 2.x, 5.x (Generate DSL according to user's drag designer)
  • Kylin 1.6 (Native read kylin Model generate sql script according to user's drag designer)
  • TextFile (File should be put at the same server with CBoard)
  • Saiku2.x (Read data generated by saiku restful api rather than read saiku chart)
  • Solr4.x, 5.x (Read Solr collection, Generate Solr Query language according to user's drag designer, 4.x cannot use the background aggregation function)

Operation path like this:

1 JDBC

Be more honest, we don't wanna to list all kinds of database products that support standard JDBC connection. If you have knowledge of JDBC, you know it doesn't make sense. Basically, all relational databases can be connected. Maybe there are some compatible bugs due to syntax, but it's easy to fix. If you came across any this kind of bugs, don't hesitate to create an issue, we'll support to fix it.

Presto

Presto can also be connected through JDBC. The raw JDBC solution of Presto has no auth function,just leave the username and password as blank.

Impala

Impala can also be connected through JDBC,Impala JDBC Connection Driver can be Impala Driver com.cloudera.impala.jdbc4.Driver, can also be Hive Driver org.apache.hive.jdbc.HiveDriver.

Impala Driver no auth URL jdbc:impala://localhost:21050/default, auth URL jdbc:impala://localhost:21050/default;AuthMech=3.

Hive Driver no auth URL jdbc:hive2://localhost:10000/default;auth=noSasl, auth URL jdbc:hive2://localhost:10000/default.

About Hive and Spark

Hive and spark-sql can connected through JDBC, but due to the high latency of interactive query, both of them are not recommend use as data source

Methods of import driver

!> Before create new JDBC type datasource connection, make source related JDBC driver jars are included in WEB-INF\lib, MySQL, SQLServer and Kylin driver are pre-included

  1. Import by add dependence in pom.xml is recommend:
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <version>5.1.24</version>
</dependency>
  1. Copy external library to path WEB-INF\lib,and then restart Tomcat server

    * Once you have complete input all required information for JDBC, click Test button and write one simple query to test your connection * After version 0.3, there are two new added option * Pooled Connection: set whether to use connection pool of jdbc * Aggregatable DataProvider: set whether to push aggregation or calculation to datasource

2 Kylin 1.6

Connect using JDBC

Driver and jdbc url like below:

Driver :org.apache.kylin.jdbc.Driver
JdbcUrl:jdbc:kylin://<host>:<port>/<project>

In order to leverage the power of kylin datasource, it'd would be better to set you datasource as aggregatable data provider and push the aggregation to kylin.

!> If you choose to use kylin query result as offline data, it's very important to control the size of result dataset.

image

Why the result on subquery in kylin1.6 is not correct

Because subquery is not well supported as of kylin1.6, which is use in cboard to push aggregation to datasource, there are some bugs about wrong query result. If you come across this kind of bug, try to debug the real executed query through Query View feature of widget designer.

Kylin Native

From version 0.3, CBoard can read directly Apache Kylin-1.6 through its restful interface. Kylin 2.x will be supported in version 0.5, coming soon...

!> Aggregation can only be done in data source when use native Kylin as datasource.

3 Elasticsearch

Elasticsearch is born for search. Nowadays more and more companies start to use it as analytical data source. Before version 5, Kibana can hardly meet the multi dimension analytic requirements. Although, after version 5, Kibana has ability to split metric by dimension. For calculated metric after aggregation, usually use to calculate convert rate, K5 can’t handle.(P.S. Maybe I have not been proficient in kibana, correct me if I were wrong)

Test Connection

4 Saiku2.x

CBoard can read result data before Saiku2.6 through it's restful interface. Due to the restful interface change in saiku3.x, it's not supported for now. Considering the theme between CBoard and Saiku chart are not that match, instead of directly include Saiku chart, we choose to read Saiku data as dataset and then render chart.

People who are familiar with Saiku known that the result of Saiku report always is a cross table with multilevel row header and column header. How does CBoard reuse this table.


The answer is all the row level will be combined as one line header. Out put columns of above cross table as below:

5 TextFile DataSource

For CBoard can only read local file that located at the same server as CBoard application. User can setup a ftp server to maintenance and upload your data file by yourself.

6 Solr

Solr datasource is added to the CBoard0.4 version. Solr4.x does not have the grouping collection function, and 5.0+ can use the datasource collection function.

Solr Datasource Configuration

Solr Test Connection