How to resolve Jasperserver version compatability problem when migrating reports from one version to another

This blog will talk about how to make the report compatible with another version where it is being deployed.

Software used :- I-report, jasperserver (any version)

Solution:-

Step1) Open Your  report(.jrxml) in i-report

Jasper Version compatibility issue

step2)Goto Tool menu->options ->ireport menu->general->compatability

Jasper Version compatibility issue2

Step 3)select version which you want of jasperserver.

Here eg: jasperreport 4.1.3

Step4)goto (.jrxml) file of report  and make some  little change like set band height whatever you want and save it and update to server-repository navigator.

Step 5)check on your jasper server

How to create Backup using C,Best Practices while creating CSV or txt file using C

How to create   Backup  using c?

Backup   is  the  process  of  taking  the  backup of  Data  and  saving it on another location  for future Reference. This process includes

  1. Creating   CSV files or txt files  of   data on
    1. Daily basis
    2. Hourly basis.
    3. By particular Date.
  2. Making zip or creating   tar for all created the files.
  3. Uploading created tar file on server or on ftp or on our drive
  4. Again we have to download that .tar file in our application.
  5. Unzip that file to get all CSV/TXT file.
  6. Read all the Data from CSV/TXT file.

 

How to create CSV/TXT  File?

To create CSV/TXT File by using C –language, first we have to decide

  1. What  is  the  content  of  CSV  file.(Table Values)
  2. From where  we can  get those content.(In our case Database)
  3. Where we are going to store these files.(Output file Location)
  4. What is the Name of Our CSV File?(CSV file Name)

So  in  our Case to get the data from Database we have to do,

  1. Open  the Database.

int open = openDB(&db, (char *)conf->dbPath, conf->dbConnectionTimeout);

  1. Check  or validate  that  Db is  opened  or  not?

If( open != 0){   closeDb();           }

  1. Create your  SQL Query to get the data from Tables.

sprintf(query,”select * from <table-name> where <condition>”);

  1. Do  File Handling , create  the file and variable.

createFile(filePath)

char *writeData=NULL;

writeData = calloc(1024,sizeof(char));

  1. Fire the SQL Query on Specific Tables and get the value in variable.

result = executeQuery((char *)query, &db, &stmt);

strcat(writeData,sqlite3_column_name(*stmt,col));

 

  1. Do the Formatting while Data is appending in variable.

if(col >= cols -1)

{

       strcat(writeData,”\n

   }

  else

  {

        strcat(writeData,”,”);

   }

  1. Then check file is present or not , if not then create it .

if(!isFilePresent(filePath))

 {

    if(createFile(filePath))

      {

             printf(“File (%s) created!\n”,filePath);

       }

 }

  1. Lastly write the value of variable in File.

status = appendFile(filePath, &writeData);

  1. Release the memory of variable.

free(writeData);

  1. Finalize your Statement.

sqlite3_finalize(stmt);

  1. Close the Database.

closeDB(&db);

 

Flow of  creation.

  1. Read the value of DB Path, Opfilepath, log path from pushagentconfig file.
  2. Read   the value   from backlogconfig  file  ,Date and Time, FTP related information.
  3. Append both into single variable.
  4. According to that variable  fire query on database to get the timestamp
  5. After that according to that Starting timestamp   and ending timestamp.
  6. Then we are reading the type ,
  7. According to  given type we assign the value of interval variable day(86400),Hour(3600),

(last-start) timestamp

  1. After getting all the detail run the  for loop from start to end timestamp. by   assigning the interval.
  2. Append the   opFilepath, date and filename.
  3. Prepare the query and call  the  function Creating CSV  by passing Filename, Cols, stmt and length
  4. Then CreatingCSV   function called, in that function we declare one variable with some memory and appended all the data’s in that variable.
  5. Write the formatted content   of variable in file.
  6. Releasing the memory. By   this way one table data come into one csv   file with header.
  7. Again   for next table new filename, query will be created and the same process is   going on.
  8. After every single File creation, we are finalizing the statement making the query-variable content to null to reuse it.
  9. Lastly closing the Database.

Exporting project folder from jasper community server

Exporting project folder from jasper community server



Exporting=Downloading

1) PuTTY
Give username and password

2) Go to the location of jasperserver cp 5.0.0(or any version)
Example:
[email protected]:cd /opt/jasperreports-server-cp-5.0.0

3) Navigate to buildomatic folder
Example:
[email protected]:cd /opt/jasperreports-server-cp-5.0.0/cd buildomatic

4) Execute js-export.sh file with the destination path.

Syntax :
[email protected]:cd /opt/jasperreports-server-cp-5.0.0/buildomatic#./js-export <location of the folder in jasper server> –output-zip <location of exporting folder>/<exporting_filename.zip>

[email protected]:/opt/jasperreports-server-cp-5.0.0/buildomatic# ./js-export.sh /reports/Demos_Sadakar –output-zip /home/sadakar/delete/sadakar.zip

Using CE setup
Using Bundled version of Java
First resource path:/opt/jasperreports-server-cp-5.0.0/buildomatic/conf_source/ieCe/applicationContext-search.xml
Started to load resources
Resource name: applicationContext.xml
Resource name: applicationContext-cascade.xml
Resource name: applicationContext-data-snapshots.xml
Resource name: applicationContext-events-logging.xml
Resource name: applicationContext-export-config.xml
Resource name: applicationContext-export-import.xml
Resource name: applicationContext-logging.xml
Resource name: applicationContext-olap-connection.xml
Resource name: applicationContext-report-scheduling.xml
Resource name: applicationContext-search.xml
Resource name: applicationContext-security.xml
Resource name: applicationContext-themes.xml
Resource name: applicationContext-virtual-data-source.xml
Creating ActionModelService object.
Creating action model infrastructure.

Nikhilesh
Helical IT Solutions

Pentaho 5.0.1 CE integration with MySQL 5.0.1 CE (Windows or Linux family)

Pentaho 5.0.1 CE integration with MySQL 5.0.1 CE (Windows or Linux )

Parts

  1. Creating databases
  2. Modifying configuration files
  3. Stopping HSQL db start up

Creating databases

Command to execute the scripting files

mysql>source  D:\ biserver-ce\data\mysql5\create_jcr_mysql.sql

Similarly execute the remaining .sql files(i.e, execute create_quartz_mysql.sql and create_repository_mysql.sql)

  1. Check the databases created using show databases command on MySQL command prompt.

 

Modifying configuration files

1. applicationContext-spring-security-hibernate.properties.

Edit the file pentaho-solutions\system\applicationContext-spring-security-hibernate.properties.

Original code

jdbc.driver=org.hsqldb.jdbcDriver

jdbc.url=jdbc:hsqldb:hsql://localhost:9001/hibernate

jdbc.username=hibuser

jdbc.password=password

hibernate.dialect=org.hibernate.dialect.HSQLDialect

Modified code

jdbc.driver=com.mysql.jdbc.Driver

jdbc.url=jdbc:mysql://localhost:3306/hibernate

jdbc.username=hibuser

jdbc.password=password

hibernate.dialect=org.hibernate.dialect.MySQLDialect

  2. hibernate-settings.xml

Edit the file pentaho-solutions\system\hibernate\hibernate-settings.xml.

Original code

<config-file>system/hibernate/hsql.hibernate.cfg.xml</config-file>

Modified code

<config-file>system/hibernate/mysql5.hibernate.cfg.xml</config-file>

 

3. mysql5.hibernate.cfg.xml

Location of the file: pentaho-solutions\system\hibernate\mysql5.hibernate.cfg.xml

Do need to change any code in this file.. Just need to check everything is perfect or not

<property name="connection.driver_class">com.mysql.jdbc.Driver</property>

<property name="connection.url">jdbc:mysql://localhost:3306/hibernate</property>

<property name="dialect">org.hibernate.dialect.MySQL5InnoDBDialect</property>

<property name="connection.username">hibuser</property>

<property name="connection.password">password</property>

4. quartz.properties

Location of the file: pentaho-solutions\system\quartz\quartz.properties

Original Code

org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.PostgreSQLDelegate

Modified Code

org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate

5. context.xml

Location of the file: tomcat\webapps\pentaho\META-INF\context.xml

Original Code

<Resource name="jdbc/Hibernate" auth="Container" type="javax.sql.DataSource"

factory="org.apache.commons.dbcp.BasicDataSourceFactory" maxActive="20" maxIdle="5"

maxWait="10000" username="hibuser" password="password"

driverClassName="org.hsqldb.jdbcDriver" url="jdbc:hsqldb:hsql://localhost/hibernate"

validationQuery="select count(*) from INFORMATION_SCHEMA.SYSTEM_SEQUENCES" />

 

<Resource name="jdbc/Quartz" auth="Container" type="javax.sql.DataSource"

factory="org.apache.commons.dbcp.BasicDataSourceFactory" maxActive="20" maxIdle="5"

maxWait="10000" username="pentaho_user" password="password"

driverClassName="org.hsqldb.jdbcDriver" url="jdbc:hsqldb:hsql://localhost/quartz"

validationQuery="select count(*) from INFORMATION_SCHEMA.SYSTEM_SEQUENCES"/>

Modified Code

<Resource name="jdbc/Hibernate" auth="Container" type="javax.sql.DataSource"

factory="org.apache.commons.dbcp.BasicDataSourceFactory" maxActive="20" maxIdle="5"

maxWait="10000" username="hibuser" password="password"

driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/hibernate"

validationQuery="select 1" />

 

<Resource name="jdbc/Quartz" auth="Container" type="javax.sql.DataSource"

factory="org.apache.commons.dbcp.BasicDataSourceFactory" maxActive="20" maxIdle="5"

maxWait="10000" username="pentaho_user" password="password"

driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/quartz"

validationQuery="select 1"/>

 Imp Note:

Delete pentaho.xml file in below location

tomcat\conf\Catalina\localhost\pentaho.xml

Reason:

Pentaho creates  on startup pentaho.xml as a copy of context.xml

6. repository.xml

Location of the file: pentaho-solutions\system\jackrabbit\repository.xml.

Comment this code means (<! – – every thing here – -> )

Active means: Remove comment

i)                    FileSystem part

Comment this code

<FileSystem>

<param name=”path” value=”${rep.home}/repository”/>

</FileSystem>

Active this code

<FileSystem>

<param name="driver" value="com.mysql.jdbc.Driver"/>

<param name="url" value="jdbc:mysql://localhost:3306/jackrabbit"/>

<param name="user" value="jcr_user"/>

<param name="password" value="password"/>

<param name="schema" value="mysql"/>

<param name="schemaObjectPrefix" value="fs_repos_"/>

</FileSystem>

ii)                  DataStore part

Comment this code

<DataStore/>

Active this code

<DataStore class="org.apache.jackrabbit.core.data.db.DbDataStore">

   <param name="url" value="jdbc:mysql://localhost:3306/jackrabbit"/>

   <param name="user" value="jcr_user"/>

   <param name="password" value="password"/>

   <param name="databaseType" value="mysql"/>

   <param name="driver" value="com.mysql.jdbc.Driver"/>

   <param name="minRecordLength" value="1024"/>

   <param name="maxConnections" value="3"/>

   <param name="copyWhenReading" value="true"/>

   <param name="tablePrefix" value=""/>

   <param name="schemaObjectPrefix" value="ds_repos_"/>

 </DataStore>

iii)                Security part in the FileSystem Workspace part

Comment this code

<FileSystem>

<param name=”path” value=”${wsp.home}”/>

</FileSystem>

   Active this code

<FileSystem>

<param name="driver" value="com.mysql.jdbc.Driver"/>

<param name="url" value="jdbc:mysql://localhost:3306/jackrabbit"/>

<param name="user" value="jcr_user"/>

<param name="password" value="password"/>

<param name="schema" value="mysql"/>

<param name="schemaObjectPrefix" value="fs_ws_"/>

</FileSystem>

iv)                PersistenceManager part

Comment this code

<PersistenceManager>

<param name=”url” value=”jdbc:h2:${wsp.home}/db”/>

<param name=”schemaObjectPrefix” value=”${wsp.name}_”/>

</PersistenceManager>

Active this code

<PersistenceManager>

<param name="url" value="jdbc:mysql://localhost:3306/jackrabbit"/>

<param name="user" value="jcr_user" />

<param name="password" value="password" />

<param name="schema" value="mysql"/>

<param name="schemaObjectPrefix" value="${wsp.name}_pm_ws_"/>

</PersistenceManager>

v)                  FileSystem Versioning part

Comment this code

<FileSystem>

<param name=”path” value=”${rep.home}/version” />

</FileSystem>

 

Active this code

<FileSystem>

<param name="driver" value="com.mysql.jdbc.Driver"/>

<param name="url" value="jdbc:mysql://localhost:3306/jackrabbit"/>

<param name="user" value="jcr_user"/>

<param name="password" value="password"/>

<param name="schema" value="mysql"/>

<param name="schemaObjectPrefix" value="fs_ver_"/>

</FileSystem>

vi)                PersistenceManager Versioning part

 

Comment this code:

 

<PersistenceManager>

<param name=”url” value=”jdbc:h2:${rep.home}/version/db”/>

<param name=”schemaObjectPrefix” value=”version_”/>

</PersistenceManager>

Active this code:

<PersistenceManager>

<param name="url" value="jdbc:mysql://localhost:3306/jackrabbit"/>

<param name="user" value="jcr_user" />

<param name="password" value="password" />

<param name="schema" value="mysql"/>

<param name="schemaObjectPrefix" value="pm_ver_"/>

</PersistenceManager>

Stopping HSQL db start up

In web.xml file

Comment or delete this code (Commenting is preferable)

<!– [BEGIN HSQLDB DATABASES] –>

<context-param>

<param-name>hsqldb-databases</param-name>

<param-value>[email protected]/../data/hsqldb/sampledata,[email protected]/../data/hsqldb/hibernate,[email protected]/../data/hsqldb/quartz</param-value>

</context-param>

<!– [END HSQLDB DATABASES] –>

 

Also comment this code

<!– [BEGIN HSQLDB STARTER] –>

<listener>

<listener-class>org.pentaho.platform.web.http.context.HsqldbStartupListener</listener-class>

</listener>

<!– [END HSQLDB STARTER] –>

 

 

You have done with integrating pentaho 5.0.1 CE with Mysql 5.5

Now login to the Pentaho server .

URL:  http://localhost:8080/pentaho

Username/Password : Admin/password

NOTE:

  • You will not find any sample working. B’z you have not installed sample data.
  • Example available in pentaho are developed on sample data so you need to execute and give the new sample data base connections(for this you need to execute .sql file of sample data).

Helical IT Solutions

Logging using Talend

Logging using Talend

Introduction: In this article, we will discuss about different methods of logging in Talend Open Studio. Talend is one of the widely used open source data integration tool in the market. Talend mainly uses three types of logging

  1. Statistics – Job execution statistics at component and job level
  2. Error – Job level errors, warning and exceptions
  3. Meter logging – Data flow details inside job

Best approach for logging is at project level. To enable project level logging, In Talend Open Studio, go to File, Project properties and enable or disable check boxes to start default logging at project level. See the screen shot below.

talend logging 1

If you enable the logging at project level, then every new job created will inherit these settings. There are more settings and options to do if you enable project level logging. See below screenshot.

talend logging 2

You can decide it to log the information to Console/File/Database. In case if you select any of File/Database or both options, then need to set few more default parameters like

talend logging 3

For file names, you can pre-or post fixes the file name with Talend Date Time stamp function. Or else it will write into the same file for every execution and flush out earlier data. In case of databases, you can have existing created database. This scenario does not work when you don’t have any database on the target server this scenario fails. For general JDBC, you need to provide above parameters, else if you select any database such as MySQL, then provide username, password and other required parameter values.

If you enable project level logging, then there is no need to separately use all these components say, tLogCatcher to Log the errors, tFlowMeter to catch the data flow and tStatCatcher to catch the Errors, flow and statistics log. Talend throws the errors or exceptions whenever it occurs and displays its complete trace on the console. tLogCatcher if used with the help of tDie or tWarn, would catch those messages and can be redirected to the required database / file based on the requirement. In order to do this we need to take care of all above components, implement and test the job.

Advantage with this approach is that you get the brief error information in logs table which is automatically created by Talend. In addition to this information Talend also prints its error log trace on console. And the negative side of this is that the console trace is not stored in the log table.

Problem: In both above approaches, Talend does not store or redirect its console trace to database or file.

Helical IT Solutions Launches “Helical Scrunch” – Press Release

Helical IT Solutions Launches “Helical Scrunch”

Helical Brings an Innovative Product to Solve ETL Problems- “Helical Scrunch”

  • One of its own kind, Helical Scrunch will reduce the time, effort & resource requirement by approx 30-70%
  • High end visualization & control of ETL jobs, status, errors, data flow, configurations etc

Regaining the faith of the clients in the company for bringing innovative products to make business more profitable and easy process working, Helical IT Solutions has launched its ambitious product HELICAL SCRUNCH into the market to solve the existing ETL issues faced by the companies.

Nitin Sahu, Co-founder, Helical IT Solutions said, “We are really excited to launch Helical Scrunch which will further lessen the complications which exists in ETL Solutions, thus resulting in saving time and resource requirement and creation of much better high quality enterprise ETLs. We have been working on this product for the past 3 months. This will be a new revolution in the way ETLs are created and used”.
He further explained, “ETL jobs are generally created for data migration, creation of data marts and data warehouse, data integration, data replication, data cleansing etc. Though this work can be handled by database SQL, yet ETL tools are used because of its ease of usage, built in objects like aggregators, easy debugging, good auditing capabilities etc. Even ETLs have many restrictions like very low visibility & control for an ETL admin, no reusability of ETL scripts, no standardization, error and logging etc, keeping all these restrictions of ETL tools in mind, Helical IT Solution has came up with a custom framework (known as Helical Scrunch), to work on top of an ETL tool, thus removing all the restrictions of the same.”
What is ETL?
ETL is shortened form of EXTRACT, TRANSFORM, LOAD.  In ETL, using any method data gets extracted from some source , then this data gets changed (transformed ) as per specific need, further changed ( transformed ) data gets loaded to another system mostly known as target system.
Problem Definition: Though there are many ETL tools available in the market, but using them also come with their own inherent problems, some of which are highlighted below:
Best Practices: Each and every developer does the ETL development according to his logic and his method of development; hence more often than not the best practices are not followed. These best practices are related to error handling, naming conventions, QA, QC etc.

Lack of standardization: Often not following the best practices on logging, error handling, naming conventions, documentation etc leads to lack of standardization between the different ETL jobs which have been developed amongst the different ETL developers.

Lack of control for end user: Generally in any ETL, an end user or IT administrator is often not able to see and monitor what exactly is happening. He has absolutely no control of the jobs, flags, status etc.

Lack of reusability: Generally, any ETL job is designed to tackle any specific problem, and not with reusability or long time picture in mind. So whenever there is any change, ETL job creation starts from the scratch.

Lack of monitoring: An end IT user or business user is having no option to monitor the progress of the job execution, what is the real time progress, logs and errors encountered if any etc.

Lack of visualization: Lack of visualization in ETL tools result thus result in an end user having no control and visibility on the history of the jobs execution, what jobs were executed, what jobs are executing, what error is being thrown etc.

Helical Scrunch
Pluggable: The Helical Scrunch has been designed in such a way that the different features are pluggable (like logging module, visualization module, status and notification module etc). This gives the developer freedom to select which all modules are to be present
Reusability: Helical Scrunch has been designed in such a way to make sure that the jobs created are usable. Having a standardized naming convention, features, documents etc further goes a long way in making sure that the jobs are reusable.
Control: Helical Scrunch provides an extensive control to an end user/IT admin via web interface. The control is very exhaustive which includes controlling and changing ETL configurations without opening ETL job, monitor data flows, controlling what to execute what not to execute etc.
Visualization: Helical Scrunch also provides, via web interface, extensive reports and dashboard capabilities. The reporting capabilities thus empower user to have real time view of the project status, error encountered, data transfer, data flow monitoring, which all jobs are executed, which all jobs are executing etc. There will also be ability to select date range for seeing the different parameters. Visualization helps in Monitoring and Analysis.
Alerting and Notification: The alerting and notification feature of Helical Scrunch helps in creating different kind of alerts. These alerts and notifications are configurable. These alerts can be set on certain events or thresholds. Notifications could include email alerts etc.
Reusability: Helical Scrunch has been designed in such a way that the jobs created are following standard conventions, best practices, naming conventions etc and thus they can be easily edited and highly reusable.
Extensibility: The architecture has been designed in such a way that easily any new module or feature can be added, which makes the entire framework highly extensible. Thus it can accommodate any new requirement or business logic or feature.
Logging: Helical Scrunch is designed in such a way to make sure that absolutely all the ETL work is properly logged. The extensive logging mechanism helps in visualization, identifying, alerting and taking corrective action.
Advantages:
Time Saving: Generally, whenever any ETL job is designed, a lot of work is involved in designing reusable components like error logging, custom logging, defining naming conventions, designing proper architecture etc. By usage of Helical Scrunch, a company can reduce ETL development time by 30-70%, and with a much better quality.
Resource Saving: Usage of this can lead to reduction of a number of resources required to execute the same project. ETL architects are not at all required and ETL developers to create generic ETL jobs and implement ETL architects are not required. The only work involved is in terms of business logic implementation. 30-70% of lesser ETL resources might be required for the implementation of the same work.
Quality of output: The ETL jobs created using this will be of a much higher quality.
Interactivity: A user interface for controlling of jobs, putting configurations, having a view of what is happening, reports, analysis etc gives a lot of interactivity and information to the end users.
Team Productive: Usage of Helical crunch can lead to the ETL team becoming effective and start working right from the day one. Helical Scrunch takes care of all the other things like nomenalcature, standardization, creation of jobs etc. Hence, ETL team can only need to focus on the problem in hand, which is actual implementation of the business logic.
About Helical IT Solutions

Helical IT Solutions is an open source DWBI company and has expertise in providing simple, practical & affordable solutions which are suitable for business users, right from CEO, CXO, line managers & to every end user of the enterprise. With a quick turnaround time, the company can provide mobile BI solutions, on premises or hosted SaaS solution, hence catering to every type of need. Helical offers services on entire BI stack, ranging from ETL, DW, Data mining, Analytics, BI solution. They also provide integration of disparate data sources and offers powerful interactive tools like balanced scorecards, personalized dashboards, key performance indicators, automated alerts, graphical mining, cross tab reporting and more! At present Helical impressive clients list include Unified Social Media, Vortecy Energy Consulting, Sage Human Capital – HR Business Intelligence, Predikto, hCentive- HealthCare insurance and many more.  Check www.helicaltech.com for more details

Top 10 Business Intelligence (BI) Trends for 2014

Business intelligence (BI) is a collection of applications and technologies which are used for gathering, storing, analyzing, and providing access to data to help enterprise users make better business decisions.

Business Intelligence has been one of the fastest growing segment in IT for some time now, CIOs and CTOs of different companies have been focusing more and more on the increased adoption and usage of Business Intelligence within their organization and for their clients as well. Gartner executive program survey of more than 2000 CIOs shows “Analytics and Business Intelligence” as the top priority followed by mobile technologies and cloud computing.

This article is an attempt to predict the Top 10 Business Intelligence trend for the year 2014

  1. Big Data Adoption: In the year 2013, big data was like a buzzword. Companies had heard about big data, but adoption wise it was very limited. 2014 will be the year when it will start becoming mainstream, we will witness really good adoption of big data by different companies.
  2. Data Overload: The amount of data generated and consumed will be increasing multiple times. Not only the traditional sources of data (like softwares, web services, social media) etc will generate immense data, but also Internet of Things (IOT) will also start contributing to the same. With IOT, literally everything will become a source of data right from fridge, printer,  machines, smart devices etc.
  3. Death of the reports: Day in day out, the amount of data generated and to be consumed by BI and thus to be visualized is increasing. All this will lead to less usage and requirement of the static reports. End user will want an ability to interact, play around with the data and derive insights. This kind of features and the huge size of data, static reports will not be able to
  4. Mobile BI: Slowly the balance of number of users viewing BI applications via mobile/smartphones/tabs is increasing. Gartner predicts that a third of BI users will be on mobile, and the consumption on the move will increase dramatically. BI vendors will put immense efforts of improving their native mobile applications, make them more interactive and releasing it on all the mobile platforms.
  5. Analytics on the rise: Companies expectations from a BI tool will shift from just being historical reporting to forward looking. Predictive analytics will become more prominent. More and more BI vendors will have predictive models, text and data analysis as one of the standard functionalities present. Thus BI suites will become more complex, leading to increase in in-memory capabilities, column oriented databases will become more prominent.
  6. Real time capabilities: With increasing velocity at which data is coming, the end user and business user requirement from a BI is changing drastically.  Having real time or almost near real time reporting and dashboarding capabilities will become very crucial for business users, and will affect a lot while selecting a BI vendor.
  7. Self service BI: With more and more data sources coming into picture, static reports and preconfigured dashboards are increasingly becoming irrelevant. Business users requirements are becoming very dynamic, and they are expecting self service BI capabilities from the solution. They want very less dependence on the IT people and want more and more control on the BI. This feature will soon become a deal breaker or deal maker in the near future while a company selects a BI solution.
  8. Collaborative BI: Collaborative BI is a growing trend which merges Business Intelligence and social media tools, thus augmenting sharing of relevant information. Collaborative BI will have features like rating, like, commenting, sharing, collaborating, brainstorming etc, thus out of a humongous number of reports/dashboards, only those which are really important are being used. More and more BI vendors are trying to incorporate collaborative BI capabilities inside their suite.
  9. Suggestive BI : Youtube videos sugest videos based on the current consumption, google suggests search results, Amazon suggests books based on purchase history and browsing etc. Soon, BI will also be expected to suggest the relevant KPI which can help a business user and decision maker make smarter decisions.
  10. Cloud BI : Cloud BI will become mainstream in 2014. The biggest factor driving this is the high cost of licenses of the different BI vendors. Over cloud, the license cost etc reduces a lot. Open source BI companies like Jaspersoft have came up with innovative pricing like pay per hour of usage, hence BI can also be afforded by small to medium companies as well. Not only cloud BI, but also big data over cloud will see a lot of interest, with amazon red shift and google big query at the forefront.

Thanks for reading the blog, your comments would be welcome : : [email protected]

Implement Load Balancing in Jaspersoft and Pentaho and Designing a Cluster

Load Balancing in Jaspersoft and Pentaho and Designing a Cluster

 

Load balancing is often done in BI infrastructure to have good scalability and availability. There can be a number of methods of implementing the same, one of the method is by using Load balancer and the BI server (jasper report server or Pentaho Server) can run behind the same. One very important factor to be kept in mind is the versions of the BI server should be the same (either both Community edition or both Enterprise edition). It’s also preferable that the BI suite version number is also the same in both the instances and the configurations are also exactly the same.

Jaspersoft Pentaho Load Balancing

A cluster is a group of server and a properly designed cluster can support many users and organizations, avoid downtime, fail-over, load balancing  and plan for future growth as well. This type of architecture can also take care of any future enhancement. An end user is not aware of this architecture and he will access the same URL.

 

Jaspersoft works on sticky session mechanism i.e. if switching happens from one server to another user sessions are lost.

 

Load Balancer: It’s a hardware or software to spread traffic between the cluster of servers. To an end user he will access the application via the same web URL, the load balancer will gauge the load on the different servers, and thus accordingly will spread the requests so that end user will have the best speed and response time. Its recommended that the Pentaho or the Jaspersoft server hardwares are exactly the same, which can thus lead to more effective work of load balancing by load balancer.

Different algorithms can be used at the time of load balancing like round-robin or load based or anything else.

 

 

Shared repository database:

This is having all the reports, folders, role, user, security and other resources. Whatever operations are done in the repository they are thread-safe, which means many operations can run simultaneously. Also there are internal locks present which prevents conflicting operations.

 

Job Scheduler:

Job scheduler is responsible for accessing and executing jobs at predefined schedule. These job schedulers also have locking mechanish, to make sure that simultaneous triggering dosent happen.

Also all the nodes need to be clock synced.

 

 

Session Management and Failover

Client session information is stored in –memory. After a user logs in or a web services clien’t sends a request with credentials, the session contains the user profile such as organization and role membership for use in enforcing permissions. For browser users, the session also stores information about the state of the web interface, for example the last folder viewed in the repository, the last search term, or the data entered in a wizard…..  In commercial editions with the Ad Hoc Editor and Dashboard Designer, the user session also stores the on-screen state of the report or dashboard that the browser user creates interactively.

 

There are two types of session management which can happen, and each of them handles the failover in a different way

–  Replicated or persistent sessions: Instantaneous session information of the sessions are continuously stored at a shared location. Whenever any failure happens, load balancer automatically redirects to another server node. Since the other server also has access to all the information, this happens seam-lessly without end user experience getting affected.

 

–  Sticky or Pinned session: Here the sessions are managed and accessed by private servers, hence at the time of failure sessions are lost. After load balancer connects to other server, a new session is initiated.

 

 

For more information about Jaspersoft BI suite or Pentaho BI suite, get in touch with us at [email protected]

PDI Kettle – Difference between CE and EE

Pentaho Data Integrator – Kettle along with Talend is one of the most famous Open Source ETL tool. KETTLE ( k- kettle, E- extract, T- Transform, T-Transport, L-Load, E-Environment).

However, it also does come in two variations i.e. Community version (free) and Enterprise version (paid). In this blog, we would try to cover what all are the additional features which are present in the Paid version.

 

–          Access Control:  Using this, admin can manage and control who all will have the right to create, modify and delete PDI transformations and jobs.

PDI Repository

 

–          Version Control:  This feature allows an admin to manage different jobs, managing and tracking their versions and transformations as well. It saves multiple copies and revisions of the job, and hence can take care of any deletion by mistake. Restoration can be, thus done, very easily.

PDI Repository Explorer

 

–           Production Control:  scheduling and monitoring jobs on a centralized server. Person can check the job and execute it without modifying. Scheduling and running the transformation at pre-defined time also possible.

PDI access controlPDI Job Scheduler

 

–          Integrated Security:  Presence of this component in PDI EE helps in recognizing users and roles from external corporate systems (like Spring Security Framework)

–          Content Management Repository:  This component is also present only in PDI EE version, it stores multiple versions of content and applying rules to content access (Apache JackRabbit)

–          Integrated Scheduling:  executing jobs on a centralized server at predetermined intervals (Quartz) is taken care by Integrated scheduling component present in the EE version.

 

 

Security Feature necessity:

–          In case if there are multiple users with different access right (like business, team leader, developer, operation etc), in those cases security feature helps in restricting the transformations. This user level security can be done via LDAP or CAS

 

Repository Feature

–          If there are multiple ETL developers who are working on the project, then it’s really important to have a repository sort of feature, so that every developer will have his own folder. Other developers can have read access to other ETL developers folder, that means they can only execute but cant modify

–          Repository also helps in maintaining the version information and copies as well

–          In CE version of ETL, the ETL copy will have to be saved separately since it dosent have repository feature

 

Data Integration Server Feature

–          This helps in remote deployment and monitoring of the transformation

–          You don’t need to be loggedin to execute the jobs

Data integration server also includes a scheduler that lets setup recurring schedules. Simple GUI allows to define things like start date, end date, logging level, frequency and when to repeat job execution, logging levels etc.

Please get in touch for more information at [email protected]

Jaspersoft System Integration (SI) Partner Helical IT Solutions

Helical, a fast growing Business Intelligent (BI) organisation, offering Open source BI solutions across verticals has been appointed by Jaspersoft as a System Integrator Partner. As per the tie-up, Helical IT Solutions will be a System Integration Partner for Jaspersoft in the India/South Asia region.

As a part of this appointment, Helical IT Solutions will provide services for Jaspersoft’s BI suite including reporting, dashboards, ad-hoc and OLAP analysis, ETL/data integration. Helical IT Solutions’ knowledge of both Business Intelligence applications and the Jaspersoft BI platform is bound to ensure successful development and deployment of BI solutions.

Taking on this appointment, Mr Nikhilesh Tiwari, Founder, Helical IT Solutions shared “We are extremely happy and delighted with this tie-up and regard this as a great achievement for our organisation. We will definitely look forward to this collaboration with Jaspersoft to be beneficial for both the companies.

 

Mr Nitin Sahu, Cofounder at Helical IT Solutions added, “We have been working on Jaspersoft BI platform for a long time and we are happy to be their SI. With our technical strength and partnership with Jaspersoft, we are hopeful of surpassing our customers’ expectations

 

Mr. Royce Buñag, Vice President Asia Pacific at Jaspersoft said “Helical IT Solutions has impressed us with their knowledge of BI solutions.  They have already shown themselves to be a valuable partner with current and ongoing customer engagements.  We are delighted that they have also agreed to be one of the sponsors for the upcoming JasperWorld event in Bangalore, which is a great way for us to showcase the collaboration to customers. We look forward to a long and successful partnership.

Jaspersoft’s open source business intelligence solutions  is the world’s most widely used BI software, with more than 8 million total downloads worldwide and more than 10,000 commercial customers in 96 countries. The Jaspersoft provides a web-based, open and modular approach to the evolving business intelligence needs of the enterprise. It has 90,000 registered members working on more than 350 projects, which represents the world’s largest business intelligence community.

Helical IT Solutions is an open source DWBI company and has expertise in providing simple, practical & affordable solutions which are suitable for business users, right from CEO, CXO, line managers & to every end user of the enterprise. With a quick turnaround time, the company can provide mobile BI solutions, on premises or hosted SaaS solution, hence catering to every type of need. Helical offers services on entire BI stack, ranging from ETL, DW, Data mining, Analytics, BI solution. They also provide integration of disparate data sources and offers powerful interactive tools like balanced scorecards, personalized dashboards, key performance indicators, automated alerts, graphical mining, cross tab reporting and more.

The press release got published at many places which includes

IndiaInfoline
http://www.indiainfoline.com/Markets/News/Jaspersoft-appoints-Helical-IT-Solutions-for-system-integration-partner/5810655543

Eletronics for You
http://www.efytimes.com/e1/fullnews.asp?edid=119811

Light-Reading
http://www.lightreading.in/lightreadingindia/news-wire-feed/285912/jaspersoft-appoints-helical-solutions-integration-partner

INfotechLead
http://infotechlead.com/2013/10/29/jaspersoft-appoints-helical-system-integration-partner-south-asia/

Channeltimes
http://www.channeltimes.com/story/jaspersoft-appoints-helical-it-solutions-as-partner-for-bi-solution/

Techvorm
http://www.techvorm.com/helical-ties-jaspersoft/

SIlobreaker
http://news.silobreaker.com/jaspersoft-appoints-helical-it–5_2267202378157523078

Digisecrets
http://www.digisecrets.com/news/jaspersoft-appoints-helical-it-solutions-as-their-system-integration-partner-for-indiasouth-asia-region/

CIOL
http://www.ciol.com/ciol/news/199620/jaspersoft-appoints-helical-integration-partner