Working with joins in d3.js

Working with joins in d3.js

D3.js is a well known JavaScript library for creating charts and other visuals using HTML, SVG and CSS. At first you may feel over-whelmed by looking at the features it has to provide. But as you start using it, everything will sink in. The main key behind understanding d3 is the how that data is related to DOM nodes, which is known as joins.

Understanding Joins

Let us assume that we have selected some DOM nodes using one of the select methods provided by d3 and stored it in a variable selection. Now we can assign data to selection by selection.data(data).

Suppose we have n points in our data but there are less than n DOM nodes, extra nodes can be created by selection.enter(). Note that these nodes will be empty.

On other hand if there are greater than n DOM nodes, extra nodes are returned by selection.exit().

The philosophy behind this is Instead of telling D3 how to do something, tell D3 what you want.

So general d3 script will look something like this:

var selection = d3.selecAll("circle");

selection.data(data);

selection.exit().remove();

selection.enter().append('circle') //....and so on

For illustartive example, you can have look at this post by Mike Bostock, the creator of D3.js.

Simple charts with C3.js | D3-based reusable chart library

 

Creating Simple charts with C3.js

This Blog aims to explain how to use a library called C3 for building simple charts. C3 is a javascript library which builds on top of famous D3.

C3 makes it easy to generate D3-based charts by wrapping the code required to construct the entire chart. We don’t need to write D3 code any more.

C3 gives some classes to each element when generating, so you can define a custom style by the class and it’s possible to extend the structure directly by D3.
Getting started with c3.js
1. Setup
Download the latest version here:
https://github.com/masayuki0812/c3/releases/latest
Then, load the scripts and css:

<!-- Load c3.css -->
<link href="/path/to/c3.css" rel="stylesheet" type="text/css">
<!-- Load d3.js and c3.js -->
<script src="/path/to/d3.v3.min.js" charset="utf-8"></script>
<script src="/path/to/c3.min.js"></script>

C3 depends on D3, so please load D3 too.
2. Generate Chart
C3 generates a chart by calling generate() with the argument object, and a element including the chart will insert into the element specified as a selector in that argument as bindto.
Prepare the element to bind the chart:

<div id="chart"></div>

And, call generate() with arguments:


var chart = c3.generate({
    bindto: '#chart',
    data: {
      columns: [
        ['data1', 30, 200, 100, 400, 150, 250],
        ['data2', 50, 20, 10, 40, 15, 25]
      ]
    }
});

C3 supports the asynchronous module definition (AMD) API. If you use RequireJS, you can load like this:


require.config({
  baseUrl: '/js',
  paths: {
    d3: "http://d3js.org/d3.v3.min"
  }
});

require(["d3", "c3"], function(d3, c3) {
  c3.generate({
    ...
  });
});

Then, you will see the chart:
C3 sample
Data can be loaded as columned data / rowed data / csv in URL/Json data.
There are serveral options to customize the chart and you can see those here:
Examples
3. Customize Chart
The chart can be customize by giving some options when generating. We will introduce some of them here.
1. Additional Axis
Introduce additional axis for data2. Add data.axes and axis.y2.show as follows:


var chart = c3.generate({
    bindto: '#chart',
    data: {
      columns: [
        ['data1', 30, 200, 100, 400, 150, 250],
        ['data2', 50, 20, 10, 40, 15, 25]
      ],
      axes: {
        data2: 'y2' // ADD
      }
    },
    axis: {
      y2: {
        show: true // ADD
      }
    }
});

 

Then, the chart will be like this:
Additional y axis
2. Show Axis Label
Show labels for each axis. Add axis.y.label and axis.y2.label as follows:


var chart = c3.generate({
    bindto: '#chart',
    data: {
      columns: [
        ['data1', 30, 200, 100, 400, 150, 250],
        ['data2', 50, 20, 10, 40, 15, 25]
      ],
      axes: {
        data2: 'y2'
      }
    },
    axis: {
      y: {
        label: { // ADD
          text: 'Y Label',
          position: 'outer-middle'
        }
      },
      y2: {
        show: true,
        label: { // ADD
          text: 'Y2 Label',
          position: 'outer-middle'
        }
      }
    }
});

Then, the chart will be like this:
Adding axis Lable
3. Change Chart Type
Show data2 as Bar chart. Add data.types as follows:


var chart = c3.generate({
    bindto: '#chart',
    data: {
      columns: [
        ['data1', 30, 200, 100, 400, 150, 250],
        ['data2', 50, 20, 10, 40, 15, 25]
      ],
      axes: {
        data2: 'y2'
      },
      types: {
        data2: 'bar' // ADD
      }
    },
    axis: {
      y: {
        label: {
          text: 'Y Label',
          position: 'outer-middle'
        }
      },
      y2: {
        show: true,
        label: {
          text: 'Y2 Label',
          position: 'outer-middle'
        }
      }
    }
});

Then, the chart will be like this:
Bar Chart
4. Format values
Format the values of each data. Add axis.y.tick.format as follows:


var chart = c3.generate({
    bindto: '#chart',
    data: {
      columns: [
        ['data1', 30, 200, 100, 400, 150, 250],
        ['data2', 50, 20, 10, 40, 15, 25]
      ],
      axes: {
        data2: 'y2'
      },
      types: {
        data2: 'bar'
      }
    },
    axis: {
      y: {
        label: {
          text: 'Y Label',
          position: 'outer-middle'
        },
        tick: {
          format: d3.format("$,") // ADD
        }
      },
      y2: {
        show: true,
        label: {
          text: 'Y2 Label',
          position: 'outer-middle'
        }
      }
    }
});

Then, the chart will be like this:
Format Values
More information about the options, please see Examples.
4. Customize Style
C3 give some classes for each element when generating. So, you can change the style of the elements by using those classes.
1. Line style
The lines have c3-line-[id] class, so this class can be used to define the style in css as follows:


#chart .c3-line-data2 {
  stroke-width: 5px;
}

Then, the chart will be like this:
Customize chart
Check Demo of this c3.js library
For more information check examples or fork on Github.

-By
Nitin Uttarwar
Helical It Solution

Making a Simple Interactive Map in HDI (Helical Dashboard Insights)

 

Creating Interactive and Zoomable Map in HDI (Helical Dashboard Insight)

 
The Goal of this blog is how to make responsive, interactive and zoomable Map in HDI (Helical Dashboard Insights):

For creating the Map in HDI, we are using D3.js , a javascript library.

The Data to use :

A special geospatial file called a Topojson. Here we are going to use a file that is comprised of all US counties. If u go to this link and copy into text file and save it as “us.json” (or anything .json) .

Since we have a county map of USA, we will need some data that is broken down by county. The us.json file we are using only has counties drawn. For this tutorial we are using data from query in json format.

Whatever data you have, make sure that there is a column that associates the information to a naming or id standard that is also present in your map/topojson.

Integrating Map in HDI:

To start integration of Map we have to change four files in HDI.

1) EFW file: EFW contain the Title, author, description, Template name, visibility of the Dashboard.

2) HTML File:HTML file name should be the same that specified in the EFW file under the Template Section.

In HTML File On Top we specify links of the external link and CSS properties.
Here we are using the ‘topojson.js’ external Library and it specified in the HDI as below:

<script src="getExternalResource.html?=Map/topojson.js"></script>

And CSS used to create Map is as follows:

.states {
  fill: none;
  stroke: #fff;
  stroke-linejoin: round;
}
body {
 font-family: Arial, sans-serif;
}
.city {
 font: 10px sans-serif;
 font-weight: bold;
}
.legend {
 font-size: 12px;
}
div.tooltip {
 position: absolute;
 text-align: left;
 width: 150px;
 height: 25px;
 padding: 2px;
 font-size: 10px;
 background: #FFFFE0;
 border: 1px;
 border-radius: 8px;
 pointer-events: none;
}

We have to declare one Map component in “Map.html” file and in this component we need to provide the link of the file where the Map chart property is defined.

3) EFWD File: EFWD file contain the Data Source Connection Properties such as connection id and connection type.It also contain Url of the connection to the database, User name and Password to the Database.

The DataSource Details used in our demo is shown as below:-

<DataSources>
        <Connection id="1" type="sql.jdbc">
           <Driver>com.mysql.jdbc.Driver</Driver>
           <Url>jdbc:mysql://192.168.2.9:3306/sampledata</Url>
            <User>devuser</User>
            <Pass>devuser</Pass>
        </Connection>
    </DataSources>

Data Map contains Map id and connection and connection Type. Map id is same as that specified in the EFWVF.

Query for the Data Map and the Parameter to be used is specified in the Tags and Parameter in the Tags.

<DataMap id="2" connection="1" type="sql" >
       <Name>Query for Tooltip </Name>
		<Query>
			<![CDATA[
					SELECT  cl.ID as id,cl.Name as name,sum(fact.votes) as votes
					FROM
					Voting_Summary as fact,region as r,contest as ct,county_list as cl
					where
					fact.region_id=r.region_id and
					fact.contest_id=ct.contest_id and
                   		        cl.Name = r.county	
			]]>
              </Query>
</DataMap>

4)EFWVF File :-

In EFWVF file we first set the chart id the chart we set the chart properties. For Map, we set the Map Properties between the tag. The properties such as Chart name, Chart type, Chart Data Source.

“Path” refers to lines drawn as instructed by our topojson file (us.json). Notice that .legend and .tooltip refer to objects we’ll designate with our javascript, but we can still set what they’ll look like here in the CSS.

You’ll see a lot of “var=”, which is setting up our variables for the code. Note that the first of the variables affect what values map to what colors. See that changing up these variables is an easy way to change the appearance of this map (as well as the CSS).

Colors are coded by RGB HEX value . There are multiple ways to scale colors, but this is the one we’ll go with here.

In Script we set the Map as Below :

Setting the Map Size, Position. And translation.

var width = 960,
 height = 500,
centered; 

Setting up the view:

var projection = d3.geo.albersUsa()
    .scale(1070)   // If scale is specified, this sets the projection’s scale factor to the specified value.
    .translate([width / 2, height / 2]);

Defing Map and Legend Color :

var color_domain = [5000,10000, 15000, 20000, 25000, 30000, 35000, 40000, 45000, 50000, 55000, 60000]
 var ext_color_domain = [0, 5000,10000, 15000, 20000, 25000, 30000, 35000, 40000, 45000, 50000, 55000, 60000]
 var legend_labels = ["< 5000","10000", "15000+", "15000+", "20000+", "25000+", "30000+", "35000+", "40000+", "45000+", "50000+", "55000+", "60000+"]
 var color = d3.scale.threshold()
 .domain(color_domain)
 .range(["#CCE0FF","#B2D1FF","#99C2FF","#80B2FF","#66A3FF","#4D94FF","#3385FF","#1975FF","#005CE6","#0052CC","#0047B2","#003D99","#003380","#002966"]);

The follow portion of code creates a new geographic path generator


var path = d3.geo.path()
    .projection(projection);

The next block of code sets our svg window;

var svg = d3.select("#chart_4").append("svg")
.attr("viewBox", "0 0 " + width + " " + height)
 .style("margin", "10px auto");
var div = d3.select("#chart_4").append("div")
 .attr("class", "tooltip")
 .style("opacity", 0);
svg.append("rect")
    .attr("class", "background")
    .attr("viewBox", "0 0 " + width + " " + height)
    .on("click", clicked);

Since our data file contains the json data returned from query and this data is used to map the tooltip.

 var pairIdWithId = {};
 var pairNameWithId = {};
var pairVotesWithId = {};
 
data.forEach(function(d) {
 pairIdWithId[d.id] = +d.id;
 pairNameWithId[d.id] = d.name;
 pairVotesWithId[d.id] = d.votes; 
 });

here d.id ,d.name and d.votes refer to the column headers of our query.And now we’ll select the svg objects we’ve created but not specified, and map our data onto them:

var g = svg.append("g");
 g.append("g")
 .attr("class", "county")
 .selectAll("path")
 .data(topojson.feature(data1, data1.objects.counties).features)
 .enter().append("path")
 .attr("d", path)
 .on("click", clicked)
 .style ( "fill" , function (d) {
 return color (pairIdWithId[d.id]);
 })
.style("opacity", 0.8)

This will draw each county as an object, each with its own values. Notice that we’ve named this class of object “county”.

If we wanted to change the style of the counties in CSS up at the top, we could just refer to .county and make changes Also the “.data” line associates information from our us.json file with the county objects.

Also important is that “color” refers to the function set above in the code. “Color” expects a number as input, but instead of a specific number, we’re going to give it our container filled with pairs of ID numbers and rate values, and use [d.id] to make sure that we read in a value for each id number.The rest is what happens when the mouse glances over the county:


.on("mouseover", function(d) {
 d3.select(this).transition().duration(300).style("opacity", 1);
 div.transition().duration(300)
 .style("opacity", 1)
 div.text(pairNameWithId[d.id] + " : " + pairVotesWithId[d.id])
 .style("left", (d3.event.pageX) + "px")
 .style("top", (d3.event.pageY -30) + "px");
 })
 .on("mouseout", function() {
 d3.select(this)
 .transition().duration(300)
 .style("opacity", 0.8);
 div.transition().duration(300)
 .style("opacity", 0);
 })

If you want to change what each label is, make sure to adjust the variable “legend_labels.”


var legend = svg.selectAll("g.legend")
 .data(ext_color_domain)
 .enter().append("g")
 .attr("class", "legend");
 
var ls_w = 20, ls_h = 20;
legend.append("rect")
 .attr("x", 20)
 .attr("y", function(d, i){ return height - (i*ls_h) - 2*ls_h;})
 .attr("width", ls_w)
 .attr("height", ls_h)
 .style("fill", function(d, i) { return color(d); })
 .style("opacity", 0.8);
 
legend.append("text")
 .attr("x", 50)
 .attr("y", function(d, i){ return height - (i*ls_h) - ls_h - 4;})
 .text(function(d, i){ return legend_labels[i]; });

Function that gives Zoom Functionality to the map


function clicked(d) {
  var x, y, k;
  if (d && centered !== d) {
    var centroid = path.centroid(d);
    x = centroid[0];
    y = centroid[1];
    k = 4;
    centered = d;
  } else {
    x = width / 2;
    y = height / 2;
    k = 1;
    centered = null;
  }

  g.selectAll("path")
      .classed("active", centered && function(d) { return d === centered; });

  g.transition()
      .duration(750)
      .attr("transform", "translate(" + width / 2 + "," + height / 2 + ")scale(" + k + ")translate(" + -x + "," + -y + ")")
      .style("stroke-width", 1.5 / k + "px");
}

By following these we are able to see the output which is as follows:
D3 Map

-By
Nitin Uttarwar
Helical It Solution

D3 Sankey Chart Integration with Jaspersoft

In this blog we will be discussing about D3 Sankey Chart Integration with Jaspersoft using HTML method of integration.

All the reports are develop using ireport 5.5 professional and jasper server 5.5

As html component of jasper server does not load any scripts in the html component, we loaded the script in one of the decorator page(jsp page). The page is located at the location:

C:\Jaspersoft\jasperreports-server-5.5\apache-tomcat\webapps\jasperserver-pro\WEB-INF\decorators\decorator.jsp

In the page we included the scripts which we want to load. We added the following code in the jsp page at line no 46:

<script type="text/javascript" language="JavaScript"
src="${pageContext.request.contextPath}/scripts/d3.v3.min.js"></script>

The script to be added should be kept at location:

C:\Jaspersoft\jasperreports-server-5.5\apache-tomcat\webapps\jasperserver-pro\scripts

Sankey Chart

Sankey D3 Jaspersoft

Sankey D3 Jaspersoft

For this chart we need to include one more js script file in the decorator page as described in the start of the document.
The js file is sankey.js and can be downloaded from
https://github.com/d3/d3-plugins/blob/master/sankey/sankey.js

Sankey diagrams visualize the magnitude of flow between nodes in a network. Sankey diagrams are a specific type of flow diagram, in which the width of the arrows is shown proportionally to the flow quantity. They are typically used to visualize energy or material or cost transfers between processes.
The diagram will contain two quantities, 1) All the nodes. 2) Which nodes are connected and with what value.

The json object should be like : http://bost.ocks.org/mike/sankey/energy.json

    Integration with JasperServer:

 
The data which we use for developing the calendar view can be fetched from any database. The data fetched from database is stored in a variable and is then accessed in the html component using the same variable. Applying this of process makes the report dynamic instead of static. Few parameters can also be added in the report which can be used in query and/or html component.
Generally for these type of charts we pass a variable which contains required data containing list of quantity1(employees or people or students, etc.) and a value associated with different type of skills. The string is created according to JSON format, so that when accessed in script tag, can be easily converted to JSON object using eval function.

Any variable/parameter can be accessed as shown below:
" var arr ="+$V{variable1}+" "

Parameter in query:
Select * from table_name
where date between $P{parameter1} and $P{parameter2}

The sample code of static chart is shown below at:

 

#chart {
height: 500px;
}

.node rect {
cursor: move;
fill-opacity: .9;
shape-rendering: crispEdges;
}

.node text {
pointer-events: none;
text-shadow: 0 1px 0 #fff;
}

.link {
fill: none;
stroke: #000;
stroke-opacity: .2;
}

.link:hover {
stroke-opacity: .5;
}

var margin = {top: 1, right: 1, bottom: 6, left: 1},
width = 960 – margin.left – margin.right,
height = 500 – margin.top – margin.bottom;

var formatNumber = d3.format(“,.0f”),
format = function(d) { return formatNumber(d) + ” TWh”; },
color = d3.scale.category20();

var svg = d3.select(“#chart”).append(“svg”)
.attr(“width”, width + margin.left + margin.right)
.attr(“height”, height + margin.top + margin.bottom)
.append(“g”)
.attr(“transform”, “translate(” + margin.left + “,” + margin.top + “)”);

var sankey = d3.sankey()
.nodeWidth(15)
.nodePadding(10)
.size([width, height]);

var path = sankey.link();
var energy = {“nodes”:[ {“name”:”Energy”}, {“name”:”Industrial Processes”}, {“name”:”Electricity and heat”}, {“name”:”Industry”}, {“name”:”Land Use Change”}, {“name”:”Agriculture”}, {“name”:”Waste”} ], “links”:[ {“source”:0,”target”:1,”value”:12}, {“source”:1,”target”:2,”value”:20}, {“source”:2,”target”:3,”value”:30}, {“source”:4,”target”:1,”value”:8}, {“source”:4,”target”:2,”value”:10}, {“source”:4,”target”:5,”value”:5}, {“source”:5,”target”:6,”value”:5} ]};

sankey
.nodes(energy.nodes)
.links(energy.links)
.layout(32);

var link = svg.append(“g”).selectAll(“.link”)
.data(energy.links)
.enter().append(“path”)
.attr(“class”, “link”)
.attr(“d”, path)
.style(“stroke-width”, function(d) { return Math.max(1, d.dy); })
.sort(function(a, b) { return b.dy – a.dy; });

link.append(“title”)
.text(function(d) { return d.source.name + ” → ” + d.target.name + “\n” + format(d.value); });

var node = svg.append(“g”).selectAll(“.node”)
.data(energy.nodes)
.enter().append(“g”)
.attr(“class”, “node”)
.attr(“transform”, function(d) { return “translate(” + d.x + “,” + d.y + “)”; })
.call(d3.behavior.drag()
.origin(function(d) { return d; })
.on(“dragstart”, function() { this.parentNode.appendChild(this); })
.on(“drag”, dragmove));

node.append(“rect”)
.attr(“height”, function(d) { return d.dy; })
.attr(“width”, sankey.nodeWidth())
.style(“fill”, function(d) { return d.color = color(d.name.replace(/ .*/, “”)); })
.style(“stroke”, function(d) { return d3.rgb(d.color).darker(2); })
.append(“title”)
.text(function(d) { return d.name + “\n” + format(d.value); });

node.append(“text”)
.attr(“x”, -6)
.attr(“y”, function(d) { return d.dy / 2; })
.attr(“dy”, “.35em”)
.attr(“text-anchor”, “end”)
.attr(“transform”, null)
.text(function(d) { return d.name; })
.filter(function(d) { return d.x < width / 2; })
.attr(“x”, 6 + sankey.nodeWidth())
.attr(“text-anchor”, “start”);

function dragmove(d) {
d3.select(this).attr(“transform”, “translate(” + d.x + “,” + (d.y = Math.max(0, Math.min(height – d.dy, d3.event.y))) + “)”);
sankey.relayout();
link.attr(“d”, path);
}

 

The steps on how to integrate it with jasperserver was discussed in my previous blog(D3 Integrating with Jasperserver).

Avi Dawra
Helical IT Solutions