Sunday, May 8, 2016

building an interactive tree with d3.js php and sqlite: an annotated webliography

I want to make a tree with rectangular nodes containing text, such that I can add nodes or remove nodes on the fly. I want the tree to be "multiplayer": everybody who visits the site sees the same tree, and when one of them modifies it, it the updates are applied to everyone's view.
I know nothing about d3, and very little about PHP, but a bit about sqlite and Apache.
For this project I'm using a minimal number of libraries, and a very simple server setup, which leads to some inefficiencies. The server is Ubuntu 14.04 with Apache 2.4 with PHP 5.6. Client side the only library I'm using is d3.js 3.5.

A consequence of the simplicity is that I don't have a good way to push data to clients, so I can't use long polling or web sockets. Instead, I rely on repeated client requests for updates, which adds some latency, but hopefully not too much.

Requirements for graph:
Nodes are boxes that can contain about 50 words
terminal nodes can be added or removed (capability to add or remove internal nodes is not necessary)
root is on the left
subtrees can be expanded or collapsed
zoom-in zoom-out capability
pan capability
sort nodes with the older siblings above newer siblings

Let's get this figured out!

(see bottom of page for a link to a working version of the code)

Leanring d3.js:

Really concise explanation of data-joins. Read this!

Documentation for the selections feature of d3. Read this too...

another good introduction to D3

Making an updatable table with d3


detecting keypresses:

scroll to bottom of div:

Example and explanation of dragging and zooming:

super-simple zoom and pan:

For my application is was critical that I tied the zoom event handler to the <svg> element, but had the first inner <g> block take the transformation. If I didn't do this, then everything was jumpy and extreme and bad.

D3 as an AJAX library:

put scripts at the end of the page to execute them after the rest of the page has loaded.

d3 can act as a replacement for jquery for ajax requests

to append arbitrary xml content to a d3 selection use
an example of an ajax request is:
d3.html("login.html", function(error, html) {"#main").node().appendChild(html);

Generic javascript AJAX:

Use the setTimeout() function to set a function to get called after a delay. For example, after receiving an update from the server, set a timeout to request another update after a certain number of seconds.


remove specific items from a javascript array with splice.


We want to make the node a box around some text. To do this, we need to know the size of text.
One way to do this is with the svg getComputedTextLength() function. (this will not give us the height of the text).
Another way is to use the offsetHeight or offsetWidth functions.

wrapping text:

Generating and modifying the graph:

Official documentation of the D3 Tree Layout.
Create a tree with d3.layout.tree()
get nodes with tree(root)
get links with tree.links(nodes)

Essentially what tree does is take as input a hierarchical JSON object, and return a similar JSON object, but with x and y coordinates added. Passing that new object to links then returns a list of (source, target) pairs, where source and target are

set sorting withtree.sort([comparator]) #this will be useful for making

Here's a stackoverflow question with exactly the same issue I have. The code here isn't terribly helpful, but the link in the comments is (see the next link down on this page)

Tutorial about how to make a modifiable tree
the book associated with the previous link

Tutorial on how to update a d3 svg based on changing json data

d3 trees, by default are top-down trees. To make them left-to-right trees, just give x coordinates instead of y coordinates, and vice versa. For example:
      .attr("r", 10)
      .attr("cx", function(d) { return d.y; })
      .attr("cy", function(d) { return d.x; });
    .attr("class", "link")
    .attr("d", d3.svg.diagonal().projection(function(d) {return[d.y, d.x]}));

A better way is to write a recursive function go through and swap x and y coordinates immediately after generating the nodes object. For example:
nodes = tree.nodes(root); {var t = n.x; n.x = n.y; n.y = t});
links = tree.links(nodes);

Use "Tree.nodeSize" to change node size (apparently only works in d3 v3+):

By default, the edges look pretty bad (concave). To make them convex, swap the x and y (for reasons I don't fully understand yet).

PHP and SQLite3:
For queries that don't return results, use "exec"

For queries that do return results, use "query"

Better yet, use the "prepare" function with execute

To process a SELECT COUNT operation, use: "querySingle"

To send stuff to an error log use the command: error_log("Error message\n", 3, "path_to/php.log");

use "list" to separate multiple return values

Associative arrays and numeric indexed arrays are the same thing in PHP:

To convert an array to json, use json_encode: 

array map (perform function on all array elements), see top comment for how to pass keys and values:


You can access html data elements from css selectors using brackets:


Lots of examples of trees

Stack overflow answer with zooming and panning of a tree

collapsible tree from example from Bostock 

Special lessons and considerations:

One frustrating aspect of D3 is that the tree function (and I presume other layout functions) modifies the data it takes as an input. I presume this is for performance reasons (because javascript copies objects by refrence, and d3 doesn't want to make deep copies of anything, so it adds references to the new object, then modifies them), but the behavior is a bit frustrating and surprising. I think the solution is to format the node data as soon as it is received from the server so that it always has a consistent layout.

When you call a function on a selection, the function is called individually on all members of that selection. So when I select all nodes, and each node has only one rect as a child, I should use'rect'), and not nodes.selectAll('rect')

If you feed an empty object to layout.tree it does not return an empty object. What it returns is a list with one member: an object with properties depth=0, x=0, and y=0. That is: a tree with 1 node. I think this is another quirky consequence of JavaScript objects. JavaScript objects are never truly empty because they have lots of stuff they inherit just by being an object. In other words: it is impossible to use tree.nodes to create an empty tree.

The solution to this is, after creating the link nodes (there won't be any, because there's only one node). Clear out the nodes list (nodes=[]). Then downstream code using .data(nodes) will work, and you can safely update the tree so that it contains no nodes. The first 6 lines of code of my graph_refresh function are thus:

function graph_refresh(root) {
 var nodes = tree.nodes(root); {var t = n.x; n.x = n.y; n.y = t});
 var links = tree.links(nodes);
 if (!(nodes[0].hasOwnProperty('id'))) {


here's the link

here are the instructions:

There are currently three users:
The passwords for all of them are "123"
You can make new users too if you want.

Once you make and join a chart, you can modify it by entering commands into the chat window.
There are currently 5 possible commands:
reply:  @X message (where X is the number of an existing node. To make the first node, use @0. The other member has to approve of a node before anyone can reply to it)
approve: #X (where X is the number of an unapproved node)
remove: !rmX (X must be an unapproved node that you are the author of)
modify: !mdX (X must be an unapproved node that you are the author of)
make public: !p (once both users have entered this, then any registered user can look at the graph)
Any time you enter something that doesn't start with !,@, or #, it will be interpreted as chat, and go into the chat window.
You can use the mousewheel to zoom in and out, and click and hold to pan around.


No comments:

Post a Comment