Well, I guess it is time to finally let the cat out of the bag.  For the past few months Michael VanDaniker and I have been working away on a new open source project we have named Axiis (you can thank Juan Sanchez for the great name.)   We are still a couple months away from our first public alpha release, but I think it is worthwhile spending some time to discuss the details of what we have been developing. 

Axiis represents a new way to conceive of and develop interactive data visualizations.   It is based on Flex and ActionScript 3.0 but has a uniquely different approach to creating visualizations than the standard Flex charts, and pretty much any other data visualization package I have seen.  Our primary goal in developing Axiis is to create very concise, expressive, and flexible framework that uses a purely declarative (markup) based approach to create anything from simple cartesian charts to very complex and involved data visualizations.  

Den ganzen Beitrag lesen…

In doing client specific work, especially for business info-graphics, I often get client requests asking “can we see this in a pie chart?” For 90% of the use cases there is usually always a more effective way to visualize data than using a pie chart. As you may know, this is due to the fact that it is much harder to distinguish value differences in a polar layout than in more straightforward linear layouts (bar/column, etc.)

Den ganzen Beitrag lesen…

This data visualization design study is the first in (hopefully) a series of visualizations depicting US and Global Finance information.  My original goal was to tell a compelling story with this visualization, and stay true to many data visualization best practices.   But somewhere along the way, the artist in me became more absorbed in the aesthetics than the underlying story, and I abandoned many of the best practices I had hoped to employ.

As a result I think the data visualization still works for the main trends.  You can see annual trade surplus/deficit information for each of the top 10 countries for each (surplus/deficit.)  The countries that are in surplus (adding cash to the US Economy) are along the top, while the countries along the bottom show the corresponding trade deficits (that are taking cash out of the US Economy.)    It is pretty obvious that overall the US has far more deficit than surplus trade, and by a huge margin, and that in later years the US deficit with China is pretty large.

I did spend a fair amount of time working on a simple algorithm that represents the flow of cash as various weighted lines for each country (as shown in the top right legend.)  Visually I believe you can get a rough at-a-glance view of how much cash is flowing from each country, while still having enough fidelity at the lower volumes to discern differences between those volumes, since in essence it is a log scale in stroke weights.

I had played around with animating the flow of cash monthly so you could see it funneling from the surplus countries to deficit countries, but I started to hit too many performance issues with the animation, and I thought this version was a good stopping point.

On the technical side, this is pretty much pure Degrafa GeometryRepeaters and CubicBezier curves.   One of the most elegant things about this visualization is that the core code that maps the visual structures is less than 60 lines of Degrafa MXML.

At some point in the future (perhaps 360|Flex in Indianapolis) I will open up the source code and walk through how it is done.   Suffice to say that I am 100% convinced that using markup to create your composite visual structures and procedural code to tweak them is a much more effective and intuitive way to code than using a pure procedural/OO approach.

After spending a fair amount of time working with Processing, which is pure OO/Procedural coding, getting back to Degrafa was refreshing change.  Things like the GeometryRepeater class are extremely powerful for this type of work – especially when you bind it to dynamic data sets.   Being able to use markup to make contextual relationships in XML to your visual forms is extremely helpful for keeping all of your abstractions front and center mentally.  With complex visualizations and lots of composite pieces the human brain (especially mine) is limited as to how many abstractions it can keep in working memory at any one given time.   By being able to describe your structures declaratively in a concise amount of space, you can then manipulate specific properties of that structure while still being able to reference the whole.   This allows me to “sketch” in my major graphic forms and then later add details, animation, and styles to each piece of the form.

There is a school of thought in the world of data visualization that I like to refer to as the “purist” approach. This outlook and view has been adopted by a few prominent individuals who specialize in data visualization, most of who are simply re-hashing or misinterpreting the work of Edward Tufte. The reason why I felt the need to comment on this phenomena, is that these practitioners speak with such an air of authority and arrogance that a neophyte reading their material would come away with a very skewed perspective on how to apply data visualization best practices in a pragmatic manner, specifically when it comes to designing effective business performance dashboards..

To help understand how to recognize this school of thought I have listed some of the more salient characteristics that can be seen over and over again when reading their materials.

1. The purist spends 80% of their time pointing out what is wrong in others work through diligent critiques while only showing brief examples of what they propose to be more optimized solutions.
2. The purist has a strong disdain for any aesthetic “decoration” or consideration within a dashboard design.
3. The purist favors “efficient’ but more obscure visualization metaphors that leave the typical information consumer confused.
4. The purist usually ignores the interactivity of the medium and usually reserve their critiques to static 2d analysis, as if the computer monitor where a piece of paper versus a device that the user can interact with (via the keyboard, mouse, etc..)
5. The purist is usually not a technologist, and as such offers advice that is more suited to float in the rarified air of academia than being grounded in the pragmatism of one who actually uses technology on a daily basis to accomplish business objectives.

Let’s take a closer look at some of these characteristics and how they truly undermine the purported goal of improving the practice of data visualization

Critique before Praise: I will give them this: the purist, if nothing else, is passionate. Unfortunately this passion seems to take the form of an almost evangelistic criticism of most software vendors and their dashboard visualization tools. In my experience when writing articles that are aimed at teaching best practices in data visualization, one of the easiest things to do is to point out bad examples of other peoples work. Unfortunately this approach does little to help readers quickly master the basics of good design; instead they are left with a catalogue of what not to do and no information on what should be done. The irony of this approach is that contrary to the visual “efficiency” the purist is trying to advocate, their literary approach is extremely inefficient as the big take away the reader has is all of the things that should NOT be done.

Aesthetics have no place in Data Visualization: While it is true that “sex sells”, and most dashboard vendors spend a lot of time drawing attention to their hyper realistic gauges and dials, it is not true that these “decorations” are ineffective. When properly used, aesthetics can play a huge rule in the effectiveness of a dashboard. Studies have shown that aesthetics play an important role in the perception, and ease of use. Dashboards have a unique set of requirements from a business and data visualization perspective, some of these requirements are quite a bit different from the requirements of doing data exploration or data analysis across large volumes of data. If a dashboard is being built to help monitor certain measures or operations that support a key business strategy or objective, I would think doing everything in your power to encourage end users to use and look at these measures would be a positive thing. The aesthetic appeal of the dashboard and the resulting emotional reaction to the visualization plays a large role in how easy users find the tool to use and how receptive they are in wanting to look deeper at what is being displayed. Using interactive and eye-catching features like realistic gauges, specular highlighting, and other visual cues helps to engage the end user and can also be used to emphasize the more important information within a dashboard. In some of my presentations and writing I talk about creating a visual hierarchy to your dashboard designs. This hierarchy is designed to reinforce the level of importance of each KPI or measure so that it reflects the importance of the business decisions that are being made from the particular metrics. Using these more engaging visualizations and contrasting with more subtle and flatter visualizations can be an ideal way to reinforce this visual hierarchy.

Efficient but Obscure: One thing I am struck with immediately when looking at some of the more esoteric visualization approaches advocated by some of the purists is that the meaning of the data being showed is not immediately obvious. While many of these visualizations are extremely ingenious and very efficient in terms how much information that can be imparted in a small amount of space, the problem is that the user ends up struggling to understand what is being shown. The user ends up spending a significant amount of attention reading through explanations, digesting the visual relationships, and learning the new graphical metaphor. Once the visual representation is understood and assimilated through repeated use we can began to let our natural pattern recognition processes within our brains help us to quickly digest the data. But this misses the point, that the goal is not to teach end users some new and novel way to visualize information, but rely upon the current mechanisms they are used to in terms of visualizing data that allow for that “instant recognition” when seeing data. When a user is forced to invest intellectual thought into digesting the meaning of the spatial and visual representation of the data – they are no longer leveraging their powerful and native pattern recognition capabilities. Not that these novel approaches to visualization don’t have their place in data analysis, they just don’t belong on a business dashboard. The trick to doing great dashboard visualization work is to capitalize upon the visualizations and metaphors that users are accustomed to (and thus recognize naturally) to impart important business information that allows the user to make more efficient and timely decisions.

Escaping Flatland:

“We envision information in order to reason about, communicate, document, and preserve that knowledge — activities nearly always carried out on two-dimensional paper and computer screen. Escaping this flatland and enriching the density of data displays are the essential tasks of information design.”
– Edward R. Tufte, Envisioning Information.

Tufte serves as the canonical figure most often quoted or referenced by the purists, and the sad irony is that the purists seem to miss some of his most fundamental points. This is no more obvious than in the critiques that show a screen shot of a particular vendor’s dashboard tool that is then dissected as if it were a static object. Very little (if any) analysis is given to the more dynamic and interactive features of these dashboards. When an designer capitalizes upon the more dynamic nature of the vendors toolset and is able to effectively leverage the technology to selectively drill, filter, magnify, and highlight relevant data, these screens come to life and provide a very rich and dynamic interface for the user to immerse themselves more cognitively with the information at hand.

The Ivory Tower: In the real world we have users that don’t have the time to understand new and novel ways of visualizing data, customers who are impressed by the “sex and sizzle” and make purchase decisions accordingly, software tools that cant do everything perfectly, data that doesn’t meet clean and pre-defined notions. Real world solution providers have experience creating businesses and successful solutions while dealing with all of these variables and more. The purists on the other hand climb up into their Ivory Towers (usually a self-promoting column they have secured in one of the plethora of industry trade publications) and pontificate upon the finer points of data visualization expecting business users to change the way they think to meet their strict views on how to best present information. If the purist truly wants to change the way business people perceive data, they would develop a product that does that, market it, sell it, and implement it. Or at the very minimum, provide pragmatic and real world examples of how to do that with common software found in today’s market place. But then again that might actually force the purist to climb down from their tower and get their hands dirty in the work of solving real world problems with real world technology.

One of the phenomenal aspects of the human brain is its ability to behave like a massively parallel set of processors, being able to simultaneously digest multiple streams of information at speeds that would stymie some of the most advanced super computers.

While traditional x86 computer processors can only process one instruction set at a time, albeit at amazing speeds, they can not process information in parallel. While a human might be able to make 1 or 2 mathematical calculations per second, and a computer is capable of billions, a human can still instantly recognize a smile, while a computer can not. All of the computers power comes from its ability to do calculations in serial, one after another, at amazing speeds; at its core all a computer is doing is performing simple Boolean equations between bits that either have a 1 or 0 value. It is just doing billions of them in fractions of a second.

But, when you apply the computers incredible serial processing capabilities to very complex mathematical problems (like using multivariate analysis to discern the meaning of a facial expression) you hit a brick wall very quickly. Using a brute force linear approach to solve these problems will bring some of the most powerful computers to its knees. So if you ask a computer to search gigabytes of text data for the word “flower” it will do an amazingly quick job of finding it (relatively easy mathematical calculations using powerful sort algorithms), but if you scan a picture of your front yard in the spring time and ask the computer to identify a blooming flower, it will have a much more difficult time.

Humans have an amazing ability to process certain types of information in a more instantaneous way, this is most evident in the way we process visual information. A very real example is the way we can analyze numbers represented as visual patterns exponentially quicker than we can analyze a set of numbers in their native form. If you were to look at a spreadsheet filled with hundreds of weekly sales numbers across product lines, versus looking at a simple bar chart with the same data, which would you prefer to determine which products sold more than others?

Lets look at this example more closely. When we look at the bar chart we can “instantly” see each bar in relationship to its neighbor and “instantly” see which ones are higher than others. This analysis and understanding happen in an instant, it isn’t even a conscious thought. It is the same as you identifying what color something is, you simply know it. Conversely, if we were looking at the numbers in a spreadsheet we need to do many more mental gymnastics. First we need to read the number which is a visual pattern in itself, and a combination of visual symbols usually consisting of one or more digits like: “123,456.” We then have to consciously make a mental comparison of each number against the others. This is a conscious and thus linear process, in which we are much slower than a computer.

So if you look at a visual representation of data you are able to process information almost instantly, but if you need to look at a raw data itself, you will be forced to consciously digest it in a linear fashion.

One of the fundamental areas we focus on in our information visualization work at BrightPoint Consulting, is to figure out ways to leverage humans naturally given pattern processing engines. To the degree we can present relevant information in visual patterns that users can take advantage of, we succeed in creating a more effective and efficient interface between the user and the data they interact with.

In the face of the exponential growth of data and information that 21st century humans need to deal with on a daily basis, technologies that allow people to more effectively process that information is not only a great opportunity for new business, but it is something that helps society as a whole. We are relieved of the tedium of repetitive information processing tasks and freed up to focus on higher level functional tasks that result from the understanding of the underlying patterns.