Glassbox

Being a Business Intelligence type, if I were given the job of devising a tool for analyzing Java applications, I would build a component to collect performance data and then present the user with a bunch of fancy reports, charts and data cubes where they could drill down to work out what their problem is. Glassbox takes a different approach as we heard at the SDForum Java SIG last Tuesday.

Glassbox collects the data from Java Apps running in an application server, analyses it for a set of common problems and produces a report that tells you exactly what the problem is. No complicated analysis, no slicing and dicing, just a simple report on the facts. Of course it can only tell you about problems that it has been programmed to analyze, however the common problems are well known. Things like too many database queries to answer web request, slow database response, slow complicated java that seems to call the same function too many times. It is the kind of solution that gets 90% of the problems with 10% of the effort of a full blown performance analysis tool. Another advantage of this approach is that the tool can be used by anyone without a lot of training or time spend getting experienced in its use.

While Glassbox has not wasted time building fancy displays, they have taken the trouble to collect their data in a straightforward and unobtrusive way. As we were shown, you just add a .war file to your application servers directory, update the application servers configuration, restart it and you are on your way. Supposedly data collection only adds 1% or so to program execution times.

All in all, Glassbox looks like a good place to start identifying problems with web apps. As it is Open Source, and easy to use, the cost of trying it out is low.

Definitive Source

When George Smoot was called at 3 am by someone with a Swedish accent telling him that he had won a Noble Prize for Physics, he thought this might be a prank. So what did he do? He went to the Nobel Prize web site to check whether it was true. We have come to the point where a Nobel laureate trusts the World Wide Web as the definitive source of information rather than a phone call.

Dashboard Design

We had a lot of fun at the SDForum Business Intelligence SIG September meeting where Stephen Few spoke on "Why Most Dashboards Don't Work". Here we are talking about Information Dashboards that let an executive pilot their enterprise to new levels of performance. Stephen is an expert on the visual presentation of information, he has just published a book on Dashboard design and he has spoken previously to the BI SIG.

The fun came from looking at examples of dashboards that had been culled from the web and picking holes in what they presented. In practice it was surprisingly easy for audience members to find problems in the dashboards shown. From these examples and some critical thinking, Stephen pulled out a list of 13 things to avoid in dashboard design and a shorter list of things to do to get a dashboard design right.

However the thing that I found most compelling about the presentation came right at the end. Stephen had judged a dashboard design competition for DMReview and he showed us some of the entries. Then he showed us a dashboard that he would have entered had he not been a judge. Of all the dashboards presented, this was the one that showed us a great deal of information in a small space and discretely guided us to the information that most required our attention.

If you want judge, you can download a version of the presentation from the Business Intelligence SIG's Yahoo web site. We had to cut the size of the file down to make it fit. You can also buy Stephen's book on dashboard design. I highly recommend it.

The Latest HP Mess

There is a lot of talk in the valley about the latest HP boardroom brouha. It seems like not a year goes by without some new HP management upset. These upsets seem all the worse for the high regard in which the company was held. If you are upset by what seems to have become of such a great company, let me set the record straight.

Firstly, remember that the great company founded by Bill Hewlett and Dave Packard is now called Agilent. While Agilent seems to have lost some of the Hewlett-Packard way, it is not as bad as what has happened to HP, the fat child spun out of the original company several years ago. HP, the computer company, started out life as couple of divisions out of 20 that lost their way.

Part of the Hewlett-Packard way is that divisions grow organically and then split when they reach a certain size. This way no division dominates, they operate as a set of peers. The Computer and Printer divisions eschewed this tradition by just growing until they were big enough to swallow the rest of the company. Worse, the Computer division gave up on organic growth and for much of the last 20 years has been growing by acquisition.

All these acquisitions, particularly the large ones have diluted the blood to the point where we can on longer see a trace of the founding principals (pun intended). So do not feel sorry for HP, it is not the company you thought it was, it is just another big dinosaur well on its way to extinction.

Free as in Peer

Laurence Lessig's writes an interesting column Wired magazine. His latest entry is titled "Free as in Beer". The column starts off talking about free, or more accurately Open Source beer. It is a good read however towards the end there is the following discontinuous comment:

Although peer production is profitable for business, writes Benkler, "we are in the midst of a quite basic transformation in how we perceive the world around us and how we act, alone and in concert with others." What he calls nonmarket peer production is a critical part of this transformation. (sic)

Beer Peer? We all know that to appreciate beer you need to open the source, and that after appreciating beer you become a pee'r, but it is not the knd of thing that needs to be talked about.

The BIRT Strategy

Software is fascinating stuff. Compared to any other engineered product it is completely ephemeral, yet at the same time it is becoming the thing that makes almost every engineered product work. Software also has a meme-like quality where certain software systems become the standard that everyone gravitates to use, however good or bad it eventually turn out to be. It seems that the trick to creating successful software is to make it really, really successful.

I got to thinking about this after listening to Paul Clenahan, VP of Product Management at Actuate Corporation and member of the Eclipse BIRT Project Management Committee talk on "Eclipse BIRT: The Open Source Reporting Framework" at the SDForum Business Intelligence SIG. BIRT is a component of the Open Source Eclipse project that provides a Business Intelligence Reporting Tool (hence BIRT).

BIRT consists of an Eclipse plug in that allows you to design sophisticated reports, a standards based XML definition of the report and delivery mechanisms that allow you to deliver reports as either HTML or PDF documents. As Paul mentioned several times, it is also very extensible, so if it does not have the capabilities that you need, you can easily add them. BIRT is Open Source software that is available under the relatively unencumbered Eclipse public license that allows commercial exploitation of the code.

From the presentation and demo, BIRT seems to be a well designed, easy to use and fully capable reporting system that is free. In fact, as the presentation wore on, the one question in my mind was why Actuate has devoted 8 developers to developing this wonderful new Open Source reporting system. What is in it for Actuate? I think that it has to do with broadening the marketplace.

While reporting tools are widely used, many more developers roll their own reports rather than use a reporting tool. Paul mentioned in his presentation that he asked a large group of developers at a conference whether they used reporting tools and the vast majority did not. Providing an easy to use Open Source tool that fits into the popular Eclipse development environment brings developers into the reporting tool fold.

Reporting tools are not rocket science. Low cost reporting tools have been around for a long time. While Actuate has excellent reporting tools, their core differentiating competence is a scalable platform for delivering reports, something that other reporting tools do not have. So broadening the market for reporting tools also widens the market for report delivery tools. If they are successful and BIRT catches on in that meme-like way, Actuate will have a much larger market to sell their products into. Open Source and an Eclipse plug-in dramatically lowers the barriers to using these tools.

Blogger Upgrade

Blogger, home of this blog is going to get an upgrade. I have used Blogger for the last couple of years as it suits my text mostly blogging style. So far my only complaint has been about Google's segregated indexing (and a spell checker that seems to work against both Blogger and Google). Many others have been less patient.

The bad news is that the new Blogger will be integrated with Google Accounts. Recently, I wrote about how I had been forced to give up my identity to Yahoo. Now, it looks like Larry and Sergey are going to get a bit of my identity as well. Barry Diller has a piece of me and Rupert has all my kids nailed down in their own little spaces. Whatever happened to freedom?

A Short Post

Everyone including the media is talking about the idea of the long tail. To me it seems like last years idea. On the other hand, this is the silly season so maybe that is all they have to write about.

Moore's Law Logic

We all know Moore's law, but very few seem to understand the inevitable logic that it implies. Moore's law states that the number of transistors on a silicon chip doubles every 12 to 18 months. In practice chips are all pretty much the same cost (within an order of magnitude or so), thus we get double the capability every 18 months. The long term consequence of this is that everything becomes digital and every digital device is eventually a single chip.

I will not waste your time you with a comprehensive history. It is sufficient to highlight a couple of trends. The first trend is to digital media, starting with music in the CD and then MP3, then video and books, and now we are on the verge of digital broadcast TV and radio. A second trend is towards the single chip implementation of all electronic devices, starting with watches and calculators in the 70's then consumer electronics like CD players, stereos, radios and TV's. Currently we have just achieved the single chip cell-phone. The trend to digital media helps with the movement to single chip implementations because it is much easier to do an all digital device than one that has both analog and digital circuits.

One device that has so far avoided becoming a single chip implementation is the personal computer. A typical motherboard has about 6 to 8 processing chips, a bunch of memory chips and some driver chips that do nothing more than pass on a strengthened signal from a processing chip. One day all these chips except for the driver chips will coalesce into a single chip, because there will be nothing else better to do with all the available transistors.

So, last week when AMD announced that they were buying ATI, I knew what it was about. AMD has the single chip personal computer on their long term road map, and they need the display drivers and other peripherals that ATI has to complete their vision. AMD has already moved the memory controller onto the processor chip. Next I expect them to announce a low end chip with all the rest of the peripherals integrated. Over time the single chip processor implementation will move up to the mid range and high end. Sometime thereafter, the single chip computer with integrated memory will become first possible and then inevitable.

The Yahoo Business Model

In the back of my mind I had always understood the Yahoo! Business Model. Yahoo! gets you to sign up for the compelling online services that they provide and in return they sell advertising targeted at you. This was confirmed at the July meeting of the SDForum Business Intelligence SIG where Madhu Vudali, Director of Pricing & Yield Management at Yahoo! Spoke on "Pricing & Yield Management at Yahoo!"

For years, I instinctively resisted signing up for any online services and deliberately avoided using any services like driving directions that would reveal part of my identity, while still making liberal use of anything that did not reveal anything about me except for perhaps a few stocks that I was anonymously interested in. Recently I have been required to sign up for a couple of Yahoo! services so they have my identity, as they do for the half billion other people who are also signed up. As I said before their services are compelling.

Madhu explained the other side of the coin where Yahoo! sells advertising. This is not the auctioned search advertising that Google has become known for, although Yahoo! also does this. This advertising is the banner ads that you see when you use all those compelling services. The ads can be very specifically targeted such as a movie ad that is shown for a few hours on a Thursday and Friday afternoon to a specific demographic.

As Madhu explained, Yahoo! uses the yield management techniques that were pioneered by the airline industry in the 70's. The airlines use yield management to get the most revenue out of every available airplane seat while keeping the airplane full. Yahoo! has a similar problem but on a much larger scale. They have a huge inventory of page views and several dimensions such as age, location and interests on which to segment the viewers and interest the advertisers.

The problem is to extract the maximum revenue out of this mix. Compared to aircraft yield management the inventory is much larger and more squishy and the number of potential products numbers in the multi-millions rather than in the tens of thousands that an airline has.

All in all, it was a very interesting presentation. Unfortunately for anyone who was not there, Madhu's presentation is not available, so this is the best that you are going to do. Sign up for the Business Intelligence SIG mailing list (sdforum_bisig-subscribe@yahoogroups.com) and do not miss another meeting.