So little time and so many things to write about. My February copy of Wired arrived before I had time to remark on the article in the January edition by Davis Byrne of Talking Heads on "The Fall and Rise of Music". In the article, Byrne describes 6 different business models for musicians in the new world on digital music. They range from the sell your soul to the record company at one end of the spectrum through to eschew any other organization so that you can create sell and distribute your music yourself. Moreover, Byrne gave examples of musicians who are using each of the business models to show that they are valid models that are used.
From his experience of record deals with big record companies, Byrne advises any young artist to avoid selling your soul to the man, or even taking the the standard record company deal where they just own everything that you do. Surprisingly, there many who hold the opposite position. In December Tech Crunch published their list of the 20 most popular posts of 2007. One of them was by Michael Arrington on "The Inevitable March of Recorded Music Towards Free". It is interesting to read the comments in response, particularly the many comments from people who live to sell their soul to the big record companies and their anger that the big record companies are becoming not so big.
I have laid out my opinion on the future of music in several previous posts. It is great to see Byrne explain the alternatives in such a dispassionate way. It is also great to know that there are forward looking people who are building next generation music businesses, for example RCRD LBL.
Sunday, January 20, 2008
Sunday, January 13, 2008
Tao of Software Engineering
Scott Rosenberg's latest book "Dreaming in Code" contains a very readable history of Software Engineering. I will review the book in another post. Here, I want to talk about a couple of papers that are not mentioned in Scott Rosenberg's history that are parts of my Tao of Software Engineering.
The first is Melvin Conway's paper from 1968 called "How do Committees Invent". Fred Brooks references the paper in "The Mythical Man Month" and gave it the name "Conway's Law", however, only recently that my good friend Roger Shepherd pointed me to the original. The thesis of the paper is: "Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization's communication structure."
While Conway's paper gives examples from several different branches of engineering, it is more applicable to Software Engineering which is the most plastic of the engineering arts. After all, a building has to have a floor, walls and a roof; a semiconductor chip is laid out on a flat surface; most mechanical devices start with a rotary power source and after that it is a question of packaging. On the other hand, software can take on any structure so it is absolutely natural that it should take of the structure of the organization producing it, whatever unsightly skew the organization may place on the software's structure.
It is not only the structure of the organization that influences a software architecture, it is the way the project is put together. For example, I have recently been working on a software project that started as a GUI demonstration. Because the GUI came first, it defined the client-server API and the data structures that passed through the API which then defined the database storage structure. We recently had to make significant changes to the API and data structures which has been extremely painful at such a late stage in the project.
This brings me to the second paper, Butler Lampson's "Hints for Computer System Design" from 1983. Like Conway's paper, it is not a difficult read. The paper is a collection of simple aphorisms on computer system design and examples to back up those aphorisms. While the aphorisms are still relevant to this day, some of the examples are creaky. For example, who remembers PL/1 or the SDS 940?
I reread this paper every few years and compare with my recent experience to see what I have done right and what could be done better next time. For example, one aphorism is "Keep basic interfaces stable". Several years ago, my very first act in a project to add a significant new feature to a large system was to go through the entire code base and change some basic APIs for the new feature. In my most recent project we have gone through months of pain because we decided to change the API just as the project was coming together.
The first is Melvin Conway's paper from 1968 called "How do Committees Invent". Fred Brooks references the paper in "The Mythical Man Month" and gave it the name "Conway's Law", however, only recently that my good friend Roger Shepherd pointed me to the original. The thesis of the paper is: "Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization's communication structure."
While Conway's paper gives examples from several different branches of engineering, it is more applicable to Software Engineering which is the most plastic of the engineering arts. After all, a building has to have a floor, walls and a roof; a semiconductor chip is laid out on a flat surface; most mechanical devices start with a rotary power source and after that it is a question of packaging. On the other hand, software can take on any structure so it is absolutely natural that it should take of the structure of the organization producing it, whatever unsightly skew the organization may place on the software's structure.
It is not only the structure of the organization that influences a software architecture, it is the way the project is put together. For example, I have recently been working on a software project that started as a GUI demonstration. Because the GUI came first, it defined the client-server API and the data structures that passed through the API which then defined the database storage structure. We recently had to make significant changes to the API and data structures which has been extremely painful at such a late stage in the project.
This brings me to the second paper, Butler Lampson's "Hints for Computer System Design" from 1983. Like Conway's paper, it is not a difficult read. The paper is a collection of simple aphorisms on computer system design and examples to back up those aphorisms. While the aphorisms are still relevant to this day, some of the examples are creaky. For example, who remembers PL/1 or the SDS 940?
I reread this paper every few years and compare with my recent experience to see what I have done right and what could be done better next time. For example, one aphorism is "Keep basic interfaces stable". Several years ago, my very first act in a project to add a significant new feature to a large system was to go through the entire code base and change some basic APIs for the new feature. In my most recent project we have gone through months of pain because we decided to change the API just as the project was coming together.
Thursday, January 03, 2008
DRM Discordance
Read this article from someone who bought a new high definition monitor for their Windows Vista Media Center PC and found out that they could no longer use the Netflix Watch Now streaming video facility. In the past I have argued that if we have to put up with DRM, it is better system engineering to put the DRM decoding in a separate box rather than try to bundle it into a general purpose PC as Microsoft is trying to do with Windows Vista Media Center.
By coincidence the article appears on the same day that Netflix and LG announced their partnership to integrate Netflix Watch Now streaming video into a future LG device. The TechCrunch take on the Netflix set top box is that it will be a hard sell, but if the alternative is the flaky behavior that we see in Microsoft Vista, maybe the masses will buy the box.
By coincidence the article appears on the same day that Netflix and LG announced their partnership to integrate Netflix Watch Now streaming video into a future LG device. The TechCrunch take on the Netflix set top box is that it will be a hard sell, but if the alternative is the flaky behavior that we see in Microsoft Vista, maybe the masses will buy the box.
Tuesday, January 01, 2008
42
We know that 42 is the answer to the question, you know about life, the Universe and everything. Also, 42 is the number of posts that I have made in this blog in each of the last 3 years. I have always intended to write more posts, there are many subjects that I want to write about, but for one reason or another, they never get written or never get finished.
This year I did manage to write some more as I started the Developer Notebook blog to cover specific technical programming topics. The Developer Notebook also has less posts than intended. I have several posts in my head that are just waiting for me to find the time and energy to write them down. We will try to do better in 2008.
So as they say in my birthplace "Awrra best furra New Year".
This year I did manage to write some more as I started the Developer Notebook blog to cover specific technical programming topics. The Developer Notebook also has less posts than intended. I have several posts in my head that are just waiting for me to find the time and energy to write them down. We will try to do better in 2008.
So as they say in my birthplace "Awrra best furra New Year".
Tuesday, December 11, 2007
Viva Fake Steve
I have just finished reading "Option$, the secret life of steve jobs" by Fake Steve Jobs. The good news is that it is an extremely funny book. I laughed out loud every few pages. The last book that had me laughing out loud in the same way was "Thank You for Smoking" by Christopher Buckley and that was well over a decade ago. Not quite so good is that while the story arc is firmly in place, the ending is a bit weak. However, a weak ending does not diminish the joy leading up to it. Well recommended.
Wednesday, December 05, 2007
Aggregate Data, Make Money
In the old days, the aggregate business was about selling large numbers of small stones for building roads and such, now the aggregate business is about collecting large numbers of data points and aggregating them into information valuable enough to support a business. We heard about one such business at the November/December Meeting of the SDForum Business Intelligence SIG where Faisal Mushtaq, CTO, Biz 360 spoke on "Do you know what customers are saying about you and your products?".
Biz360 starts as a Web version of a clippings service. They gather media articles, blog entries and other documents that reference their clients from the web and other information services. As the information is in digital format, there are opportunities for interesting analyzes that cannot be done with dead tree clippings. For example, Biz360 automatically ascribes a sentiment, negative or various grades of positive to each document so that the business can track what people think about it. They also ascribe a reach to each document that measures its importance. This captures the fact that an article in the Wall Street Journal will be seen by and influence more people than an article in say the San Jose Mercury News.
Once the data is gathered, a client can look at it in many different ways. The starting point is usually dashboard that shows reach and sentiment over a period of time. The client can look at changes in reach and sentiment and drill down to find out what lies behind them. Newspaper articles cite sources and blogs are linked so the software can derive and show the network of influence between related items.
Faisal showed us a specific example of how attention to sentiment could have helped avert a PR problem. In 2006, Dell had a problem with exploding batteries in their laptop computers. Although the story did not break in the newspapers until August, there had been a large discussion of the problem in the blogosphere for months beforehand. Faisal showed us a convincing graph showing that by the time the mainstream news media picked up on the problem, it was already old hat in the blogosphere.
Doing this kind of analysis requires some serious computing power. Biz360 has a 150 CPU processor farm with 25 Terabytes of disk storage arranged as a computing GRID. They download about 1 million media articles a day and 4 million blog posts. From this they derive roughly 200 thousand articles of interest to their clients and apply 7 million automated analyses to these articles.
Biz360 starts as a Web version of a clippings service. They gather media articles, blog entries and other documents that reference their clients from the web and other information services. As the information is in digital format, there are opportunities for interesting analyzes that cannot be done with dead tree clippings. For example, Biz360 automatically ascribes a sentiment, negative or various grades of positive to each document so that the business can track what people think about it. They also ascribe a reach to each document that measures its importance. This captures the fact that an article in the Wall Street Journal will be seen by and influence more people than an article in say the San Jose Mercury News.
Once the data is gathered, a client can look at it in many different ways. The starting point is usually dashboard that shows reach and sentiment over a period of time. The client can look at changes in reach and sentiment and drill down to find out what lies behind them. Newspaper articles cite sources and blogs are linked so the software can derive and show the network of influence between related items.
Faisal showed us a specific example of how attention to sentiment could have helped avert a PR problem. In 2006, Dell had a problem with exploding batteries in their laptop computers. Although the story did not break in the newspapers until August, there had been a large discussion of the problem in the blogosphere for months beforehand. Faisal showed us a convincing graph showing that by the time the mainstream news media picked up on the problem, it was already old hat in the blogosphere.
Doing this kind of analysis requires some serious computing power. Biz360 has a 150 CPU processor farm with 25 Terabytes of disk storage arranged as a computing GRID. They download about 1 million media articles a day and 4 million blog posts. From this they derive roughly 200 thousand articles of interest to their clients and apply 7 million automated analyses to these articles.
Friday, November 23, 2007
OLAP Vendors Capitulate
I did some research on where the Business Intelligence market was heading and came to the conclusion that there has been complete capitulation in the marketplace. The table to the right shows the market share of the top 10 OLAP vendors in 2006 (courtesy of The OLAP Report).
In 2007 the following events took place. Oracle bought Hyperion. Cognos bought Applix. Business Objects bought Cartesis. SAP announced that it is buying Business Objects. IBM announced that it is buying Cognos.
In essence, the top 10 OLAP vendors which included several important Independent Software Vendors (ISV) is being replaced by the top 4 enterprise software vendors: IBM, Microsoft, Oracle and SAP.
There are two independent vendors left in the list and neither is going to break out. Microstrategy is in the ROLAP niche, suitable for big gnarly problems that need a lot of money. Microstrategy could make a good fit with the newly independent Teradata another company in a similar niche. Infor is a private company that has been rolling up second tier software vendors in the same way that Computer Associates did in the 80's and 90's. It is the new "home for old software".
Wednesday, November 07, 2007
Fake Steve Jobs, Tech Journalist
Someone in the audience put it best when he said that he uses "The Secret Diary of Steve Jobs" as his primary source of tech news. This was at the Computer History Museum last Tuesday evening when Dan Lyons AKA Fake Steve Jobs (FSJ) appeared on stage with former Apple Evangelist Guy Kawasaki, and Brad Stone the journalist who outed him. In the meeting, Dan Lyons seemed to be a little confused that anyone would think that his column is a useful commentary on tech events, but I know what that audience member means.
I too use FSJ as a primary source of my tech news and commentary. While the blog is intended as a satire, Dan Lyons is a knowledgeable tech writer and he gets the details right. Moreover, the format means that he can be perfectly frank and even rudely negative as he often is. I get more information out of 200 word rant from FSJ on a subject such as Facebook valuation that I get out of a perfectly balanced 1500 page article in the NYT or WSJ. What is more, a spoonful of humor sugar helps the medicine go down.
There is the question of bias. Truth be told, I find it much easier to deal with information when the bias is clear and on the surface rather than cleverly buried as it is in the 'balanced' articles written by Journalists. Moreover, Journalists need to make nice with their subjects so that they will continue to get access in the future, so they tend to backpedal on the negative, or at least bury it in the n'teenth paragraph. On the other hand, FSJ in character says what he thinks, and the truth comes straight out.
The event itself was well worth attending. Dan is a very funny guy and he had us laughing for nearly two hours, mocking everyone from the Appletards in the front row to Megan McCarthy of Vallywag at the back. There were at least 200 in the audience, including several notables such as well known tech writer John Markoff and iJustine.
I too use FSJ as a primary source of my tech news and commentary. While the blog is intended as a satire, Dan Lyons is a knowledgeable tech writer and he gets the details right. Moreover, the format means that he can be perfectly frank and even rudely negative as he often is. I get more information out of 200 word rant from FSJ on a subject such as Facebook valuation that I get out of a perfectly balanced 1500 page article in the NYT or WSJ. What is more, a spoonful of humor sugar helps the medicine go down.
There is the question of bias. Truth be told, I find it much easier to deal with information when the bias is clear and on the surface rather than cleverly buried as it is in the 'balanced' articles written by Journalists. Moreover, Journalists need to make nice with their subjects so that they will continue to get access in the future, so they tend to backpedal on the negative, or at least bury it in the n'teenth paragraph. On the other hand, FSJ in character says what he thinks, and the truth comes straight out.
The event itself was well worth attending. Dan is a very funny guy and he had us laughing for nearly two hours, mocking everyone from the Appletards in the front row to Megan McCarthy of Vallywag at the back. There were at least 200 in the audience, including several notables such as well known tech writer John Markoff and iJustine.
Monday, November 05, 2007
A Developer Notebook
For some time I have wanted to write technical posts on specific programming and programming languages topics. While I could post them in this blog, they would not really fit with the lighter hearted and wider ranging posts here. So I started a new blog called "A Developer Notebook" for the technical posts. When you look at the content, there should be no mistaking the goals and purposes of these two blogs. Enjoy!
Thursday, October 25, 2007
The Facebook Platform Internals
Ari Steinberg and Charlie Cheever of Facebook gave a fascinating talk about the design of the Facebook platform when they spoke on Tuesday to the SDForum Web Services SIG. As might be expected, the presentation drew a large audience including renowned Facebook fanboy Dave Maclure.
The Facebook platform allows outside developers to build applications and offer them to Facebook users. These applications can reach into Facebook data to do their work so security and privacy concerns are paramount. Also with Facebook approaching 50 million users and reportedly 85,000 applications, performance and scaling are big concerns.
Ari spoke first on the Facebook API. After looking at other Web 2.0 APIs and finding them wanting, they decided to use a query language approach which would give applications much greater flexibility and at the same time have the possibility of being more efficient by only fetching data that is needed. The language, FBQL is basically a simplified and restricted version of SQL.
The linguistic approach is also used in Facebook Markup Language (FBML) the language that applications and users use to define and customize their pages. As Charlie told us, it is much easier to validate and sanitize markup and style sheets from a parse tree than with regular expressions or any other technique. Also the result is much less likely to have the security loopholes that seem to plague so many Web 2.0 sites.
The Facebook platform allows outside developers to build applications and offer them to Facebook users. These applications can reach into Facebook data to do their work so security and privacy concerns are paramount. Also with Facebook approaching 50 million users and reportedly 85,000 applications, performance and scaling are big concerns.
Ari spoke first on the Facebook API. After looking at other Web 2.0 APIs and finding them wanting, they decided to use a query language approach which would give applications much greater flexibility and at the same time have the possibility of being more efficient by only fetching data that is needed. The language, FBQL is basically a simplified and restricted version of SQL.
The linguistic approach is also used in Facebook Markup Language (FBML) the language that applications and users use to define and customize their pages. As Charlie told us, it is much easier to validate and sanitize markup and style sheets from a parse tree than with regular expressions or any other technique. Also the result is much less likely to have the security loopholes that seem to plague so many Web 2.0 sites.
Subscribe to:
Posts (Atom)
