Managing time and synchronization in any software is complicated. Leon Starr, a leading proponent of building executable models in UML, talked about the issues of modeling time and synchronization to the December meeting of the SDForum SAM SIG. Leon has spoken to the SAM SIG previously on executable models. This time he brought along two partners to demonstrate how the the modeling technique can be applied to a broad range of problems.
Leon started the meeting by talking through five rules for handling time and synchronization. The first and most important rule is that there is no global clock. This models real systems which may consist of many independent entities and allows for the most flexible implementation of the model on a distributed system. In practice, other rules are a consequence of this first rule.
The next rule is that that the duration of a step is unknown. The rule does not imply that any step can take forever, its purpose is to say that you cannot make assumptions about how long a step may take. In particular, you cannot expect independent steps in the model to somehow interleave themselves in some magical way. The third rule is that busy objects are never interrupted. This forces the modeller to create a responsive system by building it from many small steps so that an object is always available to handle whatever conditions that it needs to handle.
The fourth rule is that signals are never lost. This is an interesting rule as it gets to an issue at the heart of building asynchronous systems. The rule implies that there is a handshake between sender and receiver. If the receiver is not ready, the sender may be held up waiting to deliver the signal. Perhaps the signal can be queued, but then there is the problem that the queue is not big enough to handle all the queued signals. In the end you have to build a system that can naturally handle all the events thrown at it, if it is a safety critical system, or that fails gracefully if it is not.
The fifth rule is that there is no implicit order in the system, except that if one object sends signals to another object, the signals arrive in the order that they were sent. Note that I may have interpolated some of my own experience into this discussion of the rules. If you want to explore further watch this video on You-Tube and go to Leon's web site which leads to many interesting papers and discussions.
Next at the meeting, Leland Starr, younger brother of Leon, talked about a web application that he had been the lead on for his employer, TD Ameritrade. The online application is for arranging participants in online webinars. By using the UML modelling technique, he created a model that could be both used to explain how the system would worked to the business sponsors of the project and that could be executed to check that it worked as expected. Leland has a SourceForge project for his work.
Finally Andrew Mangogna talked about a very different class of applications. He builds software to control implanted medical devices like heart pacemakers. The two overriding concerns are that the medical device performs its function safely and that it runs for at least 5 years on a single battery charge. Compared to many of the applications that we hear about at the SAM SIG the implantable device applications feel like a throwback to an earlier and simpler age of computing. The applications are written in the C programming language and the code typically occupy 3 to 4 kilobytes. The program data is statically allocated and an application can use from 150 bytes to 500 bytes. Andrew also has a project on SourceForge for his work.
Thursday, December 17, 2009
Friday, December 04, 2009
Bandwidth Hogging
There are several discussions going on around the web about bandwidth hogging started by a post from Benoit Felten in the fiberevolution blog. I wrote about this issue last month in my post on net neutrality. The basic problem is that when the internet becomes congested the person who has created the most connections wins. Congestion can happen anywhere from your local head end through to a backbone and the backbone interconnects. Felten claims that there is no problem, and given the data, he is willing to do the data crunching to prove it, while others disagree.
The problem is a classic Tragedy of the Commons. There is a shared resource, the internet, and some people use more of it than others. That is fine provided that they do not interfere with each other and there is enough resource to go around. As I explained, the problem is that when there are not enough resources to go around, the people who win are the people who create a large number of connections, and these tend to be the people who use the most bandwidth. The point of a torrent client creating a large number of connections is to ensure that that the client gets its "share" of the net whether there is congestion or not. The only viable response is for everyone else to create large numbers of connections to do whatever they want to do, be it download a web page or make a internet phone call. This is undesirable because it can only lead to more congestion and less efficient use of the shared resource.
There are two parts to a solution. Firstly, the internet service providers have to keep adding more equipment to reduce congestion as internet usage grows. Everything would be fine if there were no congestion. Secondly, we need better algorithms to manage congestion. Penalizing people for using the bandwidth they were sold is not the answer, particularly when that is not the real problem. I have suggested that we should look towards limiting connections. Another thought is to kill the connections of the users with the largest numbers of connections to reduce congestion. Again, I am sure that this will have some unintended consequences.
The real problem is that unless we can all agree to be good internet citizens and get along, the forces against Net Neutrality may win. Then large companies with deeply vested interests will get to decide who has priority. The recently announced merger of Comcast, a large Internet Service Provider and NBC, a large content provider is exactly the sort of thing that we need to be wary of.
The problem is a classic Tragedy of the Commons. There is a shared resource, the internet, and some people use more of it than others. That is fine provided that they do not interfere with each other and there is enough resource to go around. As I explained, the problem is that when there are not enough resources to go around, the people who win are the people who create a large number of connections, and these tend to be the people who use the most bandwidth. The point of a torrent client creating a large number of connections is to ensure that that the client gets its "share" of the net whether there is congestion or not. The only viable response is for everyone else to create large numbers of connections to do whatever they want to do, be it download a web page or make a internet phone call. This is undesirable because it can only lead to more congestion and less efficient use of the shared resource.
There are two parts to a solution. Firstly, the internet service providers have to keep adding more equipment to reduce congestion as internet usage grows. Everything would be fine if there were no congestion. Secondly, we need better algorithms to manage congestion. Penalizing people for using the bandwidth they were sold is not the answer, particularly when that is not the real problem. I have suggested that we should look towards limiting connections. Another thought is to kill the connections of the users with the largest numbers of connections to reduce congestion. Again, I am sure that this will have some unintended consequences.
The real problem is that unless we can all agree to be good internet citizens and get along, the forces against Net Neutrality may win. Then large companies with deeply vested interests will get to decide who has priority. The recently announced merger of Comcast, a large Internet Service Provider and NBC, a large content provider is exactly the sort of thing that we need to be wary of.
Monday, November 30, 2009
Consumerization of IT
A new generation is entering the workforce and they are just not going to take it any more. Brian Gentile, CEO of Jaspersoft, did not say these exact words, but it conveys the intent of the introduction to his talk on "Consumerization of IT" at the November meeting of the SDForum Business Intelligence SIG.
Brian was talking about Generation Y, the first generation to have grown up with computers and instant communication to the extent that they take them for granted. More that that they have expectations about these tools and what they can do with them. Unfortunately, enterprise software has often created systems that are slow, ugly and so difficult to use that it can requires weeks of training. While previous generations have put up with difficult software because they know no better, Gen Y does know that it can be better and is not going to put up with software that does not match up.
Brian identified 4 characteristics that Business Intelligence, or any enterprise software must provide to meet the next generations expectations. They are:
While I have heard people argue that current Business Intelligence software does not provide a good user experience, Brian put a positive light on this trend, as if the change is for the good and the right thing to do. He is certainly positioning JasperSoft to provide these features and meet the requirements of the next generation.
Brian ended with another optimistic note. The cost of Information Technology is coming down with cheaper hardware and Open Source software. CIO's can direct the money they save to new innovative projects. A good example of this movement is Ingres talking about "The New Economics of IT" as they have been doing for some time.
Brian was talking about Generation Y, the first generation to have grown up with computers and instant communication to the extent that they take them for granted. More that that they have expectations about these tools and what they can do with them. Unfortunately, enterprise software has often created systems that are slow, ugly and so difficult to use that it can requires weeks of training. While previous generations have put up with difficult software because they know no better, Gen Y does know that it can be better and is not going to put up with software that does not match up.
Brian identified 4 characteristics that Business Intelligence, or any enterprise software must provide to meet the next generations expectations. They are:
- Elegant presentation.
- Easy access to data.
- Extensive Customization.
- Built In Collaboration.
While I have heard people argue that current Business Intelligence software does not provide a good user experience, Brian put a positive light on this trend, as if the change is for the good and the right thing to do. He is certainly positioning JasperSoft to provide these features and meet the requirements of the next generation.
Brian ended with another optimistic note. The cost of Information Technology is coming down with cheaper hardware and Open Source software. CIO's can direct the money they save to new innovative projects. A good example of this movement is Ingres talking about "The New Economics of IT" as they have been doing for some time.
Sunday, November 15, 2009
Fight Instutional Corruption
Many people think of Lawrence Lessig as a radical with an anti-IPR (Intellectual Property Rights) agenda. In practice he is no radical, in fact his mission is to find a defensible middle ground between the Intellectual Property right and the Free Culture left. One of my first blog posts discussed his talk: "The Comedy of the Commons".
Recently, I have been following his podcasts which discuss his work on copyright as well as his newer work on institutional corruption. Note, while I find these podcasts interesting, they are not for everybody. They are mostly records of lectures given to various groups. While they are accessible, they are about serious policy matters and as many of the talks are on similar subjects, there tends to be some repetition.
Recently Lessig has been working on a new project on institutional corruption called Change Congress. The issue is that large sums of money fed through lobbyists seems to have an undue influence on the lawmakers in Congress and the Senate. The money appears to have such a large influence that lawmakers are voting against the clear wishes of their constituents.
Change Congress works to highlight these cases of apparent institutional corruption. It is fighting for citizen funded elections so that the lawmakers are not under pressure to raise the money they need to get re-elected. Thus they will be less likely to be swayed by the lobbyists. Go to the site, see what they have to say, and help them with their mission.
Recently, I have been following his podcasts which discuss his work on copyright as well as his newer work on institutional corruption. Note, while I find these podcasts interesting, they are not for everybody. They are mostly records of lectures given to various groups. While they are accessible, they are about serious policy matters and as many of the talks are on similar subjects, there tends to be some repetition.
Recently Lessig has been working on a new project on institutional corruption called Change Congress. The issue is that large sums of money fed through lobbyists seems to have an undue influence on the lawmakers in Congress and the Senate. The money appears to have such a large influence that lawmakers are voting against the clear wishes of their constituents.
Change Congress works to highlight these cases of apparent institutional corruption. It is fighting for citizen funded elections so that the lawmakers are not under pressure to raise the money they need to get re-elected. Thus they will be less likely to be swayed by the lobbyists. Go to the site, see what they have to say, and help them with their mission.
Saturday, November 07, 2009
Vote for Net Neutrality Now
There is a lot of talk about Net Neutrality now, and the issues are not completely clear cut as I will discuss later. However, there is also a big threat that needs to be addressed right now.
Bills are being proposed in Washington with friendly names like "The Internet Freedom Act" whose effect would be to give more control of the internet to the big ISPs and take away power from the people who are giving us innovative services like Google, Skype and Amazon. While there is also a friendly bill, and the FCC is on the side of Net Neutrality, everyone needs to act to let their congressman know whose side they are on. Visit "Save the Internet" and take action now!
Now that you have done your bit to save the internet, we can talk about the problem. When a node on the internet gets too much traffic, the traffic control algorithm will pick connections at random and kill them. While this is good for keeping the traffic flowing in the aggregate, it tends to favor one class of user over another. The disadvantage user is the one who is using a single connection to browse the web, download a song or make a voice call. The advantaged user is using Bit-torrent which opens a large number of connections to do a massive download. It does not matter if Bit-torrent loses a connection, it has many others to make up for it, but it does matter when a web browser, or Skype conversation loses a connection.
One solution is to answer greedy software with greedy software. That is every internet application would emulate Bit-torrent and greedily create hundreds of connections in case any one of them gets stomped. While this solution puts all applications on an equal footing, it may strain resources leading to a "Tragedy of the Commons", something that should not be in our bright digital future.
Another solution would be to limit the number of simultaneous sessions a user can have. I personally feel that this would be better than having Comcast or AT&T doing deep packet inspection of my packets. However a hard limit on the number of sessions may cause all sorts of problems with software that is not expecting it, leading to deadlock and other bad behavior. Does anyone have any other ideas?
Bills are being proposed in Washington with friendly names like "The Internet Freedom Act" whose effect would be to give more control of the internet to the big ISPs and take away power from the people who are giving us innovative services like Google, Skype and Amazon. While there is also a friendly bill, and the FCC is on the side of Net Neutrality, everyone needs to act to let their congressman know whose side they are on. Visit "Save the Internet" and take action now!
Now that you have done your bit to save the internet, we can talk about the problem. When a node on the internet gets too much traffic, the traffic control algorithm will pick connections at random and kill them. While this is good for keeping the traffic flowing in the aggregate, it tends to favor one class of user over another. The disadvantage user is the one who is using a single connection to browse the web, download a song or make a voice call. The advantaged user is using Bit-torrent which opens a large number of connections to do a massive download. It does not matter if Bit-torrent loses a connection, it has many others to make up for it, but it does matter when a web browser, or Skype conversation loses a connection.
One solution is to answer greedy software with greedy software. That is every internet application would emulate Bit-torrent and greedily create hundreds of connections in case any one of them gets stomped. While this solution puts all applications on an equal footing, it may strain resources leading to a "Tragedy of the Commons", something that should not be in our bright digital future.
Another solution would be to limit the number of simultaneous sessions a user can have. I personally feel that this would be better than having Comcast or AT&T doing deep packet inspection of my packets. However a hard limit on the number of sessions may cause all sorts of problems with software that is not expecting it, leading to deadlock and other bad behavior. Does anyone have any other ideas?
Friday, October 23, 2009
Database Systems for Analytics
The question "what are the attributes of a database system for analytics?" came up during Omer Trajman's talk to the October meeting of the SDForum Business Intelligence SIG. The talk was titled "The Evolution of BI from Back Office to Business Critical Analytics". In the talk Omer gave several examples of applications that use real time analytics and explained the special attributes of each application. As he runs field engineering for Vertica, a Database Systems vendor, I am sure that these examples were based on his experience with Vertica deployments, however Omer was careful to keep his talk vendor neutral.
So what are the the attributes of a database system for analytics? Omer discussed three attributes. Firstly, an analytics database system cannot use the row level locking that is found in a traditional transaction processing database. The database system needs to provide snapshot isolation that gives a query a consistent view of the data while not preventing other operations like data loads. Having helped implement a system like this in the past, I am in total agreement with Omer.
The second attribute is the need to allow concurrency between loading and querying data. While this is related to the first attribute, it also comes with its own issues. Bulk loads are more efficient (particularly for a columnar database like Vertica), however, if you want access to the most up to minute data you need to do loads in small increments so that the data is available for query as soon as it is loaded. Managing this balance is difficult and as yet it has not been completely solved. Again, I have worked on this issue in several different systems.
The final attribute was scaleout, that is the ability to add more processing systems to handle more data and larger queries. We are building systems out of hundreds and thousands of computer systems. Scaleout is vital to effectively use these systems.
So what are the the attributes of a database system for analytics? Omer discussed three attributes. Firstly, an analytics database system cannot use the row level locking that is found in a traditional transaction processing database. The database system needs to provide snapshot isolation that gives a query a consistent view of the data while not preventing other operations like data loads. Having helped implement a system like this in the past, I am in total agreement with Omer.
The second attribute is the need to allow concurrency between loading and querying data. While this is related to the first attribute, it also comes with its own issues. Bulk loads are more efficient (particularly for a columnar database like Vertica), however, if you want access to the most up to minute data you need to do loads in small increments so that the data is available for query as soon as it is loaded. Managing this balance is difficult and as yet it has not been completely solved. Again, I have worked on this issue in several different systems.
The final attribute was scaleout, that is the ability to add more processing systems to handle more data and larger queries. We are building systems out of hundreds and thousands of computer systems. Scaleout is vital to effectively use these systems.
Wednesday, October 14, 2009
e-Readers for All
The e-Reader market is heating up, just in time for Christmas. Amazon is expanding features and bringing the Kindle down the price curve. Today came word of the Barnes and Noble e-Reader with two screens, an e-ink screen for reading and a small LCD touch screen for interactivity.
Also today I caught up with the "This Week in Tech" podcast from last weekend where they talked about the real killer features of the Kindle - wireless download and almost unlimited capacity. You can buy as many books as you want any time you want, which leads to buying many more books than you would otherwise buy. Imagine the scene, at dinner with your friends, you discuss books that you have recently read, and bam you buy the books they recommend there and then. In fact there was even a cry in the podcast "Friends don't let friends use a Kindle while drunk" (for fear that the judgmentally impaired friend may buy too many books).
When the original Kindle came out there was a tremendous outcry against it with people complaining of gadgets destroying their book reading experience and authors expecting to have their livelihood destroyed just as the music industry has been laid waste. Hint, musicians are doing just as well as they have always done, it is the music moguls with their "by the way, which one is Pink?" who have been laid waste. The Kindle stimulates the publishing industry and makes it much easier to buy books, leading to more sales where author gets a larger slice of the pie.
Competition is good, particularly for the consumer. The e-Reader needs another generation or so to iron out the kinks and bring the price down to the mass market levels. I am waiting for the $149 price point (iPod Nano) which should come by next Christmas if not sooner.
Also today I caught up with the "This Week in Tech" podcast from last weekend where they talked about the real killer features of the Kindle - wireless download and almost unlimited capacity. You can buy as many books as you want any time you want, which leads to buying many more books than you would otherwise buy. Imagine the scene, at dinner with your friends, you discuss books that you have recently read, and bam you buy the books they recommend there and then. In fact there was even a cry in the podcast "Friends don't let friends use a Kindle while drunk" (for fear that the judgmentally impaired friend may buy too many books).
When the original Kindle came out there was a tremendous outcry against it with people complaining of gadgets destroying their book reading experience and authors expecting to have their livelihood destroyed just as the music industry has been laid waste. Hint, musicians are doing just as well as they have always done, it is the music moguls with their "by the way, which one is Pink?" who have been laid waste. The Kindle stimulates the publishing industry and makes it much easier to buy books, leading to more sales where author gets a larger slice of the pie.
Competition is good, particularly for the consumer. The e-Reader needs another generation or so to iron out the kinks and bring the price down to the mass market levels. I am waiting for the $149 price point (iPod Nano) which should come by next Christmas if not sooner.
Saturday, October 03, 2009
Search User Experience Innovations
Innovations in the Search User Experience was the topic at the September meeting of the SDForum Search SIG. The distinguished panel from Microsoft, Google and Yahoo was chaired by Safa Rashtchy, a long time analyst and commentator on the Search scene.
First, Sean Suchter General Manager of Microsoft's Search Technology Center Silicon Valley told us about the latest innovations in Bing. Sean started out with some numbers, showing that the Internet is still growing at a fast pace and that search is growing faster than the Internet in general. They measure their user's experience and see that about a quarter of searches are failures, resulting in an immediate click back. On the other hand, getting on for a half the search queries are further refined meaning that the user is engaged in a search session. Microsoft will recognize these sessions and use them to improve the user experience.
To simplify the user experience, when they are confident about what a user is searching for, Bing will show one subject on the first page with a number of related links. Sean showed us two examples. Firstly for the search term "target", where they assume the person is looking for the Target chain of stores, they show a complete set of links to Target and shopping related pages with a single link to get other search results that are not related to Target stores. The second example was "ups" where they they only show links related to United Parcel Services and sending parcels on the first page.
Next up was Johanna Wright, Director of Web Search Product Management at Google. Johanna started off by telling us that that 20% of searches have never been seen before, and that Google is dedicated to serving the long tail of web searches as well as more popular ones. To show us how far the search experience has come in the last few years, she applied the search term "how to tie a tie" to an index that they had saved from 2001, and compared it with what you get today. In 2001 you got a miscellaneous collection of links to sites like "The Indus Entrepreneur" with none about tying ties. Now you get relevant links along with image and video links, a tremendous improvement.
Johanna talked about how speed is essential to a good user experience. A couple of years ago, they added related links to popular search terms like "target" to reduce the number of steps a user needs to make to get to the page they want. Google continues to work on helping users with query formulation. She showed us the options panel that you access by clicking the "search options" link on a search results page and how it can be used to refine a search.
Finally, Dr. Larry Cornett, vice president of the Yahoo! Search Consumer Products division spoke. He started by reassuring us that Yahoo! is still in the search business and that if and when the planned combination of Yahoo! Search with Microsoft goes through, they will still provide their own front end and control their user's experience. Yahoo!'s goal has always been to personalize and structure the web. We saw the new layout for Yahoo! search results in the typical Yahoo! busy style.
After the demo's, the floor was thrown open to audience questions. Someone asked about natural language support for queries. Sean told the story as he has been in the search business for a long time. In the early days of search, natural language queries were considered important research area. Then the issue went away as providing relevant answers to queries became the dominant problem. Now that giving good answers is under control, natural language queries are making a comeback. Recently Microsoft bought Powerset to help them in this area.
There were several questions about the sizes of market segments, and growth rates, particularly in the mobile space, to which the panel would not give answers. The audience did manage to uncover the fact that while adult searches are more prevalent than mobile searches, mobile searches have been growing fast since the introduction of the iPhone and other smartphones.
Another set of questions related to real time search. All three search engines have been working on improving the speed with which they update their indexes so that they are current. There is still an open question about whether the major search engines embrace real time search or make it a separate option.
First, Sean Suchter General Manager of Microsoft's Search Technology Center Silicon Valley told us about the latest innovations in Bing. Sean started out with some numbers, showing that the Internet is still growing at a fast pace and that search is growing faster than the Internet in general. They measure their user's experience and see that about a quarter of searches are failures, resulting in an immediate click back. On the other hand, getting on for a half the search queries are further refined meaning that the user is engaged in a search session. Microsoft will recognize these sessions and use them to improve the user experience.
To simplify the user experience, when they are confident about what a user is searching for, Bing will show one subject on the first page with a number of related links. Sean showed us two examples. Firstly for the search term "target", where they assume the person is looking for the Target chain of stores, they show a complete set of links to Target and shopping related pages with a single link to get other search results that are not related to Target stores. The second example was "ups" where they they only show links related to United Parcel Services and sending parcels on the first page.
Next up was Johanna Wright, Director of Web Search Product Management at Google. Johanna started off by telling us that that 20% of searches have never been seen before, and that Google is dedicated to serving the long tail of web searches as well as more popular ones. To show us how far the search experience has come in the last few years, she applied the search term "how to tie a tie" to an index that they had saved from 2001, and compared it with what you get today. In 2001 you got a miscellaneous collection of links to sites like "The Indus Entrepreneur" with none about tying ties. Now you get relevant links along with image and video links, a tremendous improvement.
Johanna talked about how speed is essential to a good user experience. A couple of years ago, they added related links to popular search terms like "target" to reduce the number of steps a user needs to make to get to the page they want. Google continues to work on helping users with query formulation. She showed us the options panel that you access by clicking the "search options" link on a search results page and how it can be used to refine a search.
Finally, Dr. Larry Cornett, vice president of the Yahoo! Search Consumer Products division spoke. He started by reassuring us that Yahoo! is still in the search business and that if and when the planned combination of Yahoo! Search with Microsoft goes through, they will still provide their own front end and control their user's experience. Yahoo!'s goal has always been to personalize and structure the web. We saw the new layout for Yahoo! search results in the typical Yahoo! busy style.
After the demo's, the floor was thrown open to audience questions. Someone asked about natural language support for queries. Sean told the story as he has been in the search business for a long time. In the early days of search, natural language queries were considered important research area. Then the issue went away as providing relevant answers to queries became the dominant problem. Now that giving good answers is under control, natural language queries are making a comeback. Recently Microsoft bought Powerset to help them in this area.
There were several questions about the sizes of market segments, and growth rates, particularly in the mobile space, to which the panel would not give answers. The audience did manage to uncover the fact that while adult searches are more prevalent than mobile searches, mobile searches have been growing fast since the introduction of the iPhone and other smartphones.
Another set of questions related to real time search. All three search engines have been working on improving the speed with which they update their indexes so that they are current. There is still an open question about whether the major search engines embrace real time search or make it a separate option.
Monday, September 07, 2009
Ikea Culture
We live in an Ikea world. I like to find excuses to visit the nearest Ikea in Palo Alto to lunch in their cafeteria, eating either a smoked salmon plate or Swedish meatballs with Lingonberry jam. The cafeteria has a great view over the South Bay and the East Bay hills. However the reason for this post is to note that Ikea has been popping up in the conversation all over the world.
In China, the Ikea stores have become a great success, for the people, if not for Ikea. This LA Times story reports that Chinese people are flocking to the local Ikea store, to test the bedding, hang out and eat in the cafeteria, maybe even buy some plates, just not to buy anything big.
Meanwhile in LA itself, several young aspiring producers have noticed that an Ikea store is just like a movie studio with lots of little well lit sets showing off bedrooms, living rooms, kitchens. Just the place to make a short episode on the cheap. The actors mike up with wireless mikes outside, rush in and take a few shots and then rush out before any employees notice. Here is Ikea Hights, a soap opera, and here is a send up of The Real World.
Finally, as reported in the New York Times, there has been outrage over the decision by Ikea to change the font in their latest catalog from Futura to Verdana. Futura is a well respected modern san-serif font that suits the Ikea style. Verdana is the generic Microsoft version of a san-serif font that comes on every computer with Windows. I am not sure why this is so important, are these people really complaining that Ikea has lowered its standards to encompass the lowest common denominator font?
In China, the Ikea stores have become a great success, for the people, if not for Ikea. This LA Times story reports that Chinese people are flocking to the local Ikea store, to test the bedding, hang out and eat in the cafeteria, maybe even buy some plates, just not to buy anything big.
Meanwhile in LA itself, several young aspiring producers have noticed that an Ikea store is just like a movie studio with lots of little well lit sets showing off bedrooms, living rooms, kitchens. Just the place to make a short episode on the cheap. The actors mike up with wireless mikes outside, rush in and take a few shots and then rush out before any employees notice. Here is Ikea Hights, a soap opera, and here is a send up of The Real World.
Finally, as reported in the New York Times, there has been outrage over the decision by Ikea to change the font in their latest catalog from Futura to Verdana. Futura is a well respected modern san-serif font that suits the Ikea style. Verdana is the generic Microsoft version of a san-serif font that comes on every computer with Windows. I am not sure why this is so important, are these people really complaining that Ikea has lowered its standards to encompass the lowest common denominator font?
Wednesday, September 02, 2009
Project Voldemort
There were three interesting trends exposed in the talk about Project Voldemort at the August meeting of the SDForum SAM SIG. Firstly Voldemort is another tuple store as opposed to a relational database, the trend that interested me the most. The second trend is implementation of systems described in academic papers. The final trend is to use Open Source as a support mechanism for a large software project. Lets break down each of these trends one at a time. By the way, the presentation was given by Bhupesh Bansal and Jay Kreps, of LinkedIn.
The relational databases have been the reliable store for serious computing for the last 20 years, but recently tuple stores and tuple processing like Map-Reduce have appeared and are starting to challenge the relational database hegemony. In the simplest terms, a tuple store is just a very degenerate relational database. Relations are based on the n-tuple, that is each row in a table contains a number of data items whereas a plain tuple is two data items, a key and a value.
As Jay Kreps explained, to get a web service application to scale, you need to distribute it over a over a cluster of computer systems, and to make this work with a relational database, you need to denormalize your database. The end point of database denormalization is the plain flat tuple store. Jay Kreps also complained that relational databases are not very good at handling data structures like the graphs of connections found in social networking applications, and semi-structured data like text.
In my opinion, tuple stores are no better or worse than relational databases at dealing with graphs between tuples. Tuple stores are more flexible for handling semi-structured data, but again this depends on the application (for more, read my comparison of Map-Reduce with relational databases). Tuple stores are certainly simpler, easier to use, more stable under load and cheaper than a relational database. I will write more about tuple stores at another time.
The second notable trend is for groups to pick up on systems described in academic papers and just implement them. Voldemort is an implementation of the Amazon Dynamo system as described in their paper at the ACM Symposium on Operating Systems Principles. We have seen several other examples of this recently. Google released a set of papers about their data processing systems including Map-Reduce, that has created a number of projects to emulate their functionality. I have written about Hadoop and Hypertable, two examples, and there are others. These are systems for doing very large scale analytic data processing, while Amazon Dynamo and Voldemort are systems for supporting rapid access to large volumes of data such as is needed to support large and complex web sites.
The final trend is Open Source as a support model. Voldemort was developed by LinkedIn, a company whose main business is providing a social and business network on the web. Their primary business is social networking, not writing and supporting a lot of complicated software. LinkedIn decided that they needed a tuple store like Amazon Dynamo and, as they could not buy it, they built it. However they decided they wanted help with support, so they released the software as an Open Source project. Now, Voldemort is being used by several organizations and at least half the people working on code are from outside LinkedIn. When Sandeep Giri started the OpenI project, I asked him why he was releasing it as an Open Source project and he gave the same reason.
The relational databases have been the reliable store for serious computing for the last 20 years, but recently tuple stores and tuple processing like Map-Reduce have appeared and are starting to challenge the relational database hegemony. In the simplest terms, a tuple store is just a very degenerate relational database. Relations are based on the n-tuple, that is each row in a table contains a number of data items whereas a plain tuple is two data items, a key and a value.
As Jay Kreps explained, to get a web service application to scale, you need to distribute it over a over a cluster of computer systems, and to make this work with a relational database, you need to denormalize your database. The end point of database denormalization is the plain flat tuple store. Jay Kreps also complained that relational databases are not very good at handling data structures like the graphs of connections found in social networking applications, and semi-structured data like text.
In my opinion, tuple stores are no better or worse than relational databases at dealing with graphs between tuples. Tuple stores are more flexible for handling semi-structured data, but again this depends on the application (for more, read my comparison of Map-Reduce with relational databases). Tuple stores are certainly simpler, easier to use, more stable under load and cheaper than a relational database. I will write more about tuple stores at another time.
The second notable trend is for groups to pick up on systems described in academic papers and just implement them. Voldemort is an implementation of the Amazon Dynamo system as described in their paper at the ACM Symposium on Operating Systems Principles. We have seen several other examples of this recently. Google released a set of papers about their data processing systems including Map-Reduce, that has created a number of projects to emulate their functionality. I have written about Hadoop and Hypertable, two examples, and there are others. These are systems for doing very large scale analytic data processing, while Amazon Dynamo and Voldemort are systems for supporting rapid access to large volumes of data such as is needed to support large and complex web sites.
The final trend is Open Source as a support model. Voldemort was developed by LinkedIn, a company whose main business is providing a social and business network on the web. Their primary business is social networking, not writing and supporting a lot of complicated software. LinkedIn decided that they needed a tuple store like Amazon Dynamo and, as they could not buy it, they built it. However they decided they wanted help with support, so they released the software as an Open Source project. Now, Voldemort is being used by several organizations and at least half the people working on code are from outside LinkedIn. When Sandeep Giri started the OpenI project, I asked him why he was releasing it as an Open Source project and he gave the same reason.
Subscribe to:
Posts (Atom)
