Bubbles
We are living in the worst economic times since the 1930s. The US economy contracted at an annualized rate of 3.8% in the fourth quarter of 2008, the corresponding figure for Japan is 12.7%, and Iceland may become the first post-depression Western economy to suffer from an outright fiscal collapse. Economists tell us that one of the reasons for this worldwide recession is a 'housing bubble' where banks overestimated a borrower's ability to pay back a loan and where house buyers--armed with cheap loans--overestimated the true worth of a house.
The recent Internet bubble is still fresh in some of our minds, where there was a similar overestimation of the true worth of Internet-enabled businesses. That bubble crashed too, with consequences suffered by the entire economy.
Unfortunately, bubbles are not uncommon in networking research. Certain topics appear seemingly from nowhere, become 'hot,' propelled by interest from both leading researchers and funding agencies, and just as mysteriously die off, leaving behind a flood of papers, mostly in second- and third-tier conferences, written by authors only too keen to jump on a trend. Bubbles lead to an overexpenditure of research effort on marginal topics, wasting resources and breeding a certain degree of cynicism amongst our brightest young minds. Moreover, they drain resources from more deserving but less hyped ventures. Can our experience with economic bubbles shed light on research bubbles and teach us how to avoid them?
Both economic and research bubbles share some similarities, such as having unrealistic expectations about what can be achieved by a company, the real-estate market, or a new technology. Bubble participants either naively or cynically invest time and money in solutions and technologies whose success is far from assured and whose widespread adoption would require the complete overthrow of legacy infrastructure. To avoid being caught in a bubble, or to merely avoid being caught in the tail end of one (being at the leading edge of a bubble is both fun and profitable!), ask tough questions about the underlying assumptions. In the midst of the housing bubble, could one point out housing prices could down as easily as they could go up? Could anyone have believed in the '90s that videoconferencing, ATM, RSVP and other 'hot' topics would soon be consigned to the midden heap of history? I think so. It only requires the willingness to question every assumption and draw the inevitable conclusions.
I think that in the end, what really inflates a bubble is money. Cheap money from venture capitalists, banks, and funding agencies makes it profitable to enter a bubble and make it grow. So it is important that the gatekeepers of funding be vigilant. They should be prepared to turn down applications for funding that smack of riding the bubble. Experienced researchers should willingly serve on grant panels, then should be prepared to be critical with even their favourite areas of research if necessary.
Finally, bubbles can be identified and quashed by an active media. The press should have more deeply questioned the Internet and housing bubbles. Research conferences in our field should do the same for research bubbles. Paper reviewers and program committees thus play the same role as investigative journalists.
This is not to say that all speculative ideas should be systematically de-funded and rejected. There should always be room for open-minded, blue-sky research. However, this activity should be limited and clearly identified. Perhaps every conference should have blue-sky sessions where all assumptions are left unchallenged (our community has done this with recent papers on 'clean-slate' designs). The best of these ideas, when proven to be sound, could then be funded and widely adopted.
Of course, I am assuming that that we can get out of bubbles by rational means. Humans are all too fallible, however, and bubble thinking plays on human foibles. Worse, there is an incentive structure that encourages bubble formation: people at the leading edge of a bubble are disproportionately rewarded and people at the tail end can point to large body of literature (emerging from top-ranked places!) to justify their work, which reduces their cognitive effort. So, bubbles may be here to stay.
Nevertheless, given the destructive effects of bubbles over the long term, I suggest that we look out for them, deflating them before they deflate us!
The recent Internet bubble is still fresh in some of our minds, where there was a similar overestimation of the true worth of Internet-enabled businesses. That bubble crashed too, with consequences suffered by the entire economy.
Unfortunately, bubbles are not uncommon in networking research. Certain topics appear seemingly from nowhere, become 'hot,' propelled by interest from both leading researchers and funding agencies, and just as mysteriously die off, leaving behind a flood of papers, mostly in second- and third-tier conferences, written by authors only too keen to jump on a trend. Bubbles lead to an overexpenditure of research effort on marginal topics, wasting resources and breeding a certain degree of cynicism amongst our brightest young minds. Moreover, they drain resources from more deserving but less hyped ventures. Can our experience with economic bubbles shed light on research bubbles and teach us how to avoid them?
Both economic and research bubbles share some similarities, such as having unrealistic expectations about what can be achieved by a company, the real-estate market, or a new technology. Bubble participants either naively or cynically invest time and money in solutions and technologies whose success is far from assured and whose widespread adoption would require the complete overthrow of legacy infrastructure. To avoid being caught in a bubble, or to merely avoid being caught in the tail end of one (being at the leading edge of a bubble is both fun and profitable!), ask tough questions about the underlying assumptions. In the midst of the housing bubble, could one point out housing prices could down as easily as they could go up? Could anyone have believed in the '90s that videoconferencing, ATM, RSVP and other 'hot' topics would soon be consigned to the midden heap of history? I think so. It only requires the willingness to question every assumption and draw the inevitable conclusions.
I think that in the end, what really inflates a bubble is money. Cheap money from venture capitalists, banks, and funding agencies makes it profitable to enter a bubble and make it grow. So it is important that the gatekeepers of funding be vigilant. They should be prepared to turn down applications for funding that smack of riding the bubble. Experienced researchers should willingly serve on grant panels, then should be prepared to be critical with even their favourite areas of research if necessary.
Finally, bubbles can be identified and quashed by an active media. The press should have more deeply questioned the Internet and housing bubbles. Research conferences in our field should do the same for research bubbles. Paper reviewers and program committees thus play the same role as investigative journalists.
This is not to say that all speculative ideas should be systematically de-funded and rejected. There should always be room for open-minded, blue-sky research. However, this activity should be limited and clearly identified. Perhaps every conference should have blue-sky sessions where all assumptions are left unchallenged (our community has done this with recent papers on 'clean-slate' designs). The best of these ideas, when proven to be sound, could then be funded and widely adopted.
Of course, I am assuming that that we can get out of bubbles by rational means. Humans are all too fallible, however, and bubble thinking plays on human foibles. Worse, there is an incentive structure that encourages bubble formation: people at the leading edge of a bubble are disproportionately rewarded and people at the tail end can point to large body of literature (emerging from top-ranked places!) to justify their work, which reduces their cognitive effort. So, bubbles may be here to stay.
Nevertheless, given the destructive effects of bubbles over the long term, I suggest that we look out for them, deflating them before they deflate us!
posted by S. Keshav at 9:59 AM
4 Comments:
Very interesting read, Keshav. I think that in addition to the two causes you list for bubbles - unrealistic expectations and the attractiveness of being at the start of the bubble - there are two other elements that come in: how "beautiful" and "spacious" an area is.
Many bubble areas get a lot of research because they combine a potentially real problem with an opportunity to try neat algorithms, mathematics, etc. Peer-to-peer is an example of this: there were tons of papers published on different topologies for DHTs, then papers saying how these are all some representations of different kinds of groups, then papers on how really you mostly need to look one or two DHTs. Multicast also had its share of room for playing with algorithms, which resulted in a lot of designs exploring different portions of the tradeoff space. Every researcher dreams of applying some clever insight to solve an important problem, and fields with a lot of opportunity for this tend to be attractive even if they lack the real problem. I'm not sure what we can do about this other than promote more conferences such as HotNets and HotOS where people can pose questions such as "is area X of research leading the systems community astray".
The second element that helps is when a field is based on an assumption so sweeping that it changes everything. Take for example peer-to-peer systems again: you could write about topologies, routing, byzantine fault tolerance, security, storage management, running SQL queries on DHTs, and so forth. If you make the assumption that "a significant portion of future systems will be peer-to-peer" then you can take every systems problem available today and pose it in the peer-to-peer setting. This leads to a lot of clever ideas and very fun times doing the research, but may ultimately be misguided. In contrast, some of the best systems/networking work is taking an *existing* type of system and challenging a long-held assumption in there to obtain a real benefit. But it's harder to find ideas like this because you need to work within a constrained setting. It would be interesting to try to list research ideas that were not part of a bubble but were highly influential.
At a very high level perhaps one of the main things researchers have missed is economics. For example, we get video over TCP and HTTP today because there's enough bandwidth and there are companies that will place the video near the client (CDNs). Multicast doesn't run in the wide area because nobody could figure out how to charge for it. NOW-like commodity-hardware cluster computing (GFS, MapReduce) has beat non-commodity solutions for powering (some) big computations, because of economies of scale. Of course it's hard to forsee such developments in advance, but perhaps looking at market trends and technology trends can help.
Matei, thanks for your perceptive comments. In my note, I was trying to find commonalities between economic and research bubbles. The two things you point out, beauty and space, are not found in economic bubbles, or at least, not to the extent in research bubbles. So, they didn't even occur to me!
That said, what you point out is true. It is the dream of every researcher to be the first to break new ground. We are all bubble-blowers! My argument is that we should recognize this, recognize the down-sides, and try to be skeptical. As I said in a panel in 1999, a researcher should be either relevant and legacy-compatible or irrelevant and happy!
This is a current grad student in networking. Your article on research bubbles has been a great read. I have to agree that these phenomena breed cynicism. Especially when you see a certain (perceived) tier-1 conference accepting very mediocre papers just because they talk about a certain "hot" area, have a "great" dataset, or have been media-hyped before submission. Research bubbles probably refer to the former, and probably pressurize reviewers to accept papers. The question of the latter two is yet another story, that can depend on one strong reviewer defending his/her opinions in a TPC. I know this has been discussed numerous times in the past, but is there light at the end of the tunnel, for posterity?
A simple test shows where the networking community is spiraling in its sense of camaraderie - e2e-interest archives in '00 and before show a very different level of participation (people raising new problems, people citing their own papers and tools, people using others' work, ...) from the rather dark mailing list that it is today.
"but is there light at the end of the tunnel, for posterity?" I don't think so. As long as top conferences get more good papers than they can (or want to) accommodate, there will be borderline accepts, and in these cases 'hotness', pre-publicity, and data sets are going to play a big role. That's just the way it is.
As for the networking community spiraling, I think that's inevitable given the size of the community and how its grown.
Post a Comment
Links to this post:
Create a Link
<< Home