A talk in which I felt I had ben asked only to say things I hadn't said before. It was part of symposium at MIT arranged by Andy Van Dam in honor of the 50th anniversary of Vannevar Bush's visionary article "As We May Think" in the Atlantic monthly in 1945. Various gurus from the history of hypertext were assembled, to talk and be taped.
(This is a text approximating to the talk)
It is a great honor for me to be invited to join you in celebrating Vannevar Bush, and especially to do so alongside our great gurus of hypertext. I feel like a steam locomotive designer in the presence of Watt and Charles and Boyle, discussing the significance of some remarks of Newton's.
Anyone who reads over the "Atlantic Monthly" article today will have been struck with the distance and accuracy of Bush's vision, and at the same time the things (such as the general purpose computer) which were just around the corner but the awareness of which he did not have the benefit. I don't want to go over this in detail, in the hope that others will also in part and together, each picking out things which strike us particularly, we will give an interesting if not complete commentary on his work.
It's interesting to me to see the core problem which he starts and from which he derives the need for the MEMEX. It's interesting because sometimes seemingly very related contributions to the networking and hypertext fields have come from apparently different problems. Ted Nelson, who coined the word, described hypertext from the literary point of view (with particular emphasis on authors getting just rewards for their work), while Doug Englebart with Augment, and I largely with the Web, were looking at helping groups of people work together.
For Bush, the daunting challenge for humanity was to cope with the awful growth of what he called "the record". He considered the plight of a researcher beset by so much research that despite narrowing his field, he could not hope to read all the relevant material. He was already feeling the threat of the information overload in 1945 -- that's before digital computers were around. The conclusion of his article suggests that the future of the world was, he felt, at stake. (Perhaps it was indicative of the times that the researcher in such a key role for society.) The thought I discuss with you today is that though the MEMEX is with is, whether we as a result feel very much more confident about our destiny.
Bush presented the problem from the point of view of a single researcher, and he provided a solution for a single researcher. The MEMEX was a machine which allowed an individual to store, rapidly retrieve documents, and to store and rapidly follow random associations between pairs of documents.
To a large part we have MEMEXes on our desks today. We have not yet seen the wide scale deployment of easy human interfaces for editing hypertext and making links. (I find this constantly frustrating, but always assume will be cured by cheap commercial products within the year.) In part the speed of the net has replaced the size of the person record store, but otherwise a web browser with an editor gives quite a good substitute for a MEMEX.
It is, then, a good time 50 years on to sit back and consider to what extent we have actually made life easier. We have access to information: but have we been solving problems? Well, there are many things it is much easier for individuals today than 5 years ago. Personally I don't feel that the web has made great strides in helping us work as a global team.
Perhaps I should explain where I'm coming from. I had (and still have) a dream that the web could be less of a television channel and more of an interactive sea of shared knowledge. I imagine it immersing us as a warm, friendly environment made of the things we and our friends have seen, heard, believe or have figured out. I would like it to bring our friends and colleagues closer, in that by working on this knowledge together we can come to better understandings. If misunderstandings are the cause of many of the world's woes, then can we not work them out in cyberspace. And, having worked them out, we leave for those who follow a trail of our reasoning and assumptions for them to adopt, or correct.
The initial web work was driven largely by my working on projects with people in remote sites. These people had great enthusiasm but little time or travel budget. (Also, as a technologists, we would all want to focus on the technical problems, leaving human interaction to the strictly necessary.) The dream is that if everybody works from day to day using the web as their notebook, mailer and calendar, (just as Englebart's NLS/Augment system allowed one to, for example) then the scaling problems of teams and organizations could somehow be solved. This is a dream.
You are probably all familiar with a few organizations which have grown from a couple of people to pass 50 or 60. You've probably noticed the sighs of the staff as they realize that suddenly they don't know everyone. You've seen projects run amok and noone know why. You've seen them leave and start small companies. You've seen books about it in airport bookstores.
Management of a growing team is a problem in a class which we can refer to as management-complete problems. Problems in this domain do not surcomb to technical analysis, but can be tackled to some degree of accuracy by means of what is called a "management fad". A fad is a heuristic which enjoys the trust of those who employ it. An essential element of a fad is that it forces those involved to think about the problem, and therefore in principle get a lot closer to a good solution than they would have done otherwise.
But seriously, history is the history of mankind trying to work together on every scale and doing more or less well. The concern, that we "perish in conflict" behind Bush's closing paragraph is quite understandable when dated 1945 and not unreasonable now. If we can find a tool for the "self-managing team(tm)" then we will have done well. But we should be aware that the political process, education, and much social activity has similar scaling problems and might also be worth a thought.
If, even given a MEMEX, we have not made progress in working in groups, then clearly we can blame the lack of good collaborative software, navigational tools, and the fact that not every one of the billions of people on the earth has an internet-connected computer.
Suppose they did. Apart from the utter horror of having nowhere to turn without seeing one of those little MEMEXes, would our problems be solved anyway? (Incidentally, I'm assuming here that some quirks of the current software are ironed out. One of these if that for some reason one cannot edit hypertext within a browser, and instead one must resort to the neolithic practise of editing the HTML source file by hand to make links. There is nothing I consider stranger than the call in a local "Help Wanted" section of the paper for HTML writers!)
... Suppose then all these minor problems are cleared up, would we be seriously empowered as Bush would like us to be, as a whole?
Let's think about scaling problems. Let's think of some large numbers. The number of web documents. The number of people in the world. The number of neurons in the brian. We're thinking of lots of things all connected together. Web objects, people and neurons all have the ability to have random associations. The neurons seem to work (on a good day) as a integrated team. The people do in parts. The web documents just sit there.
But pretty soon the web documents will start getting up and wandering around.
[expand on mobile code]
So when web objects become mobile, and start wandering around and interacting with each other, would you now put much money on them making sense as a whole?
We can draw some analogies, of course. Where people have relationships (of various sorts), web documents have links, and neurons have synapses. People and documents tend to be locally grouped into hierarchical forms; neurons in the brian have a large-scale structure but we don't know much about the details. In the case of the neurons and the people, there have been analogies drawn (Minsky in Society of Mind, for example). There have been plenty of analogies between drawn between the web and biological systems: George Bret of the Clearinghouse for Networked Information Discovery and Retrieval refers to it all as Kudzu. Someone even drew my attention to an article on a virus outbreak saying that it was spreading like the Internet! Douglas Hofstadter compares the mind to an anthill...
Ants, Neurons, objects, particles, people. In each case, the whole operates only because the parts interoperate. The behaviour of the whole is in some way dictated by the rules of behaviour of the parts. This may be a view influenced too much by physics, but I find it useful. It makes you think about how you predict the rules of the whole from the rules of the parts, and then as a global engineer (constitution writer, etc) how you can phrase the local laws to engender the global behaviour he desires.
For people, we call these rules variously the constitution, laws, or codes of ethics, for example. These rules are things which are accepted across the board. For particles, we call then the laws of physics. For web objects they are the protocol standards.
And equally as we have become used to these analogies, we know their limitations. We know that data objects have been ineffective at emulating brains. We know that people's behaviour is something specially different from the machinations of the computer, or the interactions of identical nameless particles. [...]
Vannevar Bush introduced the idea of mechanising the representation of random associations. In this festschrift occasion, let's look at what associations we have with "association".
Association - Connection - Liaison - Nexus - Tie - Link
What links do we have with "link"?
Link - Confusion
.. or so it was for many when the web seemed at first to threaten the orderly hierarchical world.
Link - Coolness
as it later became as the "click here" brigade seized and were seized by the popular imagination.
Link - Readership
is what it meant for the authors of the targets of links. Every link brought more readers. For some, this was an ego boost. For others, it implied academic respectability of the new de facto citation index.
For those who sold information, products or advertising space, this meant
Link - $
Now that is an interesting connection. The link is a unit of connectivity in hypertext. The dollar (ECU or whatever) is a unit of behaviour in the market. It is a unit in the market economy protocol which defines in part the behaviour of people. So here we see a connection between a world of objects and the microcosm of people. There many such connections, it is just that a relationship between links and dollars connects the simplest part of the hypertext rules up to the simplest human social rules, and so is easier to analyse. In fact, the web influences how we live, and how we live should (when we clear up the few advances I mentioned) influence the web in many less [materialistic] ways.
There is no question that global hypertext influences people. We've seen the reactions: great feelings of empowerment. (Empowerment is a 1990s word for what you feel -- for example when you first meet global hypertext). And misgivings, fear, despair for a normal healthy life. [Similarly, perhaps less obviously, the web is influenced by society.] Social changes change the organisations into which we are grouped, the names of documents change, and links break. Systems only fly which will work socially. Nothing works which requires the immediate buy-in of the entire world: technological change ripples through the social structure.
We are constantly refining the microscopic rules. Governments are constantly changing the laws -- and hoping that people will follow them. Bodies such as the World Wide Web Consortium are evolving the protocols, and trying to arrange for developers to follow them. We engineer the microscopic rules in the hope that the end result will be a macroscopic effect that will satisfy us. We are little like physicists tweaking the gas laws, and hoping that tomorrow the atmosphere won't accidentally condense into a small blob.
For society we have goals which are expressed in terms of the individual: fairness, rights to speak, health. We have constraints which are expressed in terms of the whole, such as global peace, the safety of the race. On a humbler level protocol designers have local constraints such as platform-independence, vendor-neutrality, interoperability, and global ones such as system stability and graceful decay, and how the system as a whole looks to a user. Now it seems that in order to achieve the goal of Vannevar Bush, of growing in race experience before we perish in conflict, we need to change the way we may think as we change the way the machines operate.
The problem Bush was addressing, or the problem of the individual researcher, was one of system topology. The poor person has successively narrowed and narrowed his or her field of interest in order to cope with the information overload, and soon is connected only to things of very local interest. [pic] The topology clearly doesn't work, because there is no path for the transfer of knowledge from one discipline and the next. In order for us to make progress, we have to think about topological features of large systems, for machines and for people.
There are some common properties between human systems and networked systems which we are already beginning to appreciate. The internet grew as a decentralized system. Its design was tasked by the US military, a centrally controlled system, but the Internet Protocols, being decentralized, outgrew that space. The Internet has bred in its engineers a respect for decentralized systems which has in some cases led to anarchic political views. Just as IP packets have no single point of control, nor do Internet news groups. A feeling grows in the community that the whole Internet design process should be able to exist on its own account in a vacuum. It is not clear that this model has survived increase of scale, or impact with the commercial world, which happened at the same time. The lessons, though, have been well learned. The web, like IP, grew because it was decentralized. There was no central authority for web sites, so they could grow at will, appearing spontaneously. Bush's vision is of a decentralized academic society, in which no central figure or library is essential to the process.
Decentralization is often a myth. In real systems, the microscopic rules which ensure decentralization are enforced by some global authority if you peek under the covers. The IP protocols are defined in an RFC which sits at a definitive location. The happy intercourse between people in the street is safe only thanks to the kindly police officer on the corner and his hierarchical attachment to the supreme court.
That brings us to another interesting feature of topologies, and that is their variation with scale. A few years ago, the world was fascinated (quite rightly) with fractal patterns. For those of you too young to remember the fad :-), a fractal pattern is one like the shape of a fern, which when you look at it closer and closer rewards you with a similar level of interest through many orders of magnitude. It is like the tree outside my office window, but it is not like my office block, whose interesting features are limited to a rectangle maybe 100 meters long, windows around a meter wide, and rivets a few millimeters wide.
Is society fractal? Yes, it certainly is. There is structure at the highest levels and the lowest levels. There are great big links formed by organizations which themselves are made up of smaller links. You can simplify society on a number of levels. You look at a newspaper and it will perhaps have a few sorties of domestic bliss or otherwise in the neighborhood, a story on the town, a story at state level, and (even in Boston), usually some stories about world affairs. (For those not from the area, the Boston paper's typical foreign news headline is "Boston woman has twins in China".)
People need to be part of the fractal pattern. They need to be part of organisms at each scale. We appreciate that a person needs a balance between interest in self, family, town, state and planet. A person needs connections at each scale. People who lack connections at any given scale feel frustrated. The international jet-setter and the person who always stays at home share that frustration. Could it be that human beings are programmed with some microscopic rules which induce them to act so as to form a wholesome society? Will these rules still serve us when we are "empowered" by the web, or will evolution give us no clues how to continue?
Look at web "home pages". "Home pages" are representative of people, organizations, or concepts. Good ones tend to, just like people, have connections of widely varying "length". Perhaps as the web grows we will be able to see fractal structure emerge in its interconnections. Perhaps we ought to bear this in mind as we build our own webs.
One of the reasons that the web spread was that the hypertext model does not constrain the information it represents. This has allowed people to represent topologies they need. We have found that people love to use trees, but like to have more than one, sometimes overlapping. We have found they need structure and involvement at all scales.
The rules of web behavior are being defined now, by programmers meeting in groups, by lawyers and politicians. They are making decisions.
For example, one simple principle is end to end confidentiality. what a simple rule, that in cyberspace, privacy should be available between two people irrespective of physical location. This rule is opposed by those who feel that it would cause society to be come as a whole unstable.
Other principles is the right to anonymity.
Well established rules are that every object may have an identifier (a URL) such that when two people get renderings of the object using the same URL, they should get renderings of the same object.
Another rule is that the "GET" operation may not have side-effects.
As we move into the world of mobile code, of secure systems, of network payment, the new principles are being silently or not laid down. These principles will define the behaviour of a new machine, a new anthill, a new brain, which is the sum of ourselves and our creations.
Vannevar Bush's MEMEX was described as a complex machine. We see it now as a cog in a larger system. We feel fairly proud that we have built MEMEX-like machines. But now we have links, do we know what to do with them? When it comes to designing larger machine we are still banging the rocks together.
But we are at a time of great creativity, of great potential for change for better or worse, and there is a feeling that in fact we may be able to bring our collective teamwork up to a level at which we can ensure our survival. We have got "the great record" at our fingertips, and maybe we may yet learn to "grow in the wisdom of race experience" such that Vannevar Bush might be proud of us.