Open Source is Not Enough!

Steven Pemberton, CWI and W3C, Amsterdam

1609

VOC Logo

The Dutch East India company, the world's first multinational, and the first company to issue stock, is fed up with the 8 to 12 months it takes to travel.

Routes

Routes to the east indies

The World in 1595

Mercator Junior's map of the world

Henry Hudson

So the VOC decide to do some research, and hire an Englishman, Henry Hudson, to discover the Northeast route.

He is contracted to do only that.

But he doesn't.

Hudson's Voyage

Then, and for centuries after, all trans-Atlantic shipping was done with sailing ships.

Hudson and the Halve Maen in the Hudson River

Steam ships

When steam ships were introduced they were not reliable enough to travel trans-Atlantic distances, they couldn't travel far without breaking down, and they were inclined to blow up. The Fighting Temeraire tugged to her last Berth to be broken up, by Turner, 1838:

An early steam ship towing a sailing ship

Niche

But steam ships were able to find a niche in lake and river transport, where the distances were short, and where they had the advantage of being able to travel against the wind and on wind-still days.

Once in the niche they could improve reliability until they were able to travel trans-Atlantic.

Once that happened, all shipping switched to steam, and all the companies producing trans-Atlantic sailing ships went out of business; not one survived into the 20th century.

Sustaining improvements

Sustaining improvements Typically, technologies improve gradually and continually over time. These are sustaining improvements. An existing technology fulfils or exceeds the market's needs.

Disruptive improvements

A disruptive technology is introducedSteam ships on the other hand are an example of a disruptive technology, a term introduced in the excellent book The Innovator's Dilemma by Clayton M. Christensen.

A disruptive technology when introduced is able to do similar things as an existing technology, but has one or more disadvantages that make it less desirable for the existing market, such as higher price, lower performance, lower reliability etc.

Niches

A disruptive technology improves to beat extant technologiesIf that were all, then of course no one would buy it, but a disruptive technology always has at least one advantage over existing technologies that despite its disadvantages allows it to find a niche, and therefore survive.

Whilst in this niche it is able to improve, typically more rapidly than the existing technology, until it performs equally with the existing technology, or at least sufficiently for the market needs, but with the added advantages that it had that allowed it to survive.

As a result the existing market steps over to the new technology and the old one dies (typically taking the companies producing the technology with it).

Examples

There are huge numbers of examples of disruptive technologies.

Christensen's book documents many other examples, such as pneumatic diggers, 14 inch disks being replaced by 8 inch disks, being replaced by 5¼ inch disks, being replaced by 3½ inch disks, and the change from mainframe computers to minicomputers to PCs.

One example: 5¼ inch disks

It is difficult to believe that the change from 8 inch to 5¼ inch disk was a disruptive innovation, but they represented different technologies. Compare the two when 5¼ inch disks were introduced in 1981:

8 inch 5¼ inch
Capacity 60 MB 10 MB
Volume 10 litre 2.5 litre
Weight 10 kg 2.75 kg
Access time 30 µs 160 µs
Cost $3000 $2000
Cost per Mbyte $50 $200

Other examples

You can probably identify lots of others examples of disruptive innovations just by thinking of things that have essentially disappeared: cars, digital cameras, CDs, DVDs, LCD TVs, ...

Why don't the existing companies just shift to the new technology?

We can see lots of disruption now as a result of the internet (newspapers, books, music, film, travel agents, shops, ...)

Is Open Source a Disruptive Technology?

Take Linux:

The stages of acceptance of a new technology

Denial

Anger

Bargaining

Acceptance

Making it better: Usability

Usability comes from designing things (software/programming languages/cookers) to allow people to do their work:

Efficient, Error-free with Enjoyment or
Fast, Faultless and Fun

Don't confuse usability with learnability. They are distinct and different.

Usability is important

What are the features of websites that you go back to regularly, that differentiate them from websites with the same purpose that you don't go back to?

Differentiating features

Forrester did some research on this:

(the rest is noise: 14% and lower)

Applied to software

Scratching Someone Else's Itch

(Why Open Source can't do Usability)

The itch

Every good work of software starts by scratching a developer's personal itch.

Eric Raymond, The Cathedral and the Bazaar

Someone else's itch

There's a closely related issue, however that I don't know how to solve yet without a big player with a lot of money, which is doing systematic user interface end user testing. We're not very good at that yet, we need to find a way to be good at it.

Eric Raymond, Why Open Source will Rule

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Intuitives live in their own private universe, depending on an internal model of external events. [...]

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Intuitives live in their own private universe, depending on an internal model of external events. [...]

When Intuitives drive to work, they watch the tectonic plates, deep in the earth's crust, rubbing together...

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Intuitives live in their own private universe, depending on an internal model of external events. [...]

When Intuitives drive to work, they watch the tectonic plates, deep in the earth's crust, rubbing together. They run into the cow.

Programmers

Open Source software is produced by programmers.

Programmers are different from most people.

For instance, a far greater proportion of programmers are intuitives than in the general public. They like the command line.

This means that when programmers produce open source software, since they are largely scratching their own itch, they will tend to produce the software for themselves, and in particular be perfectly content with the (programmer-oriented) user interface.

As an example, a recent review of photo editing software by the Dutch consumer reports magazine placed The Gimp as last because of its lack of ease of use.

Non-programmers

If Open Source software is to make inroads into the general public, something will have to be done about this.

The typical response to a complaint about a piece of open source software is "I'll show you where the source is".

But by definition the people with this particular itch won't be programmers, and they won't know how to fix the problem:

The challenge

This means that if Open Source Software is to appeal to people that are not producing it

As Raymond remarked "a big player with a lot of money" would do it (and see Ubuntu), but that is not scalable.

Robert Metcalfe

Invented Ethernet

Founded 3Com

Predicted the collapse of the internet in 1996

Has a law named after him

Metcalfe's Law

Metcalfe proposes that the value of a network is proportional to the square of the number of nodes.

v(n)=n2

Visual demonstration of the mathsSimple maths shows that if you split a network into two, it halves the total value:

(n/2)2 + (n/2)2 = n2/4 + n2/4 = n2/2

This is why it is good that there is only one email network, and bad that there are so many Instant Messenger networks. It is why it is good that there is only one World Wide Web.

The Value of things joined

Internet: distance vs cost - the true cost of communication

The Value of things joined

Internet: distance vs cost - the true cost of communication

The web: everyone sharing information shows us the true cost of information

The Value of things joined

Internet: distance vs cost - the true cost of communication

The web: everyone sharing information shows us the true cost of information

For instance Wikipedia: 100M hours work

The Value of things joined

Internet: distance vs cost - the true cost of communication

The web: everyone sharing information shows us the true cost of information

For instance Wikipedia: 100M hours work - the number of hours the US population spends watching adverts on TV in a weekend.

The Value of things joined

Internet: distance vs cost - the true cost of communication

The web: everyone sharing information shows us the true cost of information

For instance Wikipedia: 100M hours work - the number of hours the US population spends watching adverts on TV in a weekend.

For instance: Open street map.

Open source

Also an example of Things Joined

gcc → kernel → apps

The Open Source ecosphere just keeps getting better and better.

But apps are islands

In the days before graphics interfaces, almost all programs could be piped together to produce something more useful.

Unfortunately we have lost that ability.

Monoliths

Each application contains a functional part, a data accessing part and an interface part.

We need to design an architecture where those three parts are separable, so that a program's data is available to other applications (think mashups); so that a program's functionality is available to other applications (think pipes), and where the user interface is replaceable without rewriting the whole app (think usability).

We need open data, open functionality and open interfaces as well.

An application ecosphere

With all the parts of applications in separate pieces, applications become a Metcalfe network of parts that can join together: the value of the whole increasing quadratically with the number of applications.

When you write a new application, you have done more than just added an application to the pool, you have added new functionality and new sources of data to the whole.

Conclusion

Open Source is probably a disruptive technology.

Part of being disruptive is having advantages, and about getting rid of the disadvantages.

One way of improving is to design an architecture that allows us to plug things together more easily, and in the process increasing the Metcalfe value of the whole

My slides will be online