by Eric Armstrong

It's a truism in the open source world that we can work on code that already exists--fixing bugs and adding features--but it is exceedingly difficult to carry on a design discussion online. The questions, "what should we do?" and "how should we do it?", are just too nebulous. It's the kind of discussion that tends to branch forever--what features should it have?, how fast should it be?, how important is it to be fast?, which library should we use?, who's the user?, what goals are we trying to achieve?, what's the best way to achieve those goals?. And to further complicate matters, many of the answers are interconnected.

It's very difficult to manage a discussion of that kind, even if everyone is in the same room (or on the same phone call). Trying to do it online has been virtually impossible. So the projects that tend to succeed (as pointed out by Eric Raymond), have a "Benevolent Czar" at their center who puts a stake in the ground and says "this is what it is". That person generally provides the starting point, as well, and is the ultimate arbiter for decisions about extensions to the core.

But there many good ideas out there in cyberspace. What if it were possible to manage the tendency of the discussion to branch ad infinitum? What if it were possible to use those discussions to surface the important issues, identify the alternatives, make reasonable choices and, above all, provide a readable history of discussion that made it easy for someone coming along later to understand the basic architecture and find out why things are the way the are?

There is an interesting coalition of technologies that could provide those very benefits.

Issue-Based Information Systems (IBIS)

To manage the exponential explosion of questions, ideas, and arguments that occur in a far-reaching inquiry, Horst Rittel defined the IBIS concept. It was described most succinctly by Jeff Conklin in a seminal article, The IBIS Manual. (He also wrote a book on his application of the technique, Dialogue Mapping. Other white papers, including his excellent analysis of Wicked Problems, are at his Cognexus site.)

In IBIS, a conversation is divided into issues, proposals, and pro-and-con arguments over the proposals. There is also the notion of a "point of information" that helps to inform some point or another in the hierarchy. There are also questions and answers, for clarifications.

One of the main features of IBIS is the observation that an argument over whether to do X or Y, ostensibly in relation to some feature, is often in reality an argument over competing issues. So when one person wants to do X, because it's really nice for the user, and someone else is vehemently against it because it makes the system go slow, what's really going on is a competition between two important issues--usability and performance.

When a heated discussion of that sort comes about, you can bet that there are one or more underlying issues that have yet to be identified. In fact, Jeff's experience as a facilitator for such conversations suggests that the facilitator's most important assignment is to recognize such hidden issues, and surface them.

Another terrific feature of that system is that when an issue comes up, it's recorded. When an alternative surfaces, it's recorded. Since no final decision is reached until all all issues and alternatives have been examined, people relax, knowing that their ideas and objections will be visited in due course.

When finished, the structured discussion helps to assure everyone that the best possible decision has been reached. Perhaps even more importantly, it provides a readable record of the decision-making process--something that helps a newcomer come to speed, and which is vital if a decision needs to be re-visited.

Darwin Information-Typing Architecture (DITA)

At Sun, I've been investigating the DITA format, and looking into the architecture of online document-collaboration systems to edit DITA documents. DITA is an extremely well-designed system for structuring documents into reusable topics. Most important for this application, it allows the existing document types to be extended to create new topic topics. (In an object-oriented system, the Object type is the root of all others. In DITA, it is the Topic type.)

That effort gets really interesting, because DITA is an extensible format. I'm thinking we can extend it to create an IBIS-format. DITA also provides for transclusions, or content
references (conrefs), which makes it possible to
deal with the many interconnections between
proposals and the issues they are intended to resolve. With such a format, and with the ability to collaboratively edit a common document, we
stand a chance of carrying on effective design-and-decision discussions
online!

Learning to Use the System

Of course, such a system will not be all that easy to use, at first, A certain amount of user education will be required.

Structured authoring systems are difficult in general, because you always have to think about what it is you're  trying to say, rather than just saying it. So instead of just starting a new paragraph and adding a comment, you have to decide whether you're asking a question, putting in a comment, making an argument, proposing an alternative, raising an issue, etc. It requires extra effort, and it's something of a strain, because you can't put things any old place you want to.

You can't just make a proposal, for example. You have to create an issue first, so you can put your proposal under it. That's a good thing with respect to the conversation, because it means there is automatically a place for alternative proposals. But it's extra work when you're trying to express your ideas.

That extra work has caused difficulty in the past. It's one of the major objections people have expressed to systems of that kind. It's the kind of problem that goes away once you have internalized the system. At that point, you know it well enough that you always know what to do. But it causes problems in the early adoption phases, so user education is a must.

But there are two factors that suggest we can live with that requirement:

• The growth of structured document systems like SolBook, DocBook, and DITA have created a whole new class of users who are used to working with structures that enforce a certain amount of discipline.
  
  
• As Doug Engelbart has noted, the social component of any complex system is just as important as the technology component. While it's true that the system has to get smarter, it's equally true that the user has to get smarter, as well--after all, along with cars and freeways came the need for driver education.