A collection of articles, ideas, and rambling from a guy who wrote some software that one time.

Sunday, November 27, 2005

"Web 3.0", or Why Mantissa is What the Web is Missing

PC programmers originally wrote programs for DOS. It was simple. It was direct. A program could do whatever it wanted. It could write directly to video memory, it controlled the whole screen.

Later PC programmers had to learn to write programs for Windows. At first, that was a big pain. The Windows API, even Win16, is hundreds of times more complex than the DOS "API", if you could even say it had one. Applications had to be changed so that their UIs behaved like Windows did, rather than what was "best" for their application domain. Most programs had to abandon their crazy key-bindings to make room for C-c and C-v to mean "copy" and "paste". (Even we geeks only have room in our hearts for one or two sets of crazy key bindings.)

Browsers are today's "video memory"; applications put information wherever they want, with no user-interface conventions that aren't enforced by the browser itself. There is no operating system for the web, no mechanism to make different applications play nicely together on a single server, let alone on a single page. The web is in need of a major shift.

Many would have you believe that "Web 2.0" - AJAX and such - is this shift. "Web 2.0", insofar as I can tell that it refers to anything concrete, is the idea that web applications should respond to the user interactively. This is something that applications have been doing even before DOS; it is not a new idea, and in a sense, it is only the web playing catch-up in an area where desktop applications have long been the clear winner.

What I want to know is, where is the integration paradigm? Where is the clipboard for the web?

I think that whatever answers this question will rightly deserve the moniker "Web 3.0". It will be providing a leap forward over non-web applications which will really be worth something, and hopefully won't be worthy of brutal derision at the hands of industry pundits.

Web 3.0 is going to come with a price, though. By way of an analogy: DOS developers didn't understand the value of Windows programming at first. They objected: Windows is slow. It's ugly. The UIs are boring and everything looks the same. It takes too much memory. Why couldn't they just use DOS like they used to?

Today's web frameworks are generally more like DOS than like Windows, in this respect. Don't get me wrong, I don't mean they're bad: DOS was a huge step up from its predecessor, which was, basically, "you have to write DOS yourself in every application". The "Web 2.0" frameworks I'm familiar with: Rails, Seaside, Subway, Django, and TurboGears are geared (no pun intended) towards providing those parts of your application that you'd have to write yourself, if they weren't provided. They are sometimes libraries, sometimes frameworks, but their purpose is to provide you with a quick path to development and make your application interactive. They aren't really geared towards providing a context in which your application can communicate with other applications.

The infrastructure Divmod is building, we hope, will provide some of the imposed structure that has thus far been lacking on the web. The first step, by way of Mantissa, and our recent work on the "Offering" system, is to provide a way for multiple installed applications to be easily installed on the same website, an idea that Zope, among others, pioneered with some success. That's the front-end, the Windows-1.0 phase. Later, we hope to build clustering and inter-application remote integration with Vertex, the "networking" and "multitasking" that came with Windows 95, following in that same metaphor.

Let's escape the metaphor for a second. The concrete features we want to provide will be things like: You want to write a social network application. Social networks implicitly use telephones and email. With Divmod's framework you can (in the future, when we have finished these systems) write an application as a set of plug-ins which will be activated by the Address Book page, the "Email Inbox" page, and the "Place Call" page from our VoIP application. You can automatically create To-Do items for the user's existing task-tracker. Similarly, if you provide appropriate plug-in interfaces (which the system does make pretty easy and idiomatic)

In the process of writing this essay, I discovered that wikipedia says others are already speculating about Web 3.0 and with similar ideas. It will be distributed, it will be decentralized, it will involve connections within and between web services.

So, how does this all relate to Divmod? What could we hope to gain by providing this kind of technology, especially by giving it all away for free? I'll explain the part of our business model that this impacts, because I am personally always suspicious of people telling me of great stuff they'll do for me without an idea of how it's helping them.

To start with, we have a lot of different ideas for applications, and we want to make sure they can all inter-operate. All the standard reasons for using open source apply - making bugs shallow with more eyeballs, etc - and we also want to make sure that as we're focusing on our first few uses we are preparing for a broader and broader use of our technology, and what better way to do that than by having other people with other use-cases take a look at it.

We have another reason too, though. Once we're done with all of our ideas for applications, (and by all that is good and holy in this world, one day we will be done!) we want to step back and allow the service to keep growing through the contributions of our users, and eventually compensate those contributors who provide us with useful code. Taking a page from Red Hat's book, rather than attempt to sell a commodity into a market, we want to define a market for "web 3.0" products, and then be the market leader and provide a marketplace for good products, rather than simply sell our own.

By providing a structure that will allow lots of different applications to be installed on the same site, we provide a way for independent developers to provide us with applications that can be run together on our site. We can then run all those applications through our account management system and make it easy for those developers to get compensated based on usage patterns and that sort of thing. Of course, any applications we're going to run on our cluster will have to be licensed as open source so that we can have the same expectation the developer does: we won't sue them for using our stuff, they won't sue us for using theirs.

In other words, we're trying to leverage this technology shift to make a way for hackers to get rich writing open source software, without going through the process of starting a startup.

4 comments:

ianbicking said...

If I understand what you are saying, I think it is similar to some of the goals I have for Paste as well. As such, it's really orthogonal to what something like TurboGears or Rails provides -- at least, what they successfully provide right now; I'm sure there are people developing these frameworks with motivation in that direction, but it's generally just a glimmer in the eye for them at this point. Zope certainly pioneered this in many ways, along with things like PHPNuke (or whatever fork came first, beats me), or ACS, or probably several other comprehensive CMSish products, especially the ones with "portals".

In Paste I'm trying to figure out how to build this sort of thing without a framework at all, to keep it orthogonal from the frameworks. I think this makes it somewhat different from other attempts. It means pushing things out of internal APIs and contracts (which are hard to keep track of, Zope being a good example), and into more naive and as a result somewhat more formal interfaces. Exchanging strings when possible, avoiding objects when possible, using HTTP and CGI semantics whenever applicable.

The result shouldn't have to compete with application frameworks, but instead tries to be complementary to them.

glyf said...

I think the goals might be superficially the same but Paste is going in a radically different direction from the, uh, "Divmod stack". The metaphor doesn't extend perfectly, after all Divmod is no Microsoft ;-).

Paste wants to work with existing code with minimal impact, to enable the existing paradigm to extend and be easier to configure. As far as I can tell it is designed to be as lightweight as it can possibly be, so feather-light it doesn't impact your application at all.

The specific example I mentioned - plugging in your social network application at particular points, in terms of actions that can be performed on a Person - puts the level of integration much higher. You've got a notion of a "Person" in our "OS", the notion of a "message" and an "email", etc. It comes with its own domain model and encourages you to plug in there rather than plugging in at the web layer.

Mantissa wants to be the heaviest possible thing that there is. So heavyweight that it defines not just the things that your application works with, but the ground that the application rests upon. The application becomes the lightweight thing, inverting the relationship that Paste sets up. Mantissa is a framework so heavyweight it has its own gravity well, and that's it's strength: by defining so many different features, it can provide a seamless UI for the user, to a large number of different apps. I hope it can enable an entirely new kind of application, "long-tail" type stuff where the actual application is simply a light set of workflow integration between a huge set of Divmod and 3rd-party infrastructure components. In this sense, things like a blog, SMS gateway, and a calendar are "infrastructure" because vertical market apps can plug in and make use of them.

There's also an object publishing layer in there somewhere, and in my humble opinion it's pretty good, but that space is so crowded right now I just have no interest in promoting it as something to use on its own. That hasn't stopped some people from grabbing it and using it by itself anyway, but that's not what I'm talking about.

Considering the fact that things like Sine (an internet PBX) plug in to Mantissa, I doubt that Mantissa is ever going to be on the application side of WSGI. However, it would be nice if it could take advantage of WSGI (and by proxy, of Paste) by providing a WSGI container. Pretty much all the necessary functionality is in the existing web2 WSGI container, although I think that considering Mantissa sets up a file space and a database for each user, there is probably some additional configuration information that could be passed along through the WSGI interface somehow.

WSGI+Web2 could enable something for Mantissa like the DOS box does for Windows (the metaphor might not extend perfectly, but it sure is a workhorse, isn't it?): a way to run a "regular" web application under the aegis of the Mantissa environment, perhaps even in a separate process, such that individual users use all the enhanced functionality that Mantissa, Quotient, and Sine have to offer, without sacrificing the ability to use the sorts of web applications that they are already working with.

glyf said...

My whole point is that Mantissa isn't very "2.0" at all. It's "3.0" ;-).

Although there are superficial similarities to a system like Plone, it's not a CMS. Its stock and trade is not "content", but interactive applications. One of the sample applications we have already implemented is a real-time isometric perspective roleplaying game ("Divmod Radical"). Unfortunately it's more of a tech demo than anything actually fun, but nevertheless it is not the sort of thing that Plone would be useful for.

I suppose the proof will be in the pudding, of course. clickchronicle is an interesting first attempt, but when our main app (Quotient) supports telephones, ssh, SyncML and Q2Q I think it will be clearer what is cool about this manner of integration.

As to your comments on WSGI and Mantissa, yes, that's about what I was thinking; although I think the interface will lean towards the narrow side as far as interface keys go; the real benefit of WSGI in this context is portability.

glyf said...

I didn't mention where you can read about Sine, so here's a link: Divmod Sine is a Python/Twisted/Mantissa-based voice-over-IP toolkit that supports standards such as SIP.