Comments on Deciphering Glyph: Making Twisted Specific

>> I was put off as I couldn't see where to start....

2009-10-08T16:48:04.254-07:00

>> I was put off as I couldn't see where to start.

Yup. I inherited an app and am constantly wondering why the heck anything was done anywhich way and, no, a finger tutorial doesn't do me any good. I haven't even thought about finger in more years than I care to count.

>> The opposite of this has to be the django project, within a short time in the tutorials you have something useful to work with.

Yes, except when you try to go beyond that. I've heard lots of "Ok, I did the tutorial and got the awesome polling, or blogger or whatever, now what?"

Twisted's problem is much bigger, though, because it's so wide that the documentation for hooking together the disparate pieces is sorely lacking.

I've also found a lot of the documentation and examples I've tried were of the "doesn't work with current versions" variety.

I'd love to help but all I could do is serve as a guinea pig for tutorials about the stuff I actually care about (mostly web client, ajax and, the comet client and server that I wish I could figure out how to use from Twisted but can't get off square one with).

For me it's not that I don't understand the programming model as it seems it is for many new users, it's the "how do I hook this to that and why do it that way" that I stumble over, if there's anything there doc-wise to even stumble on at all.

@VanL This is exactly what is needed, as a potent...

2009-09-28T19:14:48.722-07:00

@VanL This is exactly what is needed, as a potential twisted user, the shear amount of documentation is just boggling, when I last looked into it I was put off as I couldn't see where to start.

The opposite of this has to be the django project, within a short time in the tutorials you have something useful to work with.

The problem with Twisted is not technical, its soc...

2009-09-26T19:54:04.709-07:00

The problem with Twisted is not technical, its social. People don't know where to start.

If I may digress for a minute, I was talking recently with a friend about an interesting series of books that we both read. I was explaining how much I loved that the plot got larger as the books progressed.

My friend replied, "Yes, but what I liked is that the plot started out small. It started out with a few people you cared about, and then grew once you already knew the characters."

The problem with Twisted is that the plot doesn't start out small, with characters that you care about. Yes, I know about the finger server tutorial. The problem is that 1) finger servers aren't what people care about, and 2) its not clear how to progress from the finger server to the problem that I am trying to solve, today.

This post, for example, tells me that Twisted has lots of possible applications... but it doesn't tell me where to start. There is (in some ways) too much emphasis on the higher levels in the stack. If twisted is, at bottom, an event loop, then give me a way that I can start with just the event loop.

What is the most basic thing you can do with an event loop? Have some "container" of code, and have it get called when something happens. But I really don't know how to import a reactor, implement a container, and go.

If I were writing a tutorial on Twisted, I would do it as follows:

1. The big idea: Tell me when I need to do something.

In most applications, you spend a lot of time waiting. Or your code does, at least. You have a piece of code that does something important when a button is pushed, when a network request comes, or when there is new data to process.

Twisted is plumbing. It allows you to write that one thing that is important to your application, and then it makes sure that your code is called at the right time.

For example:
class http_server()....
[Blocking, simple server. Show that there is a bunch of looping/waiting going on - and the code for that dominates returning the "Hello World" page.]

Wouldn't it be nice to have this?

class http_server()...
[Simple as possible. have it receive a request, return the hello world page.]

The difference between #1 and #2 is that in #2, we decided to let twisted take care of all the plumbing.

2. Blocks of code.

[The point of this section is to show the simplest possible code container that does something. It is a glorified function: it gets inputs, it does stuff, it returns outputs. You can steal - and simplify - the inbox/outbox concept of Kamaelia. Something comes in, you handle it, you send it on. Use something like corotwine to do it blocking, first. Then show that you can trivially adjust it to do things in nonblocking way... and that is the point! No more waiting. Your code is called when it is needed.]

3. Lets build it up, add some options...

Now, start adding options, factoring, etc.

If I had this tutorial, I would know a lot easier where to start with Twisted.

> That's nowhere near sufficient to make the Twist...

2009-09-24T16:59:32.210-07:00

> That's nowhere near sufficient to make the Twisted HTTP experience more about my application than your framework. But it's a good starting point, for some specialist who truly cared about the user experience.

This is probably another documentation issue. For writing a straight up HTTP app, it's very likely you'll never have to import the reactor. Lots of the web examples do it anyway, though. Improving the documentation so that there is an excellent new-user experience is definitely one of the kinds of polish that would make Twisted Web a much more successful competitor to the other projects out there. Now, to find someone to do that work... :)

Glyph said "...In order to sprout and take root, t...

2009-09-24T15:57:21.656-07:00

Glyph said "...In order to sprout and take root, those seeds each need a dedicated advocate, someone who cares deeply about the experience of a user or administrator who just wants Twisted to perform one particular function and doesn't want to write their own application code to make it do that."

Let's take HTTP as an example of "one particular function". Learning how Twisted can do DNS or chat or SMTP doesn't help me write an HTTP application; instead, it gets in the way. Any abstractions which serve the Twisted implementation over my user experience as an author of an HTTP application get in the way. Someone who cared deeply about my experience as a user would work to let me import everything I needed from a web-specific package, so I wouldn't have to remember that the Site class is in twisted.web but the reactor is in twisted.internet--that's a layer of abstraction which I as an HTTP application author shouldn't have to deal with.

Now, you *could* make a new specialized layer on top of twisted which did that, but IMO you don't need to; just allow the reactor to be importable from the twisted.web namespace. Ditto for the Site class, actually, since it's almost universally necessary.

That's nowhere near sufficient to make the Twisted HTTP experience more about my application than your framework. But it's a good starting point, for some specialist who truly cared about the user experience.

It's not entirely obvious to me how minor API chan...

2009-09-24T13:36:52.401-07:00

It's not entirely obvious to me how minor API changes achieve the goal of recruiting specialists to polish different areas of the project. I'm all for improving the clarity of APIs, but is this point really related to this post?

Erm... you just want the name to be different, or ...

2009-09-24T13:31:41.429-07:00

Erm... you just want the name to be different, or what? the HTTP library does not seem, at first blush, to be a reasonable place to place the global reference to the event loop.

from twisted.web import reactor ...would be a nic...

2009-09-24T10:34:08.647-07:00

from twisted.web import reactor

...would be a nice start.

Screw that. I want to be head of the secret polic...

2009-09-24T08:45:34.119-07:00

Screw that. I want to be head of the secret police. I'll root out threaded application developers wherever they may lie.