Since I'm now syndicated on Planet Twisted, I suppose I had better say the occasional relevant thing.
In addition to my main presentation (on the Divmod-developed technology, ATOP) I've got some ideas for either lightning talks or open-space sessions (depending on level of interest) I might do in dull moments at PyCon. Does anyone want to hear one of these?
- The Long Hard Road to Zope
An exploration, with heavy audience participation, of why everybody writing sufficiently complicated code in Python ends up rewriting every major aspect of Zope one at a time (object database, component model, asynchronous networking base, thread pool, web templating language, not necessarily in that order) but Zope itself is difficult to use and so routinely critcized by the community despite also being widely loved and touted as an example of Python's power. I'd love it if this could include some feedback from the Zope folks about how Zope can be integrated into these ad-hoc environments when you "really need the whole thing", how to tell when you really need the whole thing and when a partial solution might be more appropriate. - Disassembling Twisted
I want to break Twisted into lots of different pieces. Currently there is a huge amount of coupling between packages: if someone makes a problematic, compatibility-breaking change in CVS head, it doesn't matter if that change is in a horribly obscure place, it blocks a whole release, bumps a major version number and prevents all packages from being released until it can be tested on all platforms, fixed, stabilized, etc. I believe that the Python core has similar problems (and possibly more ideas for solutions) with its increasingly sprawling standard library and extremely slow major-version release cycle. On the other hand, neither the Twisted or Python Core developer communities are fairly small, so we need to be careful not to burn resources in complicated release procedures. Also, the "batteries included" philosophy has helped both Twisted and Python quite a bit, so we need to explore separating releaseable chunks and still having a large, integrated release which makes it easy to assume that lots of functionality will be available if Twisted is installed without mixing and matching a bunch of little pieces. - Network Hostility Management
How hard is it to build a system which would send arbitrary files from any network-connected computer to any other on the internet? How much help do you need from servers on the "real" internet? Most importantly, to what extent can you hide all the nasty details from the user? I would like to outline and discuss a file-transfer tool which can defeat as many kinds of firewalls as reasonably possible without being a security hole itself that firewall admins will attempt to close off. - Quotient Infrastructure
A discussion of the "big picture" when it comes to Atop, Nevow, and Quotient's app-server-like components, for those of you who have attended both DP's and my presentations. Discussion (and hopefully delegation) of separating application and infrastructure. - Component Model Experimentation
We're still tweaking the semantics of Twisted's component model, but obviously this has the potential to break a good deal of client code if we release changes. How can we map the core set of semantics that most people currently care about onto a potentially changing back-end? - Twisted Imagination
Either this is your life's true purpose or you just don't care. I would like to talk about maybe, you know, writing a working game this decade.
I can think of a few more ideas, but this is probably enough for at least two conferences. Thoughts?
Posts
7 comments:
I'd listen to 2, 3, and 6. But I won't be there, so it's a moot point!
(or maybe I will! It's right after Spring Break, so I doubt anything important would happen then...)
You will attend. Agents are en route to your place of residence to re-install the obedience chip.
Why have we had so many malfunctions this year?
(Bah, I had to go look up my old LJ user info to post this comment. Maybe if you re-enabled Anonymous posting you wouldn't have to bug us to post comments here :)
I want to get as much talking/hacking done as possible on Imagination at Pycon, so #6 is my highest preference :-)
#2 definitely needs talking about, but I think you're trying to make too fundamental a change too quickly. I'm down with gradually breaking off chunks of Twisted (.web and .lore being the first to go), after massively improving the release process. I think you're way overestimating the problem, though. For example, I *don't* bump major version numbers when incompatible changes in obscure Twisted packages take place, and I don't see a reason to. I especially think you're overestimating the problem beyond requiring a massively improved release process for our monolithic package, which is quite possible. Anyway, this isn't the forum to talk about this. Later :-)
#5 is my next preference. I *should* be more involved in this, but I'm getting kind of lethargic about that subject.
#3 I don't understand at all. What is the goal of your file-transfer thing? Is it push or pull?
#2 and #4 don't interest me.
--radix
I assume you mean #1 and #4 don't interest you.
#3 isn't about "push" or "pull" - it's about pasting a URL in your chat program of choice that, when clicked on, will *actually* retrieve the bytes that it refers to, regardless of intermediary NAT or firewalling.
#2 needs to be done. I would also like to see some of Twisted relicensed so that it could get filtered into Python 3000 or whatever. Personally, I've found uses for twisted.python.* and some of twisted.internet (like Deferred) outside of the context of Twisted entirely. In these cases, I've just ended up rewriting minimal (in the case of Failure, I didn't need all the state chucking) or different implementations (in the case of Deferred, I needed higher performance and less recursion)
#3 would be useful, but I think it's relatively obvious what the barriers are and how to get around them. At some point (NAT vs. NAT) you need a third party that doesn't have the firewall problem who is willing to give up their bandwidth to facilitate the transfer. I imagine it hasn't really been done yet because these third parties would likely want some sort of compensation for their bandwidth.
#5 how about just giving up on the component model and picking up the model from Zope or PyProtocols? Personally, I like PyProtocols. It even has a PEP, so it would have a decent chance of making Python 2.4 (or later) if Twisted migrates to it and enough fanfare is made.
#2: yes, yes, and yes. probably some of your requirements (especially wrt performance) should be evaluated for refactoring the core since they are quite common.
#3: I don't think it's obvious. NAT vs. NAT is actually an easier case to get around: by taking advantage of stateful UDP port allocation, most routers can actually do nat to nat filetransfer with only minimal help from the outside. Look at how SIP phones do nat to nat RTP. Harder is getting someone who can statically map port 80 for you behind a draconian corporate firewall, or to figure out if you are behind such a firewall. The real trick is to explain how to combine all of these techniques to make it automatic and to make error reporting sane in failure cases instead of just insanely long ping timeouts.
#5: Performance and aesthetics? Maybe those are things to discuss, but component lookup speed is already a major bottleneck in several programs that I've written, and Twisted's components are vastly simpler and faster than either PyProtocols or Zope3, at least in the cursory testing I've done. Also I _REALLY_ like the IWhatever(x) syntax in Twisted. Does either of those have a convenience like that yet? Either looking at doing an optimized C adapter lookup table, or stateless adapters?
#5: PyProtocols has an optimized C adapter lookup chain. PyProtocols *can* do IWhatever(x) syntax, it even has a compatibility layer with the current Twisted system. The big thing to note is that if you have an adapter chain from A -> B and B -> C, PyProtocols will let you adapt(A, C). If a more direct adapter is available, it will use that instead (it uses weights to choose the best one, and it does not have a problem with cycles). As far as stateless adapters go, it probably can do that too, but I don't know off the top of my head.
Post a Comment