I love a good snark as much as the next person - some might say more - but it really bothers me when people make snide comments denigrating others' free work without at least offering a cogent criticism to go with it, and I don't want to be that guy. So, hopefully before the whole Clojure community finds said tweet and writes me off as an arrogant Python bigot, I would like to explain what I meant in a bit more detail.
Right off the bat I should say that this was a bit tongue-in-cheek. I actually rather like Clojure and I think Rich Hickey has some very compelling ideas about programming in general. I watch his talk "Simple Made Easy" once every month or two, contemplating its deeper meaning, and I still usually come away with an insight or two.
I should also make it clear that I was linking to the
recur special form specifically, and not just the special forms documentation in general. Obviously having reference docs isn't a bad thing.Ivan, (or should I say, "@radian"?) you may be right; that documentation you linked to may indeed one day spell Python's doom. If Python does eventually start to suck, it will be because it collapsed under the weight of its own weird edge cases like the slightly-special behavior of operator dispatch as compared to method dispatch, all the funkiness of descriptors, context managers, decorators, metaclasses, et cetera.
A portion of my point that was serious, though. The documentation for recur does highlight some problems with Clojure that the Python docs can play an interesting counterpoint to.
The presence of the recur form at all is an indication of the unhealthy level of obsession that all LISPs have with recursion. We get it: functions are cool. You can call them. They can call themselves. Every other genre of language manages to use this to a reasonable degree of moderation without adding extra syntactic features to their core just so you can recurse forever without worrying about stack resources. Reading this particular snipped of documentation, I can almost hear Rich Hickey cackling as he wrote it, having just crowned himself God-Emperor of the smug LISP weenies, as he gleefully points out that Scheme has it wrong and the CL specifications had it wrong with respect to tail call elimination, and that it should be supported by the language but also be explicit and compiler-verified.
The sad thing is, my hypothetical caricature of Clojure's inventor is actually right! This is a uniquely clever solution to a particularly thorny conceptual problem with the recursive expression of algorithms. The Scheme folks and the Common Lisp folks did both kinda get it wrong. But the fact that this has to be so front-and-center in the language is a problem. Most algorithms shouldn't be expressed recursively; it is actually quite tricky to communicate about recursive code, and anyway most systems that really benefit from it have to be mutually, dynamically reentrant anyway and won't be helped by tail call elimination. (My favorite example of this is still the unholy shenanigans that Deferreds have to get up to to make sure you don't have to care about callback chain length or Deferred return nesting depth.)
Also, if you want to be all automatically parallelizable and web scale and "cloud"-y, recursion and iteration are both the wrong way to do it; they're both just ways of tediously making your way down a list of things one element at a time. What you want to do is to declaratively apply a computation to your data in such a way as to avoid saying anything about the order things have to happen in. To put it more LISPily, (map) is a better conceptual foundation for the future than (loop) or (apply). Of course you can do the naive implementation of (map) with (recur), but smarter implementations need application code to be written some other way.
The language style choices of the manual in this case is also telling. The Python docs that Ivan linked to go into excruciating detail, rephrasing and explaining the same concept in a few different ways, linking to other required concepts in depth so the reader can easily familiarize themselves with any prerequisites, while still essentially explaining a nerdy part of the language that you can ignore while still using it productively. Every Python programmer ignores descriptors while they're learning to write classes and methods, despite that they're using them all the time; this ability to be understood at different levels of complexity is a strength of every good language, and python does particularly well in that regard. Of course one could also make the case that this is just because Python has so many dusty corners hidden behind double-underscores, and a better language would just have less obscure junk in it, not make understanding the obscure junk optional, but I digress.
The description of recur, by contrast, is deeply flawed. It is terse, to a fault. It introduces the concept of "recursion points" without linking to any kind of general reference. It uses abbreviations all over the place ("exprs", "params", "args", "seq") without even using typesetting to illuminate whether they are using a general term or a specific variable name.
But, by far, the worst sin of this document is the use of the words "variadic" and "arity". There is really no excuse for the use of these words, ever. Take it from me, I am exactly the kind of pedantic jerk who will drop "arity" into a conversation about, for example, music theory, just to demonstrate that I can, and as that kind of jerk I can tell you with certainty: I have no excuse.
It should say: "a function that takes a variable number of arguments". Or possibly: "it must take exactly the same number of arguments as the recursion point".
This was particularly disappointing example to me because Clojure strikes me as a particularly... for lack of a better word, "optimistic" lisp, one that looks forward rather than back, one that is more interested in helping people find comprehensible and composeable ways to express programs than in revisiting obscure debates over reader macros or lisp-1/lisp-2 holy wars. But the tone of the documentation for recur aims it straight at the smug lisp weenie crowd.
As I hope is obvious, if not initially, then at least by now, I don't think that Clojure will fail (or succeed) on the merits of one crummy piece of documentation. It's a much younger language than Python, so it may have a ways to go in its documentation practices. It also comes from an illustrious heritage that I can't expect to see none of in the way that it talks about itself, no matter how unfortunate certain details of that heritage are. Heck, at the parallel point in Python's lifetime, it didn't even have descriptors yet, let alone the documentation for them!
Still, I don't think that this issue is entirely trivial, and I hope that the maintainers for the documentation for Clojure, at least the documentation for the parts of the language you have to see every day, take care to improve its accessibility to the less arcane among us.
Posts
8 comments:
I'm not a big clojure user, but I do have one suggestion: it might be that the recur documentation is terse and confusing in order to manage the political concers of growing a language community.
The recur documentation looks terse and confusing to a normal person but it also would not be out of place in the R6RS scheme standard or the CL hperspec. Despite the profusion of lisp implementations (or maybe because of?), lisp hackers have always had to spend time reading the damn standards instead of actual made-for-humans documentation. And the standards look like that (especially the CL standard -- the scheme folks are better at writing).
Clojure has to appeal to people who never touched a lisp before but have experience with python/ruby/java/perl. But it also wants to appeal to lisp people. And one way to appeal to those people is to offer a bit of the random crap documentation that they're used to, that they've come to expect. The terseness is both familiar and requires extra thinking; that element of struggle reassures prospective clojure writers from the lisp world that switching to clojure won't devalue their investment in lisp, that there is too stuff in this clojure thing that only they will get.
--msalib
"recur" is definitely a wart.
The shape of simple programs written in dialects of ML is pretty much identical to that of Scheme programs - named lets, explicit accumulation, consing and reversing - but, as you hinted, that's pretty much never how application-level code is written, it's how SRFI-1 procedures like (map) and (fold-left) are implemented.
I'm not sure how the bit about "extra syntactic features" is applicable to Lisps in general. Clojure is really the only Lisp anyone's talking about which decided to turn recursion into syntax.
I'd imagine most people would get on board with the suggestion that recursive functions are difficult to reason about, but if you accept that a recursive call in the tail position is not recursion in any meaningful sense (I'm not talking about frame allocation - it really doesn't feel like recursion as it's written, it just feels like a natural way to iterate without necessarily mutating state), I think you'd find that incidences of nested/tree/linear recursion aren't any more common in Lisp programs than those written in other languages.
I really wanted to get into Clojure - Lisp is fun, but what's no fun is feeling guilty about basically committing to writing everything from scratch when you decide to build something in it (because your pet implementation only has six users, or an unhealthy ecosystem or whatever). This is coming from someone who hates CL (rplaca, in a word), and thinks Scheme would be pretty much perfect if it thought a little bit harder about extensibility/dispatch with regard to types - (string=)/(char-set=), (length)/(string-length), etc. - garbage. Oh, and if it had standardized a module system way back when, but I guess everyone figured Scheme programs would always be written on chalkboards, instead of in text editors.
Anyway, I thought Clojure could be a great environment for writing massively concurrent programs, given some of the language features and the j.u.concurrent integration: I decided my "Hello World" was going to be an asynchronous echo server.
Unfortunately, aside from a couple of unmaintained, bitrotten Github projects slouching in the direction of Netty integration, I was basically going to have to copy & paste a bunch of Netty boilerplate and stick in some extra parens if I didn't want to spawn a thread per connection.
That fact alone seemed kind of suspicious - either the community isn't sharing, or everybody really likes threads, or I missed a well-designed NIO wrapper with a more Lispy API. I'd have thought the kind of people who'd be attracted to Clojure would be all over something like that.
Man, if Dylan had Python's module system, was a little stricter than Smalltalk about what can go where, and treated whitespace significantly (instead of a bunch of "end"s), that'd be a hell of a language (or, put another way, if there was no distinction between statements & expressions in Python, and it included define-syntax). If this dialect of Dylan compiled to Python bytecode, so it could actually do real things, that'd be awesome. As soon as I get someone to agree that it's a good idea, I'm going to start writing it. Anyone? It'd be slow as hell unless it copped some shared-structure magic.
I hope some of that made sense; running low on sleep.
You might want to check out http://clojuredocs.org/quickref/Clojure%20Core before passing judgment on Clojure documentation. It's clean, easy to navigate, and gives lots of great examples how to use the functions.
And regarding the recursion rant, I'd like to point out that the whole point of a functional language is that you don't write explicit loops whenever possible. Clojure provides a very rich library of [3] higher order iteration functions which you should be using. Loops are low level constructs you'd generally use as a form of optimization.
@Yogthos,
While I have some sympathy for the "loops are low level" argument, recursion is also low level, but harder to think about.
And like I said in the post, I realize that this function isn't all there is to Clojure's documentation.
Moe,
Would you consider aleph bitrotten? It looks pretty well-maintained to me (changes as of 2 days ago) although I would make no representations as to its quality...
Glyph - no, not bit-rotten. Thanks for pointing that out. I'm fairly certain I looked at it (although I may be wrong - Lamina, one of its dependencies, is a library written by the same guy. It's interesting enough that I would have remembered it) and I'm wondering why I didn't write any code against it - it actually looks like a pretty good library.
As for the recursion thing, I'll try it one last time: Everybody hates recursion. It's hard to think about. It requires an uncomfortable amount of computation for a human being to figure out what's going on in the simplest of tree-recursive functions (for me, at least). I don't think Lisp nerds like frowning over this kind of stuff more than anyone else.
That thing where a function returns the value of invoking itself is called recursion, but doesn't come with any of the baggage associated with the word. Have you written many tail recursive functions which didn't mutate state? I'm guessing you've written your share, but I've a hard time believing you found them difficult to reason about. I'd rather be thinking about a very involved named let which doesn't include any exclamation marks than a trivial for loop which manipulates mutable objects.
Have you seen http://clojure.com/blog/2012/05/08/reducers-a-library-and-model-for-collection-processing.html ?
I believe it addresses your concerns.
Have you seen http://clojure.com/blog/2012/05/08/reducers-a-library-and-model-for-collection-processing.html ?
I believe it addresses your concerns.
Post a Comment