He's reading too much into this. At loads this low, it can't be a language design issue. There's probably just a bug in Mzscheme.
Hard to say whether this post is deliberate linkbait or just overeagerness to see this as evidence confirming a pre-existing bias. Probably a little of both.
But you say it likes it's a bad thing. Do you think I don't have a valid point? Are you not debugging a memory leak in a production Lisp app?
With Lisp you can put together a site in a few hours. But how many days must later be spent debugging issues like this? Will this time spent debugging be counted against your productivity gains you attribute to Lisp?
This is exactly the sort of problem Erlang has nailed down. And it requires a whole new thinking about program structure. Is it possible you don't see the problems with Lisp because you can't imagine there is a fundamentally better way?
You can't possibly have a valid point, because you're starting from practically zero data. All I have is vague theories about what might be wrong, and what you know about what's going on in our software is a proper subset of what I know.
Why don't you wait till you have more data before wasting all our time with stuff like this? What's the rush?
> Why don't you wait till you have more data before wasting all our time with stuff like this? What's the rush?
I used the opportunity of your reliability failure to make a larger point about reliable software. I admit was deliberately provocative and terse in the posting, but I have futher clarified things in the comments. I thought about writing a long, detailed, well balance article about this. But that wasn't going to happen, I don't have the time. But then I thought why not just write what I think, but not everything I think? So I did.
Sorry, I might be completely wrong about the source of your problems, I am definitely theorizing as to what you are seeing and I don't think I've implied otherwise. But I will say these are the sorts of things that crop up again and again in Java and other GC'd mutable state languages.
A lot of the buzz around Lisp web applications does relate to mutable state accomplished through continuations, but that's not a requisite to using Lisp, nor is it the only advantage to using Lisp.
I manage state via relational databases, and haven't had to deal with any memory-leak bugs. I even found it easy to prevent leaking of database connections.
Mutation is definitely optional. A lot of people even still count Lisp dialects among the functional languages.
"I used the opportunity of your reliability failure to make a larger point about reliable software."
Your argument construction is not very reliable either. You're basing "large points about reliable software" on... on what exactly? You can't overgeneralize like that, at least not while remaining convincing.
It is very possible that Erlang could have a better way than Lisp, and that Lispers can't see it until they learn Erlang. Entirely possible.
But an arbitrary bug in an arbitrary Lisp program is nothing even remotely like sufficient proof for such a statement, unless you are trying to claim that Erlang programs are inherintly bug-free, regardless of how they were implemented...?
> Are you not debugging a memory leak in a production Lisp app?
But, you're confusing the actual application with the Lisp implementation. Arc (and Mzscheme for that matter) are not as mature as Erlang's 20 years (or whatever). If this was running on a more mature Lisp system, I don't think we'd be discussing the same thing.
To be honest, this is not the only such case. Eve Online folks migrated part of their system from SBCL to Erlang for seemingly the same reasons [http://lambda-the-ultimate.org/node/2102].
Having said that, it is still not a language issue, since we have systems such as Termite/Gambit Scheme [http://toute.ca].
You don't get to pick and choose implementations to compare. Mzscheme was probably chosen because it's a pretty well known scheme implementation and seems popular... likely because it's well done in many ways. But you have to take the bad with the good.
> But how many days must later be spent debugging issues like this? Will this time spent debugging be counted against your productivity gains you attribute to Lisp? This is exactly the sort of problem Erlang has nailed down.
"So the Erlang VM died suddenly with a failed memory allocation. This is actually quite okay [...] However, the Erlang VM didn't restart automatically. Apparently we need to configure something for that to happen, only we can't figure out how to make it work. The documentation is either wrong, confusing or the feature is buggy. This is the kind of stuff we keep hitting in Erlang. It's fantastically productive for many tasks, but using some of the built-in libraries and features can be a huge time sink. (inets and xmerl immediately spring to mind)."
Yes, those things need to be pointed out. I don't want people to think that Erlang is a miracle productivly booster. It's not. The productivity gains are only there in certain application domains, and of course it also require good libraries to support it.
But Erlang to me isn't as much about programmer productivity as it is about reliability. But when building highly reliable systems, of course it's a big productivity boost.
I should point out we've abandoned xmerl (and xml altogether), and things have been much simpler since. We are in the process of moving off the inets http server to mochiweb. Hopefully that will also go smoothly.
It's sort of linkbait, and it'd be more interesting if he had something more substantive to say... however, Erlang certainly is adept at being a robust, stable server, and has had a lot of time and brain power poured into those efforts, so I think they have a right to be proud. It'd be better to find another way of saying it, though.
Tangent: In your troubleshooting, have you already made sure you aren't occasionally receiving a malformed request that is causing the server to puke? In other server software that has displayed that sort of behavior, I've seen that be the problem more often than a traffic issue. I mean; you would have noticed a memory leak type of situation by now, I'd imagine.
I poked around some in news.arc from the download on arclanguage.org; it looks like the newscache macro does quite a lot of heavy lifting... I recall reading someone having some weird issue with the way w/uniq's gensym works (vs. mzscheme's) - is it possible some change has been made to that functionality in the past month or two?
Unless of course Patrick is generating requests that perform functionality, like logging in, up/down voting etc... Then the idea is totally moot since you'd be hitting a much larger cross-section of functionality.
I don't really know Arc and it's intimidating making suggestions knowing the caliber of people looking at the issue; but I figured sometimes an opinion from someone who isn't intimately familiar with the code can be at least a little helpful. If not, feel free to ignore this post completely =D
have you tried the latest mzscheme? i remember you were running arc on v352, the libraries have changed a lot since then, so maybe the gc did too... iirc there's a patch on anarki or someplace to make it work.
Hard to say whether this post is deliberate linkbait or just overeagerness to see this as evidence confirming a pre-existing bias. Probably a little of both.