Category: General

It’s a bad night’s coding when your new app crashes the ruby VM … 🙁

It’s an actor-style flexible process manager. It’s great. Only it crashes the VM. Sigh, it’ll have to wait for another day.

deadlock 0xb7cb8484: sleep:-  - /usr/lib/ruby/1.8/monitor.rb:285
deadlock 0xb7cb8628: sleep:-  - /usr/lib/ruby/1.8/monitor.rb:285
deadlock 0xb7cdc708: sleep:J(0xb7cb8088) (main) - ./actor.rb:34
deadlock 0xb7cb8088: sleep:-  - /usr/lib/ruby/1.8/monitor.rb:285
deadlock 0xb7cb81f0: sleep:-  - /usr/lib/ruby/1.8/monitor.rb:285
deadlock 0xb7cb8308: sleep:-  - /usr/lib/ruby/1.8/monitor.rb:285
/usr/lib/ruby/1.8/monitor.rb:285: [BUG] Segmentation fault
ruby 1.8.6 (2007-06-07) [i486-linux]

Aborted (core dumped)

Hurrah, Erlang OTP R12B-0 has been released! The new version includes my fix to the http client. This is good news because it means that my erlang bindings for Amazon’s S3 storage service are now usable without having to patch OTP. Check out the retro distribution page for s3erl. Instructions in the README.

At the moment, the s3 library allow you to
– create, list and delete buckets
– read, list, delete and write objects

Things which I’d like to add if I get the time
– Improved error handling
– Support for EU-based S3 buckets. (not too hard)
– Support for streaming objects to/from disk (inets supports streaming to disk, but not from disk just now) (harder)

UPDATE:  You can now get the source from the public mercurial repository; “hg clone http://hg.nobugs.org/s3erl”.

Emacs is a journey, not a destination.
BAD: “\\(.*\\):\\(.*\\)”
GOOD: (rx (group (zero-or-more anything)) “:” (group (zero-or-more anything)))

This has been a pretty fun few weeks. I did a talk (“Agile at Amazon”) at an AgileScotland/ScotlandIS event in Glasgow. My main claim was that there are two vital ingredients for effective software project. Firstly, you must accept that during a project you will go through a learning process. You learn about technology, the business and the problem domain. Your development process needs to embrace this learning process, and make it easy (not painful) to change direction throughout the project as you learn new stuff. Secondly, software development is done by human beings, not robots. A good development process will accept this, and play to our human strengths whilst protecting us from our weaknesses. If you want a successful project, you need engineers who are enjoying their job, learning a range of relevant skills and working in a supportive environment. The rest of my talk described what a typical month at Amazon Scotland looks like.

Ezra Cooper kindly continued the talk-karma by agreeing to come along to Amazon and give us a talk on the work he’s doing on the Links project, It was a really good talk, and delivered well. I’m pleased to see that Ezra got a lot out of our questions during the talk. Thanks again, Ezra!

In other spoddery, I’ve been using the amazing Omniscient Debugger again. This time I’m trying it out on non-toy examples, and have been feeding bug reports back to Bil. If you want to see the debugger in action, check out the wonderful Bil delivering a talk at Daniel Friedman’s 60th birthday celebrations.

Finally, I’m excited to be going to a week-long course on the Foundation of Computing Science in a few weeks. I’ve done a fair amount of reading in this area in my spare time, but it’s great to have an opportunity for a bit of focused learning (and question asking!).

Capistrano (formerly switchtower) is the ‘ruby on rails’ way to deploy your code out onto your fleet of production/test servers. I’ve previously written my own ‘deploy’ script which had similar goals to capistrano, but I’ve only recently tried capistrano itself.

Having now used it, I think it makes a poor design decision. Capistrano runs a “svn co” operation on the deploy host itself. I think it’d be much better to grab a clean set of sources on your local development box, and then rsync/scp that to each deploy host.

The capistrano way is poor for the following reasons. Firstly, since I access my svn respository over ssh I need to have my private key on my deploy host. This is non-optimal for security reasons. I want to keep close tabs on my private keys, and I don’t want it living on a (potentially) compromisable public-facing host. Why not do all the svn stuff locally and keep your private key local?

But it gets worse. Your deployed rails app ends up (by default) being a ‘live’ checked out copy, complete with .svn directories. Additionally, the default rails setup unfortunately exposes the very top level .svn directory in your deployment. This leaks some information: nothing critical, but in security terms any leakage is bad. To find examples, look no further than the list of apps on the RoR site. Examples include 37signals [fixed] and penny arcade and strongspace and iconbuffer [fixed] etc.

Like I say, the information leaked in this way is not too critical. But, if you were into social engineering, knowing the hostname of their internal svn server in addition to login names for several developers could be just the info you need.

What’s the moral to this story? It’s the old lesson of minimal privileges. There’s no real need for the deployment hosts to have ssh access to the svn repository. Nor is there any need to have your deployment version be a ‘live’ checked out version of your source code. I think I will be sticking to a deployment method where I check out locally and rsync to the production hosts …

Update: This method also reveals the contents of some .htaccess files … append .svn/text-base/.htaccess.svn-base to your favourite rails app URL.