Ninjas on a Penny Farthing

  posted  
Use Twitter? Follow on Twitter.

Ruby Summer of Code Wrap Up


In late May, 2010 I started the Ruby Summer of Code. The premise was simple but in hindsight much more complex than expected – Make deployment and Continuous Integration against multiple rubies simpler (e.g. as simple as RVM was on the desktop). Since then, I’ve worked for 2-3 months and very recently finished my project. In order to wrap it all up, I figured it was about time I blogged about what I did as well as what my experience was like. It’s important to note early on (and I’ll expand later) that I was very lucky that my primary mentor, Wayne Seguin, not only pushed for my project to get accepted but has also been an immense help the whole way through – It would of been almost impossible for me to get as much done as I did get done otherwise.

In the given the length of time, even though I didn’t manage to complete everything I’d hoped to, I’ve done my very best to fulfil my primary goals (I’ll let you decide for yourself) with a bunch of little bits adding up to make my goals easier. With that said and done, I now present to you part one through five of my work on Ruby Summer of Code.

Part 1. RVM itself

The biggest thing I worked on during RubySOC was RVM itself. With Wayne’s guidance, most of my time (especially for the first month and a half) was spent clearing out as much of the pivotal tracker as possible. This work eventually led in part to the recent RVM 1.0 release. On top of the actual development, I put in as much time as possible manning the #rvm chat room. Doing so gave me an even greater appreciation for the superb work Wayne does and the way he runs his community.

In terms of features contributed, since starting near the end of May, the following have been added (many are still undocumented whilst I attempt to get some free time post-RubySOC whilst juggling university, a job, OSS work like RVM as well as work for a local not-for-proft, Youth Tree). I’ve also implemented better passenger / capistrano support and a few other things which I suggest checking out my archives for more information about.

Trusted rvmrc support

As one of the newest updates, this was literally implemented in the hours before the release of 1.0. Namely, shortly before we were ready to release 1.0, Wayne was contacted about a potential security risk and hence we did our best to implement support for managing rvm trust. For more information, make sure you run rvm help rvmrc on rvm HEAD which contains some pointers about manipulating rvmrc trust as well as reading Wayne’s RVM 1.0.0 blog post.

Better Set Operations

Again, for more detail, check out Wayne’s post. Essentially, I implemented a new --with-rubies to option to let you pass ruby strings (e.g. all-rubies) to rvm set operations to have more granular control over the set of rubies it runs a given operation against.

If you haven’t used them before, set operations let you do things like run tests, rake and other similar tools against a subset of all installed rubies / gemsets.

A bunch of new rvm tools

On top of all of this, I worked on implementing a bunch of tools to perform general maintenance / scripting tasks. Since they’re so many in number, I’ll just give a summary here but I suggest looking into them on your local install – In most cases, when called without arguments they will show usage information.

  • rvm rvmrc – tools for manipulating rvmrc files / rvmrc trust.
  • rvm cleanup – tools to cleanup your rvm install / reclaim temporary disk space used by rvm.
  • rvm snapshot – tools to save a snapshot of your rvm install (e.g. ruby names, gemsets etc).
  • rvm repair – tools for common repair operations e.g. regenerating environment files.
  • rvm wrapper – tools to generate wrapper executables.
  • rvm env – tools to get data about the environment.
  • rvm tools – tools to get the current ruby string identifier, the identifier for a given path and similar things.
  • rvm docs – tools go generate rdoc and ri documentation.
  • rvm disk-usage – tools to get an overview of the amount of space rvm is using.
  • rvm rubygems – tools to help you change the rubygems version for a 1.8-compatible ruby.
  • rvm migrate – migrate a given rubies gemsets to another ruby (see below).
  • rvm upgrade – upgrade a ruby, see below.

tar.bz2 out of the box

A small but helpful change – We switched all MRI rubies (those from ruby-lang.org) to use tar.bz2 archives instead of tar.gz archives – the result being even smaller downloads for common rubies, making your rvm installs slightly faster.

Patch Support

One of the handiest features if you intend to patch your interpreter, rvm now implements a --patch flag which lets you pass in either the absolute path to a patch, the name of a patchset or the name of patch to be applied when installing a ruby.

For named patchsets, it looks up your interpreter directory tree (e.g. for ruby-1.8.7-p302, it maps to the directory ruby/1.8.7/p302, starting under ~/.rvm/patchsets) to search each directory for the named patchset. I’ll be blogging more about this in the coming weeks, but all you need to know now is patchsets are like gemsets – a way to define a set of common patchs to be applied to a ruby.

Named patches are looked up in a similar manner but start under ~/.rvm/patches. As an example, rvm ships with a readline-fix patch which patchs ree and 1.8.7 versions less than p299. When called as such:

rvm install ree --patch readline-fix

And readline 6.0, it will make it correctly respond to Control-C instantaneously (vs. waiting for the enter key). RVM in this case will basically look for the readline-fix.diff file in:

  • ~/.rvm/patches/ree/1.8.7/2010.02/readline-fix.diff
  • ~/.rvm/patches/ree/1.8.7/readline-fix.diff
  • ~/.rvm/patches/ree/readline-fix.diff
  • ~/.rvm/patches/readline-fix.diff

In this case, it’ll find the patch in ~/.rvm/patches/ree/1.8.7/readline-fix.diff, applying it to the code before installation.

Named Rubies

Named rubies let you install multiple versions of a given ruby along side each other. E.g. when you want a version of 1.8.7 that is stock along side one that has a patch you’re working on. To use it, you’d so something along the lines of:

# Install the default 1.8.7
rvm install 1.8.7
# Install with the path to a patch
rvm install 1.8.7 --name gc --patch ./gc-patch.diff
# Use your normal ruby
rvm use 1.8.7
# Use the ruby version with your patch
rvm use 1.8.7-ngc

This features not only makes it easier for users to test patches along side their normal installs but it is also designed to make it trivial to try out different compile options (e.g. one with readline 5, one with readline 6) and to then compare the results side by side.

rvm migrate and rvm upgrade

rvm migrate moves all of the gemsets from one ruby to another. E.g., rvm migrate 1.9.2-rc2 1.9.2-p0 will move the gemsets from your install of 1.9.2 rc2 to 1.9.2-p0.

Taking it even further, rvm upgrade is similar to rvm migrate but it also installs the ruby first. As an example, a user may run:

rvm upgrade 1.9.2

With only 1.9.2-rc2 installed, and it will automatically: 1. Download and Install 1.9.2-p0 2. Migrate the Gemsets across 3. Update references e.g. in aliases, wrappers and the like.

Ideally, the idea is that it will make it easier users to switch out rubies when a new release is made available (such as the rc2 to p0 example above).

Part 2.The RVM Ruby API

Whilst Wayne was away for a week or so, I had the opportunity to implement a ruby wrapper around the bash scripting api. I did my best to implement a well structured library that wraps most of the current rvm api, in both a raw wrapper fashion as well as a more ruby-like style. One of the best examples of this in practice is my recent rvm + passenger blog post which uses the ruby api to switch the gemset for a current ruby.

I’ve attempted to document it all as much as possible so if you’re interested at all, make sure you check out the documentation on rubydoc.info. To give you an idea of what using it is like, using the ruby api you can script the following:

# Load it. Other option is to install the rvm gem (keep your normal
# rvm install) and then load it.
$:.unshift File.expand_path("~/.rvm/lib") # or your rvm dir
require 'rvm'

# Example 1. Switch gemset with current ruby:
p Gem.path
RVM.gemset.use! "rails3"
p Gem.path

# Example 2. Show information for the current ruby.
puts YAML.dump({
  :ruby    => RVM.tools.identifier.split("@").first,
  :gemsets => RVM.gemset.all,
})

# Example 3. Show the ruby string for a given dir:
p RVM.tools.path_identifier("~/Code/Big-Help-Mob")

# Example 4. Get the default ruby
p RVM.aliases.show("default")

# Example 5. Run something as ree.
RVM.environment("1.9.2") do |e|
  p e.tools.identifier
  p e.gemset.all
end

Part 3. CI tools

For the CI-specific part of my RubySOC project, Wayne and I settled on using the fantastic Hudson CI application. I’ve since released three forked versions of the ruby-related plugins for hudson and going forward I’ll be working with the maintainers of the existing plugins to make them more rvm friendly.

If you’d like more information on this part, I suggest checking out my previous post on rvm and hudson integration.

Part 4. Misc. Documentation

Other than actual code, I’ve spent a lot of time work on various parts of the rvm site such as cleaning up the existing integration pages. There is still more to be done post RubySOC (given the rapidly changing state of things) but when it comes down to it, we’re working hard to try and not only help the docs catch up to rvm itself but to also make them better in general.

As part of this, I’ve been working on documenting things (such as the passenger setup) in blog post form – These posts will shortly be integrated into the official documentation. We hope that this will make it easier to find longer / more structured tutorials for new users.

As a side note, if you do have spare time and wish to contribute, we always welcome more love for the rvm-site – It’s easier to get started with than rvm itself whilst still having a direct effect on many of our users. To get started, simply take a look at the rvm-site repository on GitHub.

Part 5. RVM-based CI Virtual Appliance.

The final thing I worked on (and that is mostly complete, but still has a few bugs) was an Ubuntu-based Virtual Appliance that makes it super simple for any user to get started with rvm and hudson. I’m still doing a few tweaks to this stage (which is exceedingly hard given none of my machines at home support the virtualisation software I need) but I’ll be publishing this next week some time when I get a chance to work on it at work.

My thoughts on the program.

In the few months that I’ve been working on RubySOC, I’ve come to just how rare an opportunity it as – one all passionate students should attempt to pursue at least once. Apart from the fact you get paid to work on OSS (which is pretty kick ass in itself), it’s a fantastic chance to get to work with some of the smartest people in the ruby community in a situation that encourages experimentation on cool ideas and collaboration among peers.

For me, three things really stood out on top of this. For starters, I got a chance to learn a hell of a lot – Before Ruby SOC I considered my understanding of shell scripting (at least in bash / zsh) to be basic at best. Now, I’m competent enough to write larger parts of rvm where needed and to also start scripting most things in my env as I encounter them (Once you’ve started doing so, it becomes almost like an addiction. If I get the time, I’d like to work on omg-wtf-bashbq as an alternative to oh-my-zsh but for bash – but that’s in the future.

Secondly, I got a chance to work on problems faced by real people within the ruby community. I say that not only in regards to making ruby deployment with multiple rubies easier, but also in the regard that I’ve gotten a chance to quite often help out people in the IRC channel. It’s been incredibly fulfilling to help people fix issues they’re encountering even more so if it’s a bug you can solve and push a fix for in mere minutes.

Lastly, and most importantly out of all of it for me, Ruby SOC gave me the chance to work with a mentor as awesome my own primary one, Wayne E. Seguin. I can’t stress enough how much of an honour it’s been to work with Wayne – not only has he been very patient with me as I learnt different things but he’s also taught me a hell of a lot of invaluable knowledge. I guess it’s hard for me to express in words just how much of a pleasure it has been working with Wayne, especially given how busy a guy he is (work, kids as well as almost-more-time-than-humanely-possible working on rvm) but none the less I’m forever grateful for the chance RubySOC has given me and the time I’ve gotten to work with him. Going forward, I’ll be continuing to work with Wayne on RVM and even keeping him on as a mentor of sorts – huzzah!

As another side note, If you use and like rvm, I strongly suggest sending a donation to show your appreciation for the many, many hours of work Wayne puts in of his own time to make rvm possible – It constantly amazes me at how dedicated he is to it. Even better, wait until you see some of the awesome projects he has planned for the near future.

Closing thoughts

Overall, I’ve had a real blast doing Ruby Summer of Code – Not only did I get to work on a seriously awesome project, I got to work with great people like Wayne and I got a chance to help work on problems faced by people in the ruby community.

Given the huge scope of the project, I unfortunately missed a few things I’d proposed (such as the web UI for RVM) but got to spend a lot more time expanding other aspects (such as getting RVM 1.0.0 out the door). I got a lot out of doing Ruby Summer of Code and I’m very thankful to both Wayne and Leah Silber / the RubySOC team for organising it – It’s seriously been a killer experience.

Lastly, I want to give a shout out to local RubySOC sponsors The Frontier Group – I’ve started working for them since the end of the program but before then not only were they half project sponsors but they also hosted me for most of the program in the form of a desk, general in person discussions, being guinea pigs or various things as well as providing a fast iMac and VM to test things such as compilation on.

Interested in more?

I plan on being a more active blogger now that RubySOC is over – If you’re interested at all, I suggest:

  1. Checking out my Twitter
  2. Subscribing to my blog feed