Abstract

Is CoffeeScript really so great? Why use it instead of good ol’ JavaScript? Which of my favorite syntax goodies does it steal from Ruby? It looks Pythony with all that significant whitespace. How can I use it in my pre-3.1 Rails project? How do I convert my legacy JavaScript? Why is Internet Explorer no good at exploring the Internet? Brian Morearty answers most of these questions. He’s a Ruby aficionado who has used CoffeeScript for his client-side coding for a few months. In this talk he does a quick demo of how to install CoffeeScript, convert your JavaScript to CoffeeScript, and include it in a pre-Rails-3.1 application. But most of the talk is on details of the language, including stuff it stole from Ruby that make it just dang fun for a Rubyist to program in CoffeeScript. And he even covers one thing that is *gasp* better than Ruby.

Video

Run Terminal and type:

  $ purge

My memory went from  to  the last time I did this. Not too shabby. (Why did it also clear up some Active Memory? I have no idea.)

That’s all you have to do. It can take up to a minute. While purge is running your system will be slow. After that, with all that free memory, it’ll be zippier than before.

Don’t listen to all the remarkably obnoxious fanboys or official Apple support pages telling you just to leave Inactive Memory alone because it will be freed up quickly if it is needed, or that the system knows better than you do when to free memory. Trust your experience. In my experience it will not and it does not. 

Here is the purge man page:

PURGE(8)                  BSD System Manager's Manual                 PURGE(8)

NAME
     purge -- force disk cache to be purged (flushed and emptied)

SYNOPSIS
     purge

DESCRIPTION
     Purge can be used to approximate initial boot conditions with a cold disk buffer
     cache for performance analysis. It does not affect anonymous memory that has been
     allocated through malloc, vm_allocate, etc.

SEE ALSO
     sync(8), malloc(3)

Do you ever re-run the same Rails test file over & over while working on a feature and trying to “make it green?” Change the code, run the test, change the code, run the test, etc. (I’m talking about when you want to re-run a specific test, not using guard to autodetect which tests to run.)

When you do that you have to wait a long time for the Rails environment and your app’s initializers to load. You pay this price every time you re-run the test—even if you’re not changing your initializers. I get ants in my pants sitting around waiting for the environment to load.

I just learned about a new gem that doesn’t require any changes to your tests and that speeds up this test-code-test-code cycle by only loading your environment once: it’s called spin.

https://github.com/jstorimer/spin

Basically you do this:

• In one terminal window type spin serve. This launches the spin server, which waits for commands over a socket.
• In another terminal, when you want to run the tests in a single file, type spin push path/to/testfile.rb
• You can push multiple filenames at once if you like.
• One limitation: it does not let you use the :linenum syntax that rspec uses to run just one test in a file.

That’s it. I tried it on a test file that takes 8 seconds (after everything has been loaded). Total time to load and run this file went down from 26 seconds to 13 seconds when I used spin.

I looked at the source. It’s short and simple. Here’s how it works:

• spin serve preloads your app’s config/application.rb.
• Then it waits for commands on a socket.
• When a command comes in it forks a new process and runs tests in the fork. The fork doesn’t have to reload the Rails environment or initializers because they’re already loaded.
• When done, the fork just exits and spin is ready for a new command.

Remember that if you do change your initializers or anything else that is loaded by application.rb you should restart the spin server. Another time you may want to restart it: after checking out a different branch in git.

Safe and easy. Faster tests. What could be better?

Dear GitHub,

Hi guys!

Guess what: GitHub’s great. Yeah, you know that.

But hey: could you please fix GitHub notifications? They seriously suck. No, really, I mean it. They’re the worst. Sorry, I don’t mean to hurt your feelings. I know a human being designed the notification system.

GitHub notifications are a productivity sink. Despite having fifteen checkboxes for customization in the Notification Center, I still can’t make them work in a way that helps me. Don’t get me wrong, I am not asking you to fix it by adding more checkboxes to the list. I am asking for an overhaul of how the notification system works. Below I will try to clearly lay out the problems I see and what I would like the system to do instead. The overall goal is to change notifications from a productivity sink to a productivity boon.

First I will describe myself and how I use GitHub, to help you decide if I am the kind of customer you want to serve. I understand that an important part of running a successful business is that you can’t be all things to all people. So here’s me:

• I am an independent software consultant doing Ruby and JavaScript programming. I have a long-term gig at a company that has 81 private repos on GitHub. I am part of their GitHub Organization so I have access to all the repos. Combined, these repos get a ton of activity–there are probably about thirty developers working on them full-time.
• The company I work at uses GitHub for code reviews, too. People sometimes post in-line comments in the code reviews.
• I have written some open source software. Not a ton, but a few little things. I host them on GitHub. This has worked out well for me. All Ruby and JavaScript.
• I am also the maintainer of an open source software project that someone else wrote in Python and posted to a bulletin board. It’s for exporting iPhoto pictures to a folder structure. I wanted to make some changes for myself but let other people make changes too, so I posted it to my GitHub account. It has become popular enough that I get a pull request once every few months. I try to respond to them and pull them.
• I participate in the occasional issue discussion on the Rails repo and a few other repos.
• When I’m at work I want to be notified about events related to work but not be distracted by the open source projects I work on (except during the occasional break from work).
• When I’m at home I want to be notified about events related to the open source projects I work on but not be distracted by work-related events (except when I occasionally check to see if anything’s going on while I’m out).

Ok, so that’s me. I would characterize my use of GitHub as heavy–every hour of every workday–in my professional work and light-to-medium in my open source work.

So How Can GitHub Notifications be Improved?

Well, these things would help:

• In the Notification Center I want to set up preferences at two levels: one for projects I’m Actively Involved in and another for projects I Occasionally Participate in. For example if I submit a pull request for any repo, whether I’m Actively Involved or not, I want to be notified of comments on my pull request. But I only want to be notified of pull requests from other people on the projects I’m Actively Involved in.
• Every new repo someone else creates is assigned the Occasionally Participate level for me. Every new repo I create is defaulted to the Actively Involved level for me. These statements are true whether the repo is Public or in an Organization I belong to.
• For any repo I have access to, I can change whether I am Actively Involved or Occasionally Participate.
• These levels will be especially helpful for the work I do in an Organization. I can stay updated on events in the two or three repos I use daily but prevent spam on repos in my organization but that I never touch. I’d like to see pull requests for my active repos but not for all repos in the org.
• I want the dark gray Unread Notifications icon at the top of every page to reflect the current Context (personal, organization #1, organization #2). Hovering over it would quickly show the notification count for each context. The counts in that tooltip should be clickable to go to the Notification Center for that context.
• I want the Notifications page to show me only the notifications for the currently selected Context. (Myself or whatever Organization context I’m currently in.) And it should be easy to switch to a different Context from that page.
• I want to specify one email address for my own account’s notifications, another email for my organization #1 notifications, another for my organization #2 notifications, etc. It’s really frustrating that I need dozens of rules in my personal GMail account to forward GitHub notifications to the correct work email accounts.
• I want the notification emails to include the other user’s full name in the From field. It’s ok to still include the GitHub username but please also include their real name, just like you do on the user’s home page. E.g., my email client should tell me the email is “From: defunkt (Chris Wanstrath).”
• I want the notification emails to provide more context about exactly what happened. The emails are horrible! It’s impossible to tell from the Subject line or even the Body what happened. What am I being notified about, exactly? Oh, somebody filed a pull request? For Pete’s sake would it hurt to tell me that without forcing me to look at the URL at the end of the email? Right now the subject line is the title of the pull request, the body is the body of the pull request, and that’s it. I’m expecting the words “Pull request” (or whatever the notification is about) in the email subject line AND at the top of the body.
• And don’t even get me started about emails telling me about comments on commits. The emails say nothing about the code being commented on or who made the commit. The subject line should mention if it’s “on your commit” or “on defunkt’s (Chris Wanstrath’s) commit.” Notice I used the username and real name. And I want to see this in the body:
• The surrounding code that’s being commented on.
• The comment.
• The line where the comment was made.
• Who made the commit! Was it me? Sheesh. (Username and real name.)
• Who made the comment. (Username and real name.)
• I want email notifications to include a link at the bottom: “Delete this from the Notifications page.” I want to click that link from my email program and have it kick off a URL that deletes the notification off GitHub so I don’t see it again the next time I go to the Notifications page.
• I want every notification–in email and on the Notifications page–to tell me why I was notified. E.g., “You received this notification because you have turned on Comments on commits in MegaCorp repositories.” The link takes you to the page where you can turn it off. Doing this on the Notifications page will be tricky because you don’t want to clutter up the page too much. Just show me a tiny question mark on the row so I can hover over it and see the explanation in a tooltip.
• I want the Notifications page to have a link right at the top that says “Edit Notification Settings” and takes me to the Notification Center tab of the Accounts page.
• If I delete all the notifications one by one on page 1 of the Notifications page, it should not look like I have emptied out my notification list. It should be clear that there are more. How about an AJAX UI that pulls in the first row from the next page and adds it to the end of this page each time I delete one?
• Related: if I delete all the notifications on p. 1 and click Next to go to the next page of notifications, it should not skip past the new first page of notifications. I.e. it should show me the notifications after the last one I deleted. This won’t even be necessary if you do the AJAX UI I mentioned.
• When I am cleaning out my notifications in the Notifications page I want to be able to see at a glance–without drilling down on the notification–what each notification is about. I should see all of the following. Some of your notifications do include all of them but some don’t. The ones that don’t are very frustrating. E.g., “So-and-so discussed a commit 3 hours ago.” Really? Thanks. In what repo? Come on. Are you really going to make me guess? And was it my commit or someone else’s?
• What repo was affected
• Who took the action I’m being notified about (username, real name, and little avatar)
• What the action was (commented on my commit, submitted a pull request, …)
• When it happened
• A one-liner about the event (the pull request subject, first line of the comment, etc.)
• If it’s a pull request, what is its current status? (Open, Closed) In other words, did someone already pull it after it was added to my inbox?
• I want the Notification Center to let me turn off not just email notifications but also notifications in my Notifications page. I really don’t have time to care about things that happened in about 78 of the 81 repos my organization owns. And when my Notification icon is showing me big numbers of unread notifications, it’s the Little Boy Who Cried Wolf. I end up having to ignore the dang notification icon because so many of those notifications are irrelevant to me. And you know what that means: when a real big bad wolf comes into town, I will ignore the cries for help and the po’ little boy will be eaten. (Or I might not respond to a notification I should have known about.)
• A note on that last point: I know it would be confusing to have two checkboxes for each notification type (one for the Notification page and another for whether you want to get email). You can avoid Checkbox Overload with this simple fix:
• Keep the checkbox at the top titled “Receive email notifications (global setting).” This would control whether I get email notifications at all.
• The event-specific checkboxes in the list should not be labeled “Email.” They should be labeled “Notify.” If I check a box, I want that notification in my Notification page–and by email if the box at the top was checked. If I don’t check the box, I don’t want either.
• I want to mark individual notifications as Read or Unread from the Notifications page.
• You’ve been adding a lot of nice shortcut keys to GitHub pages lately. I want to navigate the Notifications page with keystrokes, a la GMail. j=down, k=up, o or Enter=open, U=mark as unread, I=mark as read, #=delete (or maybe y).

Yesterday evening I got back from RailsConf 2011. It was a lot of fun and I saw a lot of really good presentations with good technical content, and some that were both useful and funny (double dream hands and Ke$ha).

But I also missed some of the best talks. O’Reilly has posted some of the videos and is working on posting more.

But how to decide which ones to watch?

On the plane ride home I wrote a one-page site to answer that question. It scrapes the star ratings from O’Reilly (hope you guys don’t mind!) and shows the sessions with the highest-rated ones at the top. I didn’t have wifi so I had to guess what CSS selectors I should use to scrape the RailsConf site. Once I got home I fixed the selectors, polished it up, and pushed it to Heroku.

Here it is: the Best of RailsConf 2011.

If you find problems, please fork and improve it.

I knew we were in a bubble. Don’t get me wrong—I’m no soothsayer and I couldn’t tell you exactly when the bubble would burst, but it was clear it would. So I joined Intuit, the only company I interviewed with that was actually making a profit.

It turned out to be a pretty good choice.

The nearly 12 years I worked at Intuit were amazing. The QuickBooks team warmly welcomed me. I worked with fantastic people. I became the tech lead for the UI Toolkit team. I worked on a lot of elegant little touches that make the user experience more delightful, like a better quickfill (autocomplete) control. I trained engineers on how to build great user interfaces. I stayed there til 2005 when I decided to try something new, but Intuit is such a great place to work that I just transferred to another group. I moved around a couple more times, always staying in the Small Business Division, and since 2008 I have been doing web development in Intuit’s Grow Your Business division. This is the group that makes Intuit Websites.

I recently decided the time has come again to change jobs. But this time is different. Instead of looking for a job as an employee somewhere else, I’ve decided to do something I’ve been thinking about for a number of years: I am becoming a software entrepreneur and consultant/trainer.

Both? Well, yeah—I’ll be spending 30 hours a week consulting and training (focusing on Ruby on Rails and JavaScript) and the remaining time starting my own business.

My Startup

During my “entrepreneur” time I am creating a website with tools and community for small real estate investors. I am an investor myself, having bought houses and condos in Southern California, Arizona, and even Melbourne, Australia. When I got started with investing I was willing to invest anywhere in the U.S., but I had one big question:

Which market should I invest in?

The answer depended on what was I looking for in a real estate investment. Some investors look for equity growth, others for cash flow. Some look for a market on an upswing, others for a market in a trough. Some look for job growth, some for immigration, some for vacancy rates, and on and on. The thing is, all this data is available for free from the U.S. Government but it’s really hard for a small real estate investor to find. And once they’ve found it, it’s also hard to collate it and get a quick answer to the simple question of which market to invest in. Real estate agents can be helpful but unless you happen to find one who specializes in finding investing opportunities around the country, they won’t really know what to recommend. In that case they’re likely just to tell you that their own market is a good one to invest in.

So a few years ago I started working on a web app to solve this problem and recruited my friend Ed to help out. Think of it as Kayak for small real estate investors: you give it your criteria, drag some sliders, check some boxes, and it shows you the results. Except instead of showing you flights it shows markets. It’s fast and easy and you don’t need to pore through thousands of lines of data from government reports. The web app does all that for you instantly.

Working on the app was fun and we got a lot done over several months but it was hard to find enough time for that when we also had full-time jobs and families we wanted to see. But now that I’m consulting 30 hours a week instead working 40 hours a week, that leaves me more time every day. So I’m working on the app for real estate investors again.

I’ll make an announcement here when it’s ready to try. Subscribe to my blog so you don’t miss it. (Go here and click the Subscribe button in the right hand column.)

My Consulting and Training Business

To keep the lights on I am also doing Ruby on Rails consulting and training. I can make web apps work great on mobile phones and desktop browsers with a single code base, scale Rails apps, and a bunch more. See my About page if you’d like more info on my services, and my Contact page if you want to contact me about work.

I Like Stuff That’s Fun

I’m already having a ton of fun and I’ve barely started. It’s great to be working with Ruby on Rails full-time, controlling my destiny, and creating my future. And I’m really enjoying my first consulting gig.

Stay tuned.

In some of my projects an identity map would have saved a lot of ugliness. I’m going to start using this feature in small projects as soon as it’s released. Once it has proven itself to me it will become part of my app template for all my projects. (It’s turned off by default, which seems reasonable for now.)

Read more about it in the link above.

# controllers/application.rb
  before_filter :set_user_time_zone

  private

  def set_user_time_zone
    Time.zone = current_user.time_zone if logged_in?
  end

Problem is, the current user’s time zone will leak across to the next request on the same thread. If the next one is by a logged-out user, all times displayed on that next request will show in the first user’s time zone.

To prevent this, use an around_filter:

  around_filter :set_time_zone

  private

  def set_time_zone
    old_time_zone = Time.zone
    Time.zone = current_user.time_zone if logged_in?
    yield
  ensure
    Time.zone = old_time_zone
  end

You might be tempted to combine the first two lines of set_time_zone into one line with the comma assignment syntax and add “if logged_in?” to the one line in the ensure clause. Don’t do it. If you did that, the user’s time zone would leak out to the next request when the user logs out.

I’ve submitted a Rails patch to use this method in the sample code for the Time.zone= docs.

This is for people on a Mac, using rvm and homebrew.

I ran across this with the very nice Sucker gem, which lets you easily access the Amazon Product Advertising API. Amazon requires SHA-256 in its API calls, which is where my trouble began.

If your Ruby app got this error message in openssl/digest.rb:55:

Unsupported digest algorithm (sha256).

it’s because when you installed (compiled) Ruby, you linked in an older version of the openssl library.

I Like Stuff that Finally Worked

Here’s how I fixed it:

brew install openssl
rvm remove 1.8.7
rvm install 1.8.7 --with-openssl-dir=/usr/local/Cellar/openssl/0.9.8o/

To test it, run this in the Terminal:

ruby -ropenssl -e 'p OpenSSL::Digest::Digest.new("sha256")'

If you don’t get an error message, it worked.

Things That Didn't Work

Installing and building openssl from source (openssl.org) did not work.

Installing the openssl rvm package and removing+reinstalling ruby did not work.

What finally worked was the steps above. Woo hoo!

By the way, I did not have to upgrade to Snow Leopard.

What about Ruby 1.9.2?

I still haven't gotten it working in Ruby 1.9.2. I run this:

rvm install 1.9.2 --with-openssl-dir=/usr/local/Cellar/openssl/0.9.8o/

and I get this output:

ruby-1.9.2-p136 - #fetching
ruby-1.9.2-p136 - #extracted to /Users/Brian/.rvm/src/ruby-1.9.2-p136 (already extracted)
ruby-1.9.2-p136 - #configuring
ruby-1.9.2-p136 - #compiling
Error running 'make ', please read /Users/Brian/.rvm/log/ruby-1.9.2-p136/make.log
There has been an error while running make. Halting the installation.

So I go look in make.log and I see this at the end:

Generating RDoc documentation
./miniruby -I./lib -I.ext/common -I./- -r./ext/purelib.rb  ./tool/runruby.rb --extout=.ext  -- "./bin/rdoc" --no-force-update --all --ri --op ".ext/rdoc"  "."
/Users/Brian/.rvm/gems/ruby-1.8.7-p330/gems/rdoc-3.4/lib/rdoc/rdoc.rb:79: warning: already initialized constant GENERATORS
uh-oh! RDoc had a problem:
undefined method `coverage_report' for #<RDoc::Options:0x7841a0>

Does anyone know how to fix it?

(You may notice that the title of this post has a small difference from the earlier ones. Since the last post a couple weeks ago, Rosey has turned 12.)

Calling ‘puts’ over and over again is alright, but there’s nothing more fun for a fledgling programmer than asking the user a question, getting the answer, and then writing back a witty reply. In Chapter 4 of Learn to Program, Rosey learned about variables and assignment but the real fun came in Chapter 5, when she learned how to call gets to get a string from the user and then use the String.+ method to write back that string in the middle of a response.

Near the beginning of the chapter, though, I decided to take a detour and explain computer memory. This is the sentence in the book that triggered this:

“To store the string in your computer’s memory for user later in your program….”

The words “store the string in memory” reminded me of something I can still clearly remember: way back when I was 14 or so and first learning to use WordStar on my family’s Osborne 1, I found it hard to understand the difference between “Open” and “Save.” The picture in my mind of what’s going on inside the computer was extremely vague. I didn’t understand the difference between RAM and the Hard Drive, and why the word Open should mean moving something from the hard drive to RAM and Save should mean moving something from RAM to the hard drive.

I’m a believer in remembering what I struggled with in a subject, and trying to explain those things when I teach the subject. I hope that makes me a better teacher.

RAM vs. HD, Open vs. Save...and the BRAIN

So I quickly sketched the attached diagram, explaining the pieces as I went. As I suspected, the difference between RAM and HD was pretty abstract for her and took some time even though she is an experienced game player. She got it, though, and we moved on the fun stuff.

The chapter on variables and assignment was surprisingly easy. The first few pages of the next chapter (Chapter 5, “Mixing it Up”) were useful for her as well, although not incredibly exciting. They explained things like why you can’t do ’5′ + 5, and why ’99.999′.to_i returns 999.

But this was all just leading up to the awesome part. In section 5.3 the book introduces the gets function, and in 5.5 you ask the user’s name and then print it back in a sentence. Rosey was so excited the first time she ran that. It was fun to watch.

Finally she did the exercises for the chapter, which were really kid-friendly. The first one tells you to write a program that asks for a person’s first, middle, and last names and then greets the person using their full name. She had fun with that, writing out prompts like “That’s an…um, interesting middle name.” And the next exercise was cute: ask for a person’s favorite number, add one to it, and tactfully suggest that as a better favorite number because it’s bigger. She got the essence of the program right on the first try but she was missing the conversion between strings and numbers. I made sure not to point out the errors as she was typing. I think that was the right approach, because the interruptions might have prevented her from getting the basic logic down right.

One last thing to note: I remember being confused about the assignment operator when I learned to program, and she was too. In particular, she thought the value should go on the left and the assigned variable should go on the right:

fav_number + 1 = new_fav

Take the programmer part out of your brain and read that code out loud. It makes sense, doesn’t it?

“Favorite number plus one equals the new favorite number.”

But alas, the computer doesn’t think so. She grudgingly accepted that and moved on.

I’m not sure when our next lesson will be. Hopefully sooner than the next holiday, since that’s in May. :-)

See you all next time….

Want to do some quick and dirty profiling to see how long some Ruby code takes? Do this in irb or in a Ruby project:

require 'rubygems'
require 'active_support/core_ext/benchmark'

puts Benchmark.ms do
   # put your code here
end

The output is how long in milliseconds the code took to run.

This requires ActiveSupport but does not require all of Rails. If you don’t have ActiveSupport, install it with gem install activesupport.

My title: “Exposing New Twitter’s Secrets (Without a TSA Full Body Scan).”

Want to build your own single-page web app? If my proposal is accepted, come attend my talk and we’ll reverse-engineer New Twitter to see how they did it. Maybe you will get some good insight and ideas.

First I had to set some ground rules. Well, one ground rule:

Let’s stay on topic.

Kids always want to tell you random things. Unless they’re teenagers, of course, in which case I’m told they close off in their room for about four years. So I gave her a piece of paper and a pencil. If she thought of anything unrelated to programming that she was dying to tell me, she could write a reminder to herself on the paper and tell me later. I think it helped. At one point she was juuuuuuuuuuuuust about to tell me something but she caught herself and just wrote it down. I think having this release valve helped get the idea out of their head so she wouldn’t be distracted and could concentrate.

So, down to work. We installed Ruby on her laptop with the Ruby One-Click Installer. Yes, it’s her laptop even though we have three kids. Each kid has their own laptop. Refurbished corporate PCs are pretty cheap. And we installed Notepad++ too, since I love it.

Rosey had fun playing with the echo command on the command line, echoing everything I said. Having a computer do this is only slightly less annoying than having a person do it. Then she did this at the prompt:

C:\>:-)

And to our surprise, there was no error message. It just came back with another C: prompt. I guess that’s because in the Windows batch language a line that starts with a colon is a label—and you can put anything you want after it. Fun.

We finished Chapter 1 in a hurry and I was worried it wasn’t the best first day. Fortunately, she said yes when I asked if we could just do the first two paragraphs of Chapter 2 since I saw that’s where the instant gratification begins. She was hooked after this worked:

puts 1 + 2 # => 3

She experimented with some other math. I told her about * and / for times and divided by, and she did:

puts 300 / 8

but was surprised when the program printed 37 instead of 37.5. At that point I became pretty impressed with the order in which the book introduces new concepts, because I had read ahead and knew that the next page describes integers vs. floats, and goes on to explain why integer math always returns an integer. We read through it, typed in the examples, and were having fun. The book impressed me with its clairvoyance again when Rosey got tired of typing the spaces between the mathematical operators and asked if they are really needed. I showed her the next paragraph, which said “the spaces in the program are not important; they just make the code easier to read.”

Rosey also tried one of her favorite phrases:

puts I like pie

but she got an error message because she doesn’t know yet that strings have to be quoted. (The command-line echo command didn’t need them.) I decided not to bother explaining that for now, since the next chapter goes into it.

At one point the book said, “Don’t type commas into your numbers” because it just confuses Ruby. But I taught her a little-known Ruby trick, using underscores where you would have used commas. She’s learning ancient ninja secrets.

puts 1_000_000

I thought the exercises at the end of Chapter 2 would be hard. They ask the reader to solve problems like “How many hours are in a year?” and “How many seconds old are you?” It turned out they were a piece of cake. She raced through them, stopping only a few times to figure out whether to divide or multiply.

Next up: Rosey’s next reading assignment is Chapter 3, “Letters,” which talks about strings. Christmas vacation is over and school starts up tomorrow, so her homework will take precedence. But hopefully we’ll be able to keep the pace going and not lose steam.

So far so good!

In the last year I taught my daughter Rosey, who is 11 years old and in sixth grade, the basics of HTML. With help and training from me she made a few web pages with her own drawings, including some clever puzzles. We hand-coded all the pages instead of generating them with a drag-and-drop tool. That was her choice. (Hooray!)

She came away knowing some HTML. She knows it well enough to give a bit of an explanation to her friends, which I think is awesome. I tried to teach her a bit of CSS and JavaScript too. She may have had an inkling of how they work, but I didn’t do a very good job on those parts. I think it was mainly because I didn’t have a structured way to teach them. I admit I was pretty much all over the map. HTML was easy to teach tag by tag–start with <html>, <body>, <p>, and <img>. But with JavaScript to do anything useful on a website you have to go quickly past the basics and understand functions, the DOM, and more. I ended up writing the JavaScript myself and hoping she’d at least know it exists so she could come back some day.

Now it’s a new year, and next week she’ll turn 12. I asked her if she’d like to learn to program in a language that can actually make things happen instead of just show things on a page. Somehow I must have used just the right combination of excitement and pleading, because she said yes.

Very cool. I get to teach my daughter to write software! I’ve been waiting for this day for a long time. Almost 12 years.

She already knows a couple of boys her age who have started learning to program but most of the boys have not. I’m excited that she will be ahead of a lot of the boys, since I hear one reason girls are sometimes behind in learning computer stuff is the intimidation factor: they take a class and see that most of the boys have already learned a lot of it on their own, so the girls feel like they’ll never catch up. I totally get that, and want to prevent it.

I Like Stuff that’s Ruby

Decision #1: what language to teach? I wanted to teach an interpreted language because interactive interpreters make learning easier. I primarily considered JavaScript and Ruby. I think they’re both great choices because (a) they’re both interpreted, (b) neither requires an immediate understanding of object-orientation, (c) I know them both really well and love them both, and (d) they’re actually useful.

I also considered “kid” languages like Scratch or Logo. But I’ve tried Scratch and honestly I didn’t get it. If I don’t get it, how am I going to teach it? I did learn Logo years ago and it was fun. But I was more excited about teaching JavaScript or Ruby.

A couple years ago while browsing the Programming section of Borders Books in Palo Alto I noticed Chris Pine’s book, Learn to Program, and wondered if it would be good for a middle-schooler. I read more about it, and found a review titled “Great for Kids.” The author is using Learn to Program to teach their 8-year-old daughter and loves it. Plus I learned that the author actually teaches programming to kids my daughter’s age. That’s good enough for me! So I got the book and will start teaching her to program.

(By the way, if you haven’t been to Border’s on University Ave. in Palo Alto, do yourself a favor and go inside. It’s cool. It was a beautiful, classic movie theater called The Varsity when I moved to downtown Palo Alto in 1988. Later they converted it to a bookstore but they kept the marquee.)

Lesson 1

So Today I went out and picked up the book. No way could I wait for an Amazon shipment. Then I read the Preface, Introduction, and Chapter 1. Then I asked her to start with the Preface and read for 20 minutes or so and stop wherever she is, or to finish Chapter 1 if she wants. She ended up stopping at the same spot as me. Tomorrow we’ll go over Chapter 1 together. It’s mostly just about installing Ruby and a text editor. But I’ll also go over some of the high-level concepts I remember being incredibly confusing to me when I first learned to program. Specifically:

• When I type a line of code into the computer, how should I think of this in comparison with an English sentence? Is it Declarative? Imperative? Something else? (Fortunately I think she has learned these terms in school. If she hasn’t, I can quickly describe the differences.) I think for a beginner the easiest explanation is that all lines of code are imperative. More specifically, you are telling the computer what to do. When you say “print 3″ it’s like telling someone else–someone who always follows orders–to print the number 3 on a piece of paper. Of course some computer programming statements are more like declarative, interrogative, or conditional statements. But the first thing a beginner learns is the imperatives like “print.” (I peeked ahead at Chapter 2 and it starts with “puts 1+2″.)
• What is the difference between RAM and the hard disk? How should I think of “saving” or “opening” a file? My daughter has saved documents before but I don’t think she quite groks RAM vs. disk yet.

Expectations

I enter this adventure with excitement and high hopes but also realistic expectations. She may get bored quickly or feel it’s too hard. I’m going to do my best to keep it light, fun, and low pressure. But if it doesn’t work out, it’s no big deal. You have to try things, right?

I will continue to blog about this adventure. Stay tuned right here on the I Like Stuff Blog.

def index
  @posts = Post.all     # not lazy; returns Array
end

It turns out this is by design. See Force loading – all, first & last.

So: what do you use if your view has fragment caching and you want your controller’s index method to delay the query until you’re sure you’ll need to use it?

At first I did this:

def index
  @posts = Post.where('1=1')  # returns ActiveRecord::Relation
end

But the documented way is to use the scoped method:

def index
  @posts = Post.scoped        # returns ActiveRecord::Relation
end

Another alternative I recommend, which lets you do lazy execution queries in any version of Rails: move the queries to helper methods.

At first I was having fun with SQLite. It’s lightweight, doesn’t use much memory, runs fast on a local dev machine. A single database is stored in a single file. No server to configure, no nothing. Nice.

But I gave up on it after about five days because of an annoying limitation that bit me twice: if you add a NOT NULL column (:null => false in your migration) to an existing table, SQLite requires you to specify a default value—even if the table is empty.

It makes sense to require a default value on a NOT NULL column if there are already rows in the table. All relational databases require that. But if you don’t want to specify a default and the table is empty SQLite shouldn’t force you to.

I found discussions where people suggested a workaround of “Can’t you just drop the table and create a new one? As the table is empty, you won’t be losing any data.” Sure, but it’s far more error-prone to recreate the table because it’s not an incremental change. You have to carefully reconstruct the table and all indexes from all past migrations, which became unwieldy for me even in a 5-day-old project. It really is more error-prone: I tried it and screwed up a couple of times. That was it for me, so I decided it’s time to switch back to MySQL for local development.

The other experiment—using different databases in local development and production—is still going fine. We’ll see what snags I hit that make me change my mind about that too. I still wouldn’t do that on a big project but this is a little ol’ thang.

The most notable changes:

• It uses less memory. This matters if your iPhoto library is huge, because you will no longer overwhelm your Mac’s memory when exporting the photos to folders. It now uses a SAX parser so it no longer has to load the entire iPhoto XML file into memory.
• It shows better progress than before, telling you how many events/albums it has processed so far and how many there are total.
• If you want, it can copy photo metadata (image name, description, keywords, faces, and rating) into the exported photos.

Story #1: If You Fragment Cache a View

If you want your Rails website to run faster and scale better, you might decide fragment caching will help.

And chances are, you will put memcached on the server and write cache statements in your views.

But if you are still on Rails 2.x then you’re not using Arel’s lazy execution. So you might notice your controllers are still querying the database even for views that are cached and won’t use the data. So you will probably add read_fragment calls to the controllers to prevent that.

If you used a custom key in the view’s cache statement you will want to use the same custom key in the read_fragment statement.

But if it’s a complex key, that might not feel DRY. So you might refactor the key out to a helper function you can call from the controller and the view, passing the result in to read_fragment and cache. You might even put the helper in application_controller.rb if the view is a partial that’s called from several controllers.

Then you might feel bad that your fragment cache logic, which started out nice and clean and small in the view, is now spread out among three source files. You might get a headache. You might think there must be a better way. And you might ask for a glass of milk.

Story #2: If You Remove Data from a View

If your product manager wants you to output some data in a view, you might ask for the data in your controller. You might use ActiveRecord’s find method since you’re still on Rails 2.x and can’t use Arel yet.

And chances are, if you query data in your controller, you’re going to store the result in an instance variable.

If your product manager decides three months later that the data doesn’t need to be shown after all, she may ask your coworker to remove it. Your coworker may do just that–remove it from the view.

Your coworker may not realize the query in the controller is no longer needed. He may not even think of removing it.

Your server may worker harder than it has to, continuing to query the database on every page view to get data that’s never shown on the page.

Your server might get a headache. It might ask for a glass of milk.

Story #3: If You Combine Stories 1 and 2

If your product manager asks you to show data in a view, chances are you will put a database query in your controller to pass the data to the view.

When traffic gets high and your server gets overloaded, you may decide to use a fragment cache. You might use :expires_in => 5.minutes so your server only has to query the database once every five minutes.

You might get the same headache you got in bedtime story #1 because you have to add a read_fragment call in your controller, a cache call in your view, and a reusable key function in your application controller. But the glass of milk might help and your headache may go away.

When your product manager says you no longer need to show the data in your view, chances are you will remove the fragment cache and its contents from the view. But you might forget about the query in the controller. You might leave it there.

But no big deal, right? The app still works. Your tests still pass. You’ll only be querying the database once every five minutes, after all. Wrong. Remember: calling cache from the view is what writes to the cache. By removing the view’s cache statement you have removed all cache writes. So there’s never anything in the cache. So read_fragment will always return nil.

And if read_fragment returns nil, your query will run.

On. Every. Request.

By removing the data from the view you have gone from querying the database once every five minutes to querying it on each and every page view, even though you never need to show the data.

You’re going to need a really big glass of milk to get rid of that headache. Hopefully you’re running New Relic and will see the huge increase in DB queries immediately after deploy so you can look into it quickly.

The Promised Happy Ending

There are several possible solutions to the problems in these three bedtime stories. The third one is my favorite by a long shot:

1. Not the solution: perform the query in the view instead of in the controller. Obviously this is wrong. I won’t discuss this further.
2. Not the solution: use Model-View-Presenter. I won’t discuss it either because I think it needlessly complicates matters. We Rails programmers tend to believe in the KISS principle, and there is a simpler solution.
3. The solution: move your queries out of the action methods and into memoized helper methods in your controller.
   1. Give the helper methods the same name that the instance variables used to have. (@company becomes the company method).
   2. In each helper method, assign the result to the instance variable. Make sure the assignment uses ||=, not plain old =, so the helper runs the query only once no matter how many times it’s called.
   3. In your view, never rely on instance variables having been previously set by the controller. Remove all the @ signs from the views and almost all @ signs from the controllers. (Exceptions are in new and create. See below.) What you’re left with are function calls to the helpers. Each helper will perform the query the first time it is called within a single request.

This solution is pure beauty. Here’s why:

• It solves bedtime story #1. You can remove all those sloppy read_fragment calls from your controller. The only reason they were there was to limit database calls to only when the view needs it. But by the time your helper function is called, you know the view needs the data and wasn’t able to find a fragment in the cache. So just go get the data. This means you can also remove the reusable key method from application.rb and put the cache key inline as a parameter to the cache statement. Your code will still be DRY because you only need the key in that one place.
• It solves bedtime story #2. If you ever remove your data output from your view, you are removing calls to the helper functions. If they’re not called, they won’t query the database. So even if you forget to remove them from the controller, they’re doing no harm.
• It solves bedtime story #3. If you ever remove a fragment cache and its contents from the view and thus never write to the cache any more, it does not increase the number of times your controller queries the database. Since you’ve removed the view code that calls the helper functions, you actually reduce the number of database calls to zero.
• It’s simple to understand.
• It’s easy to implement.
• The resulting code is clean. Your controller functions are shorter and therefore sweeter. You don’t have to add new classes. You don’t have to write any new code.
• It does not require putting database calls in the wrong layer (the view).
• It does not require explaining a scary new pattern name to your team and to every new person who joins your team.

Does It Work in Rails 3?

Sure, it works in Rails 3. You will get the same benefits from using it in any version of Rails. Rails 3 has Arel with its lazy evaluation of queries, but the default generated controllers don’t take advantage of this feature.

Update: previously I said it won’t really be needed any more after you update all your find calls to Arel’s implicit query that runs when you start iterating over results. It turns out I was wrong: even in Rails 3 the default index function calls ModelClass.all and the other functions call ModelClass.find. Both of these run an immediate query, not a lazy one.

Putting performance improvements aside, this technique also DRYs up your controller code a bit even when you’re using Arel. Instead of putting @company = Company.find(params[:id]) in every controller method, you just write it once.

It’s Not Following the Rails Conventions. Will Other Gems Work?

Gems will still work. Let’s say, for example, you use Ryan Bates’ CanCan gem. There’s a code sample in the CanCan README that goes like this:

def show
  @article = Article.find(params[:id])
  authorize! :read, @article
end

It looks like you have to initialize @article, right? But you could rewrite it this way:

def show
  authorize! :read, article
end

def article
  @article ||= Article.find(params[:id])
end

Notice I took the @ off the “article” parameter. It is now a function call to the your helper method that calls Article.find. In this case you’re not avoiding a database call, but you’re not doing any harm either. And your show method is shorter. And sweeter.

Update: In the comments below, in response to Ben’s note about CanCan’s load_and_authorize_resource function, I posted a better way to write your helpers when using CanCan so they don’t run the query or do any authorization until we know the data is needed.

What About Non-Read Operations? Create, Update, Delete?

The new and create methods should still set the instance variable like they did before. update can just call the helper (company.update_attributes instead of @company.update_attributes) and destroy can do the same (company.destroy instead of assigning @company and then calling @company.destroy).

But all views—whether for any of the seven RESTful methods or any non-RESTful method—should just reference the helper method, never the instance variable. Since the helper method is memoized (@company ||= Company.find), it will just work because the instance variable was already set.

I Like Testing

Your tests will need to use @controller.post instead of assigns(:post). Understand, though, that the test might do a DB query by calling post if the view never called it.

Code Sample: the Old Way

Here’s an example CompaniesController that follows the Rails convention of doing the database query in the controller, assigning the result to an instance variable, and referencing it in the view:

# GET /companies/1
# GET /companies/1.xml
def show
  @company = Company.find(params[:id])

  respond_to do |format|
    format.html
    format.xml  { render :xml => @company }
  end
end

And the corresponding show.html.erb:

  <strong>Name:</strong>
  <%=h @company.name %>

<%= link_to 'Edit', edit_company_path(@company) %> |
<%= link_to 'Back', companies_path %>

Code Sample: the Better Way

Here’s how you can rewrite it to avoid bedtime story problems:

helper_method :company

# GET /companies/1
# GET /companies/1.xml
def show
  respond_to do |format|
    format.html
    format.xml  { render :xml => company }
  end
end

def company
  @company ||= Company.find(params[:id])
end

And the corresponding show.html.erb:

  <strong>Name:</strong>
  <%=h company.name %>

<%= link_to 'Edit', edit_company_path(company) %> |
<%= link_to 'Back', companies_path %>

For completeness, here is how you would implement the index method:

helper_method :company, :companies

def index
end

def companies
  @companies ||= Company.all
  # or with will_paginate:
  # @companies ||= Company.paginate :page => params[:page] || 1
end

Thanks

Thanks and apologies to If You Give a Mouse a Cookie and other books in the series by Laura Joffe Numeroff and Felicia Bond. A wonderful series of books that I recommend reading to small children.

Thanks to Struts 2 for helping me discover this pattern. (Yeah. Whoa.)

I’ve pulled his changes into the script on my github repo.

Announcing: http://ilikestuffblog.com

I found a nice little script written by Derrick Childers (scroll down to his comment; don’t use the original shell script) that did almost exactly what I needed. Derrick’s original version exported all events by year, except it didn’t work in all cases because it was based on the year of the photo’s filestamp, which might be different than the year it was taken (e.g. if you went back to retouch a photo). Guillaume Boudreau added support for exporting albums and removed the year folder because it didn’t work well, but I put it back and fixed the it to generate the year based on the event date.

(Update: I changed it to generate more than just the year. I generate the entire date of the event, followed by the event name.)

I tested it with:

• Leopard 10.5.8
• Python 2.5.1
• iPhoto 8.1.2

Derrick tested his original with:

• Snow Leopard 10.6.2
• Python 2.6.1
• iPhoto 8.1.1

I’ve posted it to Github. Feel free to fork and modify to your taste.

It turns out ActiveSupport extends Time and DateTime with a method to do this: change. As the documentation says, “The time options (hour, minute, sec, usec) reset cascadingly, so if only the hour is passed, then minute, sec, and usec is set to 0. If the hour and minute is passed, then sec and usec is set to 0.”

To truncate a Time to the second, minute, hour, or day:

  time.change(:usec => 0) # zero out milliseconds
  time.change(:sec => 0)  # zero out seconds, milliseconds
  time.change(:min => 0)  # zero out mins, secs, & usecs
  time.change(:hour => 0) # zero out hrs, mins, secs, usecs

To truncate a DateTime to the minute, hour, or day:

  datetime.change(:sec => 0)  # zero out seconds
  datetime.change(:min => 0)  # zero out mins and secs
  datetime.change(:hour => 0) # zero out hrs, mins, secs

Sensibly, changing the year, month, or day of a DateTime does not reset the lower elements to zero.

ActiveSupport also adds the change method to Date, but since the only elements are year, month, and day, it never zeroes out the smaller elements when you set a larger element.

One sentence keeps repeating in this essay:

“The proposal was never implemented.”

Pilgrim’s point is that shipping software wins over specs and ideas. Or in his words, “The ones that win are the ones that ship.” (Although he points out that shipping code is necessary but not sufficient for success.) A number of times HTML was “retro-specced” to match what had already been implemented.

I like that. In 2006 I coined the term “Action-Oriented Programming” on a Joel on Software discussion board:

Regarding motivation to get started implementing a product idea once you have one:

Whatever you do, do something.

Us programmers, we tend to overanalyze things. Lots of up-front design, trying to decide if it’s the right thing to do, not sure if we have enough education yet, etc. But from what I’ve read about successful entrepreneurs, they are DOERS. They’re Action-Oriented.

So I’m inventing a new term:

Action-Oriented Programming: writing the damn program instead of thinking about writing it.

The HTML spec evolved through Action-Oriented Programming. Read the History of HTML essay and you’ll see why.

P.S. Hey, I know I didn’t invent the idea of favoring working code over detailed specs. But I like the name I gave it. :-) I’m pleased to see that Action-Oriented Programming has become popular with the rise of agile techniques and “Getting Real.”

Looksee: Clearly Show an Object’s Methods

In Irb when you want to show an object’s methods you might do something like this:

>> print obj.methods.sort.join("\n")

But you end up with a massive list of methods that includes all methods–public, private, and protected–from the bottom-most class up the class and module chain to Object.

Enter Looksee.

>> require 'looksee'
>> lp obj

Now all of obj’s methods are shown color-formatted (to distinguish among public, private, and protected) and grouped by where they appear in the class and module hierarchy. It’s a lot easier to find what you’re looking for.

Get Looksee here.

Awesome_print: Better than Pretty_print or YAML::dump

Here’s one I just learned about today. Often in Irb you want to dump an object to the console to see its attributes. And if you just “print” it the output is hard to read. You may be familiar with pretty_print and YAML::dump as alternatives to make the dump output easier to read. Between those two I prefer YAML::dump because it outputs an attribute on each line so it’s easier to scan the list.

But awesome_print is easier still. The output is nicely indented, formatted, and colorized. When dumping an Array it includes the index number in [brackets] before each element. The index number itself isn’t so important but it shows it in white so it’s easier to find the separation between array elements.

Get Awesome_print here.

Here are some first impressions. I’ve only read lightly about the language so please correct me if anything is factually incorrect.

I’m not super-pumped about using Go, but if I were still using C/C++ every day I probably would be. I now think of Go as C (runs about as fast as C, strongly typed, compiled, pointers) but with a few especially nice enhancements:

• garbage collected (yay!)
• memory-safe – the pointers can’t go out of bounds and I think arrays also can’t go out of bounds if you use “slices”
• extremely fast compiles.

It is missing some of the things I like about C++. E.g.,

• You can’t use integers or pointers as Boolean expressions
• I don’t think it has a convenient syntax for getters+setters with a default backing variable, like Property in C# or attr_accessor in Ruby. (It bugs me how verbose Java is when creating simple bean properties. I hope Go will avoid this problem.)
• I don’t yet see a way to do automatic cleanup when a variable goes out of scope–a feature of C++, C#, and Ruby that I love because it helps reduce code duplication and add robustness

It appears not to be OO, at least not in the way we’re used to. It doesn’t have classes but it does have interfaces. It seems like if you want to make a class you define an interface and then create an instance of it. It doesn’t have traditional inheritance but it does have “embedding,” which is a nice alternative. If you embed someone else’s implementation of an interface into your type, your type now exposes that implementation without having to delegate all the methods one by one.

Lately I’ve been into Ruby, which has different strengths than Go. But if I were to start doing systems-level programming again and needed a good mix of raw performance and garbage collection I would definitely give Go a close look.

So you can do this:

1.in? [1,2]          # => true
3.in? [1,2]          # => false

instead of the slightly more awkward:

[1,2].include?(1)    # => true
[1,2].include?(3)    # => false

Despite its name, in_enumerable doesn’t require an enumerable type.  It uses duck typing to work with any type that has an ‘include?’ method, such as Array, Hash, String, Range, Set, and even Module.

Installation

gem install in_enumerable

Usage

require 'rubygems'
require 'in_enumerable'

Side Note

I noticed that typing ri Object#in? reveals that a similar method supposedly already exists on the Object type. And if you google it you can find the source code for it. (It’s almost the same as mine.) But it doesn’t actually seem to be implemented, at least not in Ruby 1.8.6. So I made my own.

I just read two great suggestions for using Google Analytics to track page views in places where it might normally be hard:

1. How to track a goal that doesn’t end in a unique page. E.g., where the goal is the “Show” page in a RESTful app, but you only want Google to track a goal conversion when you get there through the “create” action.
2. How to track a goal that is reached via AJAX instead of via a full page refresh.

  $.ajax( { url: "someUrl", success: function() {
    // do something impressive with the results
  }, error: function() {
    showError("Ohs noes. Tell me when you fix your AJAX.");
  } } );

But you start using your app and you notice that your error message also appears when you navigate away from the page before an AJAX call has finished, or when you hit Escape to cancel the AJAX call. That kind of sucks—it looks ugly to have the error message appear when there really was no error.

I did a little looking around and found what looks like a cross-browser-friendly way to tell if an error really occurred or if something else happened, like the user navigating away or hitting Escape.

The trick is to check the response headers in the XMLHttpRequest object. If there are no response headers (null or empty string, depending on the browser), the server did not respond yet. That means the user aborted.

Here’s a function that takes an XMLHttpRequest and tells you if the user aborted it.

  /**
   * Returns true if the user hit Esc or navigated away from the
   * current page before an AJAX call was done. (The response
   * headers will be null or empty, depending on the browser.)
   *
   * NOTE: this function is only meaningful when called from
   * inside an AJAX "error" callback!
   *
   * The 'xhr' param is an XMLHttpRequest instance.
   */
  function userAborted(xhr) {
    return !xhr.getAllResponseHeaders();
  }

And here is the updated application code to call this new function. You can easily adapt this to another JavaScript library than jQuery:

  $.ajax( { url: "someUrl", success: function() {
    // do something impressive with the results
  }, error: function(xhr, textStatus, errorThrown) {
    if (!userAborted(xhr)) {
      showError("Ohs noes. Tell me when you fix your AJAX.");
    }
  } } );

Oh, did that “textStatus” and “errorThrown” catch your eye? I already looked at those. You can’t use them to tell if the user aborted. They return the same values whether the user aborted or the server returned with an error. (I only tested a 500 error.)

Refraction to the rescue.

Refraction is a new Rails plugin from Josh Susser and Pivotal Labs that helps you easily implement URL rewrites in Ruby, rather than writing the rules in web-server-specific lingo. Your code gets called by Rack, although you don’t need to know anything about Rack to write rewrite rules.

Here’s an example. Let’s say you want to duplicate Twitter’s /@username routes (did you know Twitter supports URLs like that?) but have found as I did that routes.rb doesn’t seem to work with a “@:username” rule. Instead, create a RESTful UsersController and then put this in your refraction_rules.rb file:

Refraction.configure do |req|
  # rewrite /@username as /users/username
  if req.path =~ %r{/@(.+)}
    req.rewrite! :path => "/users/#{$1}"
  end
end

The notes in the blog post (see link above) mention that there is already a Rack rewrite plugin called Rack::Rewrite but “Rack::Rewrite only really gives you access to the request URI, whereas Refraction appears to give the entire request object… that can be much more useful for determining where to send someone.”

See the “Creating your own wrapper tokens” section of the tutorial link above to see what I’m talking about. It didn’t work before but it does now. Or will, as soon as it gets pulled back into the source. Until then you can get it here. After it’s pulled back in I’ll probably delete my fork.

Update: my change was pulled back into the master.

The change is called “Pluggable JSON Backends” and it was implemented by Rick Olson, a.k.a. Technoweenie. (I have the utmost respect for Technoweenie. I’ve used his code before and it’s wonderful stuff—it’s clean, it’s simple, and it works.)

Instead of this:

my_model.to_json

we are now encouraged to do this:

ActiveSupport::json.encode(my_model)

I Like Stuff that’s Simple

It’s a little thing and I don’t want to blow it out of proportion, but this is a pattern I would like to see the community avoid. Once people see this style of coding they start to copy it.

I think I understand the motivation behind the change, but I believe the new style of coding is harder to read and write. It reminds me of all the years I wrote C++ code and wished I could just extend a class outside its original definition. Ruby can do that and I love the fact that Rails isn’t shy about taking advantage of this capability to make programmers’ lives easier and code simpler.

It may be too late to have this debate when it comes to JSON. Maybe this was the only way to meet the requirements Technoweenie had. I don’t know. And I understand sometimes it’s necessary to do something a little ugly to achieve a goal that’s more important than simplicity or readability. But I am writing today to encourage the Rails core committers, and the rest of us in the Rails community, to please avoid this pattern when possible. If you are considering writing a method that looks like this:

ClassOrModule::namespacey_function.method_name(obj)

please pause and consider whether the users of your API would find it easier if you instead made it look like this:

obj.method_name

Thanks!

Before we continue, a word of caution: I’ve mentioned in this blog and in the Ruby on Rails Podcast that even though multithreading is really fun to think about and play with, I approach it with reluctance in real-life projects because it makes the code more complex, makes it a lot harder to debug problems, and is hard to manage when there are multiple programmers who all have to work in and understand the threaded code. But there are still some problems for which threading is the right solution.

I Like Stuff That’s Clean and Small

Dataflow is a beautiful bit of programming. It’s small, clean, and tested. It implements write-once variables with automated thread synchronization in just 52 lines of code. (Plus 120 lines of tests.) It supports:

• instance variables
• local variables
• dynamic values loaded into data structures such as arrays
• It doesn’t seem to support class variables but I guess a constant can serve as a write-once class variable.

Here are some code samples, copied from the README.

# Local variables
include Dataflow

local do |x, y, z|
  # notice how the order automatically gets resolved
  Thread.new { unify y, x + 2 }
  Thread.new { unify z, y + 3 }
  Thread.new { unify x, 1 }
  z #=> 6
end

# Instance variables
class AnimalHouse
  include Dataflow
  declare :small_cat, :big_cat

  def fetch_big_cat
    Thread.new { unify big_cat, small_cat.upcase }
    unify small_cat, 'cat'
    big_cat
  end
end

AnimalHouse.new.fetch_big_cat #=> 'CAT'

# Data-driven concurrency
include Dataflow

local do |stream, doubles, triples, squares|
  unify stream, Array.new(5) { local {|v| v } }

  Thread.new { unify doubles, stream.map {|n| n*2 } }
  Thread.new { unify triples, stream.map {|n| n*3 } }
  Thread.new { unify squares, stream.map {|n| n**2 } }  

  Thread.new { stream.each {|x| unify x, rand(100) } }

  puts "original: #{stream.inspect}"
  puts "doubles:  #{doubles.inspect}"
  puts "triples:  #{triples.inspect}"
  puts "squares:  #{squares.inspect}"
end

It doesn’t take long to read the Dataflow code (it’s only 52 lines, after all) but it did take me a while stepping through it in the NetBeans debugger to wrap my head around how it works. Also the name of the variable-assignment method is a unintuitive to me. Assignment is done by calling the unify method. Apparently this name comes from the concept of unification, which I think means: provide a bunch of algorithms whose variables have dependencies on each other and let the system work out the dependencies and execute the algorithms in the correct order to assign values as they are needed. Anyway, using a method called unify for assignment takes a little getting used to.

Note: Larry’s README says Dataflow was inspired by the Oz programming language, not the Erlang programming language. But I’m more familiar with Erlang so that’s what I can compare it to. The primary difference between Ruby-with-Dataflow and Erlang is that in Dataflow you declare a variable and then assign it a value, whereas in Erlang you have to assign at the moment you declare it. That’s how Erlang makes variables write-once: if you can only assign a value when you declare a variable, obviously it will only be assigned once. Dataflow lets you assign to the same variable multiple times but raises an error if you assign different values, so it’s equivalent to write-once. (It uses the != operator to decide whether the values are equal.)

Interop with “Normal” Ruby

The README also says, “The nice thing is that many existing libraries/classes/methods can still be used, just avoid side-effects.”

It’s true that you can write a program that uses Dataflow for some variables but also interops with non-Dataflow code as long as that code is thread-safe. I’m not quite sure what he meant by “just avoid side-effects.”

But How Do You Assign New Values to Variables?

If a variable can only be written once, what do you do when you need to change it? Obviously programs need to deal with this. For example, what if you need to loop over an array and keep track of the index as you go? Erlang handles it by heavy use of the stack and threads, so whenever you need a new value you call a function (which spawns a thread) and the function declares a new variable, assigning the new value to it. So there’s a lot of copying of values.

In Ruby with Dataflow I imagine you would do something similar: either call a function or spawn a thread for each iteration, passing in the current value, and have the function or thread declare a new local variable which is value+1. This style of programming takes some time before it becomes natural. It’s not yet natural for me.

There could also be performance implications. Erlang’s interpreter optimizes tail recursion and converts it to an iteration (really a GOTO) under the hood so the stack doesn’t blow. I don’t know if any Ruby interpreters do that. As of a few years ago they didn’t, according to my Google search. Johannes Friestad wrote in 2005, “Recursion, tail or no tail, works just as well as any other method call in Ruby. Plenty of thrive without optimizing for tail recursion, Java is one of them. The combination of a small stack and lack of tail recursion optimization does mean that in Ruby, recursion can hardly replace every other looping construct the way it can in Lisp. You’ll be the judge of whether that is important.”

Update: Larry writes in the comments that “this library makes JRuby shine over MRI due to its green threads + native thread pool implementation.” I’ve only used MRI and I didn’t know about that aspect of JRuby but it’s pretty nice. It sounds like if you’re going to use Dataflow you might want to use it with JRuby rather than MRI.

Possible Concerns

Dataflow is really cool but I do have a few potential concerns about it:

1. Even though Dataflow makes it easier to write thread-safe code, it doesn’t fix the fact that it’s hard to debug multithreaded code. Stepping through multithreaded code in a debugger is complicated, especially when the code switches thread context on the fly.
2. Speaking of debugging, if the debugger tries to show you the value of a Dataflow variable that hasn’t yet been assigned, the debugger thread itself will be put to sleep. In NetBeans this means the “locals” pane stops working (but you can still debug) and if you hover the mouse over an unassigned variable, you don’t see anything in the tooltip. In rdebug it’s worse–if you eval a variable that doesn’t yet have a value, rdebug hangs because its main thread gets put to sleep.
3. You can’t assign nil to a Dataflow variables because nil is used to indicate that it hasn’t yet been assigned. I would like to be able to assign a value of nil and have that be different from “unassigned.” This would be a pretty easy fix to make to Dataflow without bloating memory–all unassigned variables could reference the same constant:
   UNASSIGNED = Object.new
I removed this concern because it’s been fixed. Dataflow now differentiates between nil and unassigned.
4. Memory overhead: Dataflow is as efficient as possible with memory usage but it does incur some overhead on each variable. Compared to unthreaded programming, it is a lot. But compared to manual thread synchronization it’s probably about the same amount of memory you would have used for synchronization data structures anyway. It depends on how you do your manual synchronization. Each variable has, in addition to its value:
   1. a Mutex
   2. an Array (initially empty) of references to Threads that are waiting for it to be initialized
   3. a Monitor condition to wake up the Threads that are waiting for it to be initialized
   4. a Boolean to track whether it has a value yet (but cleverly, this boolean doesn’t get assigned until the variable is assigned, which saves some memory)
5. More than the overhead per variable, I wonder about the memory overhead of constantly copying values rather than reassigning them. If the stack gets too deep you could run out of memory from all the copying. (See my description of looping above.) You also make the garbage collector work pretty hard. If you loop by spawning threads instead of using recursion, you incur a lot of overhead since threads are expensive compared to function calls. This is why Erlang’s interpreter has its own threading system instead of using the one in the operating system–threads have to be as cheap as function calls. In Ruby they are not.
6. Related to memory overhead, I wonder about the performance overhead. In addition to deep stacks and lots of threads, every time you call a method on a variable it gets routed through method_missing and Mutex.synchronize even if you have already called that method on that variable. (It does this so its method_missing override can put your thread to sleep until the variable has a value.) This could be expensive but it’s impossible to know for sure without profiling it. If it turns out to be a problem, method_missing could rewrite itself the first time after the variable gets assigned a value so all subsequent calls don’t have to be synchronized.

That reads like a pretty big list of concerns but without actually using Dataflow I can’t tell how many of them will actually cause problems. I still think it’s cool. :-)

Try it if You Need Threading

I mentioned at the beginning that I’m cautious about using threads but there are some problems for which they are the right solution. Next time I’m confronted with such a problem on a Ruby project I will drop in the Dataflow gem and give it a try. It looks like a pretty good way to do threading in Ruby.

See the big honking warning at the bottom for an explanation of why this plugin doesn’t just override the behavior of link_to_remote.

I Like Stuff that’s SEO-Friendly

The following example shows a “Next” link in paginated output.  Clicking the link in a browser results in an AJAX call (using the POST method) that retrieves just the “page” partial and inserts it into the “results” div on the page with a highlight visual effect.  When a search engine sees the link, however, it will send a GET request to the same URL, and the entire page (not just the partial) will be sent in the response.

Putting this in the view (home/index.html.erb):

<div id="results">
  <%= render :partial => "page" -%></div>
<%= link_to_seo_remote "Next",
  { :update => "#results",
    :url => { :action => "next_page" },
    :complete => visual_effect(:highlight, "#results") } %>

Produces (pay attention to the href attrbute):

<div id="results">
  <!-- first page of results shown here --></div>
<a href="/home/next_page"
  onclick="new Ajax.Updater('#results', '/home/next_page',
  {asynchronous:true, evalScripts:true,
  onComplete:function(request){new Effect.Highlight(&quot;#results&quot;,{});}}); return false;">
  Next
</a>

In  the controller (home.rb), render just the partial if called in an XHR (AJAX) request:

def next_page
  if request.xhr?
    render :partial => "page"
  else
    # Render the entire page, including the "results" section.
    render :action => "index"
  end
end

WARNING ABOUT INCORRECT USE OF THIS FUNCTION

Sorry but I have to yell for emphasis here.

When Google crawls your site it will follow all links on a page in advance, even before the user clicks on them.  Adding :confirm => “Are you sure?” WILL NOT HELP because it generates JavaScript that Google doesn’t execute.  So when you use link_to_seo_remote, DO NOT ALLOW destructive links to be placed in the href attribute.  Instead, override html_options[:href] to link to an intermediate page with “Are you sure?” and a BUTTON (not a link.  The crawler will not click the link, so the data will not be deleted.

See Using Rails AJAX Helpers to Create Safe State-Changing Links and search the page for “request.post?” for an explanation and some sample code.

Does it Have Tests?

Why, yes. I’d like to thank the Rails Community for not tolerating code with no tests. It was soooo tempting just to release this without writing automated tests but the peer pressure got to me.

And I’ll also like to thank Cake for awesome music.

To get the code

ruby script/plugin install http://github.com/BMorearty/link_to_remote_with_seo.git

Check it out. See if you can figure out what the program does before you follow his link to the answer.

(This snippet is just a taste of it.)

Here are some some choice quotes from the talk, along with my own commentary.

“All programmers want to rewrite the code they’re forced to maintain. They’re almost always wrong.”

Man, is that ever true. (Note that he said almost always. After all, his talk is about a successful rewrite.)

I’ve seen it again and again. Programmers tend to believe the code they’re maintaining (that someone else wrote) sucks and they could write it much better. Often that’s because they haven’t taken the time to understand the code base. As Joel on Software says, “It’s harder to read code than to write it.” I think usually (but not always) the cost of rewriting it far outweighs any benefits. What you’d typically end up with after a rewrite is:

• A few years have passed
• You’ve spent a ton of money on the rewrite
• The app still has bugs–just a different set of bugs. (Another quote from the Joel article: “The idea that new code is better than old is patently absurd.”)
• A new generation of programmers will join the team soon. They will complain that the code base sucks and needs to be rewritten.

Having said that, I know there are times when a rewrite is the right thing to do. But that’s a discussion for another day.

Something I think his team did correctly: they made a goal of finishing the rewrite in four months, not two years. A massive two-year rewrite has an extremely low chance of succeeding.

“EJB3 is a whole big boxcar full of crazy.”

Now that’s just funny. (He said that after saying EJB3 is much better than earlier generations of EJB, by the way.)

“At this point our performance architect will maintain that Apache is unsuitable for use in any production web serving environment, in general. (And only nginx with its polling model is the right way to go.)”

I don’t agree but it’s a great quote.

“I actually kind of like the thread-unsafety of Rails. I mean it simplifies the programming model quite a bit for simple web sites. You know: I’m handling one request; I understand how to scale that.”

I totally agree with that. As someone who loves writing software, I think threading is fun and awesome and there are situations where it’s a must–I once even thought about writing a book about threading on Win32. When I was first introduced to Ruby on Rails I had a kneejerk “are you kidding me?” reaction when I heard it wasn’t thread-safe. But I’ve since formed the opinion that single-threading is really nice when you can get away with it because of its simplicity. It helps developers focus on the task at hand rather than spending a lot of time debugging threading problems. In a multi-threading environment it’s too easy for developers who understand threading to introduce code that then gets broken by other developers–and it’s too hard to write tests that will catch the breakage the moment it occurs.

By the way, the speaker’s next sentence was “Obviously our fast service-side application is multi-threaded and we have good benefits from that.” So he’s not saying multi-threading should never be used.

“Testing was a big part of the decision. You know, that was actually one of the things which drew me so strongly to the platform once I started understanding it. I had spent years myself as a Java developer trying to figure out how in the heck to use JUnit to do anything useful on my web site. And maybe that was just a failure of imagination on my part, but when we started looking at Rails we didn’t have to figure it out. It was obvious how to test each level. Both the unit tests for the models, and the functional tests and the integration tests. It was all there in the framework. And not only was the framework built to make it easy, but the community expected it. You know, I’ve never seen a development community that was so involved and oriented towards writing test code–writing test automation–than this one. And so that was a big part of our decision.”

So true. I have found that when it’s obvious how to write effective tests and where to put them, I will write tons of tests. If the framework greases the wheels of test-writing and make it pain-free, I will write a lot more and better tests. Rails does a lot better at this than other frameworks I’ve used, although I still think it could use improvement. And I love the emphasis placed on automated testing in the Rails community.

Well, that’s it. To see the whole talk, go to http://www.infoq.com/presentations/straw-yellowpages. And enjoy the grouchy comments by Java developers below the video.

Sprinkle unclosed HTML tags and apostrophes all over your fixture data and test code.

Then use assert_select liberally, which will barf on the console if it sees unclosed HTML tags–even if you were selecting some other part of the document.

I Like Stuff that’s Safe

Here is what a posts.yml file might look like:

test_post:
  id: 1
  subject: <script> attack!
  detail: "sql injection: '; drop table posts;"

(If you use an apostrophe in YAML you have to quote the whole string.)

So assert_select has this handy side-effect I mentioned where it tells you about your malformed HTML. Since Rails tests don’t actually run in a browser, you need some other way to know that you’ve forgotten to escape data. Unclosed HTML tags in your fixtures, yeah, that’s the ticket.

And remember, you don’t need to call assert_select on the element that contains the bad data. Just call assert_select on anything and it will parse the output to make sure it’s well-formed.

  def test_show
    post = posts(:test_post)
    get :show, post.id
    assert_select "body"
  end

The idea is that by sprinkling XSS attacks through your fixtures and using assert_select whenever you’re testing other stuff, the XSS attacks will become apparent.

If you do need to assert that the output is correct, you can call CGI::escapeHTML:

  def test_show
    post = posts(:test_post)
    get :show, post.id
    assert_select "span", :count => 1,
      :text => CGI::escapeHTML(post.detail)
  end

I can’t haz SQL injection attacks

I admit that putting SQL injection attacks in the fixtures is a bit contrived and may not help. A better way to catch SQL injection attacks is to pass apostrophes into the app from your test code, so go ahead and sprinkle your test code with beauties like this:

  def test_update
    post :update, posts(:test_post).id,
      :detail => "sql injection: '; drop table posts;"
  end

The secret to making this work is:

1. apostrophe
2. semicolon
3. SQL statement
4. another semicolon

You want to use a SQL statement that will cause a test to fail. It would be coolio if there were some way to make the current test succeed and subsequent tests fail, but I’m not sure I know a way to do that consistently. But at least if you use a “drop table” statement, you’re going to cause subsequent tests to fail (if there are any subsequent tests that use that table) because a schema change does not happen in a transaction. So even if you’re using transactional fixtures, the next test will fail anyway cuz the dang table is gone.

In an attempt to better understand instance_eval and class_eval, I just read Khaled’s post on Ruby reflection. It helped, and I came up with a memory crutch I can use to remember when to use each of them:

Use ClassName.instance_eval to define class methods.

Use ClassName.class_eval to define instance methods.

That’s right. Not a typo. Here are some examples, shamelessly stolen from his post:

# Defining a class method with instance_eval
Fixnum.instance_eval { def ten; 10; end }
Fixnum.ten #=> 10

# Defining an instance method with class_eval
Fixnum.class_eval { def number; self; end }
7.number #=> 7

I Like Stuff that’s Backwards

Why is it the reverse of what you might expect? Because Fixnum.instance_eval treats Fixnum as an instance (an instance of the Class class), thus any new functions you define can be called on that instance. So it’s equivalent to this:

class Fixnum
  def self.ten
    10
  end
end
Fixnum.ten #=> 10

Fixnum.class_eval treats Fixnum as a class and executes the code in the context of that class, thus any “def” statements are treated exactly as if they were in normal code without any reflection. It’s equivalent to this:

class Fixnum
  def number
    self
  end
end
7.number #=> 7

There are still some things about Ruby reflection that mystify me but at least I think I’ve got this one nailed.

class MyModel < ActiveRecord::Base
  usesguid
end

Pretty nice.

Just one problem.

It’s HECKA slow.

On my Windows machine it was taking a whopping 0.4 seconds to create a GUID with this plugin. On my Linux VM it was a lot faster, but still slower than it should be (0.0322 seconds–just 31 GUIDs per second).

Download the Faster Plugin

If you use MySQL for your database and you’d like to download my modified usesguid plugin which is way faster, type this from the main directory of your Rails app:

 script/plugin install git://github.com/BMorearty/usesguid.git

Or download it here and copy it into vendor/plugins/usesguid.

Then add the “usesguid” statement (see above) to any models that you want to have guid ids, migrate the id columns to binary varchar(22), and add this to your environment.rb file:

ActiveRecord::Base.guid_generator = :mysql

Here is a sample migration for creating a new table with guids, as opposed to changing an existing one to use them:

create_table :products, :id => false, :options => 'ENGINE=InnoDB' do |t|
  # This table uses guid ids
  t.binary :id,   :limit => 22, :null => false
  t.string :name, :limit => 50, :null => false
end
# Since the t.column syntax can't specify a character set and collation...
execute "ALTER TABLE `products` MODIFY COLUMN `id` VARCHAR(22) BINARY CHARACTER SET latin1 COLLATE latin1_bin NOT NULL;"
execute "ALTER TABLE `products` ADD PRIMARY KEY (id)"

I Like Stuff that’s Fast

Read on to find out why the old code was so slow, and how the code got 2100 times faster.

I investigated to see why it takes so long, and found that every time it creates a GUID, it calls UUID.timestamp_create. This in turn calls UUID.get_mac_address, which spawns a new process (ipconfig on Windows; ifconfig on UNIX-based systems) and parses the output. The reason: to discover the network card’s MAC address. (Hey yeah, even Windows has a MAC address.)

But the MAC address never changes. It’s hard-wired into the network card. So why bother querying it every time you create a GUID? Launching a whole new process every time we need a GUID is overkill.

My first thought was to write a plugin on top of the plugin. My plugin would cache result of UUID.get_mac_address. I tried it, but found a problem: there’s a bug in UUID.timestamp_create. If it executes too quickly on a system whose clock resolution is not high enough, it returns the same GUID multiple times in a row. Whoops! Kind of defeats the purpose of GUIDs.

So I decided to take advantage of the fact that MySQL has a “SELECT UUID()” syntax, and I wrote a new GUID creator in the UUID class that calls MySQL to generate GUIDs. (Obviously this only works if you have MySQL.) I called this new creator “UUID.mysql_create.” The first time it is called, it calls MySQL like this:

SELECT UUID(), UUID(), UUID(), UUID(), UUID(), ... ;

It selects 50 UUIDs in a single round-trip to the database and stores the results in memory. Each time a new GUID is required, it plucks one off the list. When the list is empty and another one is required, it goes and gets another 50.

On my Windows machine, creating a GUID with UUID.mysql_create now takes 0.0001937 seconds, which is over 2100 times faster than the 0.4 seconds it used to take. On my Linux VM it’s 0.0001671 seconds, or 193 times faster than the 0.0322 seconds it used to take.

All these changes were made in a new file, uuid_mysql.rb. But I also made a number of changes to the usesguid.rb file:

1. Added a configuration option so you can specify which creator to use. The default is still timestamp_create, but to use mysql_create you just put “ActiveRecord::Base.guid_generator = :mysql” in your environment.rb file.
2. Fixed the code so it respects the :column option, which lets you override the column that stores the primary key.
3. Delayed the assignment of a guid until just before creation (before_create) rather than just after “new” (after_initialize). This has two benefits:
   1. It more closely mimics the default behavior of autoincrement columns, which doesn’t assign an id until after creation
   2. It is faster. After_initialize gets called every time a model object is instantiated, including all objects return by a call to find. (But don’t worry, it wasn’t generating GUIDs for all those objects; it was just being called and bailing out when it saw there was already an id).  Before_create only gets called for newly created model objects.

I thought about making it even faster by calling CoCreateGuid() on Windows and calling a UNIX C function to create a GUID when on UNIX, but it’s so fast now that it hardly seemed worth the extra effort and the extra platform-specific code.

So that’s it. Enjoy it!

When a Rails test fails, I look for it in test.log to see if there are any clues there. But it’s pretty hard to find the portion of the log associated with the test that failed. In this sample section of a log, where does the processing begin for the test called test_should_require_email_on_signup?

Which test is which? Where does my test start?

It’s hard to find. Now imagine running rake on all your tests and sifting through the whole test.log looking for one test whose name you know, but the test name isn’t in the log.

So the other day I wrote a bit of code in my test_helper.rb file to make the log a lot easier to sift through. Here’s what the above log looks like with this code in place:

Ooh, nice-n-clear

Ahh, that’s more like it. Now it’s easy to tell where test_should_require_email_on_signup begins. If you scroll up and look at the first log again, you’ll see that there isn’t even a blank line separating that test from the previous one. (See how the test starts on the SELECT count(*) statement?)

Here’s the code. Drop it into test_helper.rb for your Rails project. To me this seems like a nice little example of how Ruby’s open classes can benefit developers (while understandably considered harmful by some). In a language without monkey patching, I would have to resort to something more painful like changing all my tests to be derived from my own subclass of TestCase, and put this code in that class.

Enjoy!

class ActiveSupport::TestCase
  # This extension prints to the log before each test.  Makes it easier to find the test you're looking for
  # when looking through a long test log.
  setup :log_test

  private

  def log_test
    if Rails::logger
      # When I run tests in rake or autotest I see the same log message multiple times per test for some reason.
      # This guard prevents that.
      unless @already_logged_this_test
        Rails::logger.info "\n\nStarting #{@method_name}\n#{'-' * (9 + @method_name.length)}\n"
      end
      @already_logged_this_test = true
    end
  end
end

P.S. I didn’t spend the time to figure out why my callback was being called multiple times for each test. I just inserted the guard you see in the code above to prevent the same test title from being shown multiple times.

Drop me a line in the Reply section below and let me know what you think–especially if you’ve figured out why each one is called multiple times when running from rake or autotest.

Updated 6/18: I corrected the code above because WordPress automatically inserted a “mailto:” tag when it saw the @ sign.

Updated 2/26/10: I changed Test::Unit::TestCase to ActiveSupport::TestCase.

You wanna do that in your Rails app that runs with Mongrel? I show you how. Sit down. And along the way I’ll show what I learned about writing custom Mongrel HttpHandlers and why you shouldn’t store instance variable in them.

I remember seeing it there long ago when Google was new. I liked it because:

1. Faster is better
2. It shows Google focuses on helping me go fast
3. It reminds the Google developers to focus on helping me go fast

So I’m developing this awesome web site now and I’m using Ruby on Rails with Mongrel. I wanted to pull a Google and show the server response time as text content in my pages, as a reminder to myself and my co-developer that it’s super-important to keep things fast.

I can haz question

How do you put response time in a page using Rails?

So I did some research. My first inclination was to create an around_filter in my ApplicationController. In fact the documentation for ActionController::Filters even uses a BenchmarkingFilter as an example.

But if I put a filter there what would it benchmark? Just the time spent in the controller and view? But what about the time spent in Rails outside my controller and my view? I want a benchmark that’s higher up the chain, kinda close to where the request is received and the response is sent back, so the time displayed in the page gives a truer picture of how long it took to handle the request.

More importantly: let’s say my server gets backed up with a queue of requesteses (through my amazing connections I get on the Yahoo! home page and there are more simultaneous requests than I have request handlers). I’d like to profile the total time from when each request first came on board until I said buh-bye to the response. If I use an around_filter I’m not including a crucial piece of the response time: how long my user waited in line before even entering Rails because my server was serving someone else first.

If you’re at the ball game and you go get a hot dog and there are 30 people in front of you in line, are you excited that it takes only 1 minute to serve each person? Hell no. You’ll be there 31 minutes, including the time to serve you. You’ll miss two innings and the game-winning homer by Mr. Steroid. 31 minutes is the response time that matters because that’s how long the customer is waiting.

Plus you know you always get stuck in the slow line so it’s more like 50 minutes.

So that’s when I realized that I was asking the wrong question. The question isn’t how do you put response time in a page using Rails.

The question I shoulda had

How do you put response time in a page using Mongrel?

Mongrel sits higher up the food chain than Rails. It’s the little servy thing that doles out HTTP requests to ActionController.

I did a little digging and found that Mongrel has something called handlers, also known as filters.

Handlers are how Mongrel handles requests. When Mongrel gets a request it runs through all its handlers, passing the request to each one. (It calls them all; by default it doesn’t stop when one of them handles the request.) Handlers are not just for extensibility–they’re the way Mongrel works. There’s a RailsHandler that comes with Mongrel. That’s the one that passes requests off to Rails. There’s also a DirHandler. In a Rails app it uses that one to handle requests for files in the public/ folder. There are other handlers for other things. Like a CampingHandler for Why the Lucky Stiff’s Camping.

Each handler has a process method that Mongrel calls to process requests.

Each handler also has an optional request_begins method. For each request Mongrel calls the request_begins methods in all the handlers first, then it calls the process methods in all the handlers.

But here’s the sweet part. Rails may be single-threaded but Mongrel ain’t. Mongrel calls request_begins on all the HttpHandlers the moment it sees a new request. This happens in a separate thread from the one that’s currently processing a request in Rails (or some other handler), so it does not even have to wait for the current customer to be served before starting the timer on the newly added customer.

Go doggy. The stopwatch starts as soon as someone gets added to the back of the line.

By the way, Mongrel won’t call request_begins unless you tell it to. Here’s how you tell it to. Put this inside your subclass of HttpHandler:

  def initialize(options={})
    @request_notify = true
  end

Cool. I can create a new BenchmarkFilter class a subclass of HttpHandler. I check the time in its request_begins method–which gets called before any of the process methods (including the RailsHandler’s one)–and check the time again in the process method for my handler. Mine will get called after the Rails one if I install it at the end.

class BenchmarkFilter < HttpHandler
  def request_begins(params)
    # stuff goes here
  end

  def process(request, response)
    # and here
  end
end

Technically my filter won’t be called quite early enough because some other filters might be called first, and some other “process” methods might be called later. But I’m not going to stress over that.

About those nasty instance variables

The natural thing to do is store the start time in an @instance variable or maybe a @@class variable but I discovered this doesn’t work well in HttpHandlers. For performance reasons Mongrel only creates a single instance of your HttpHandler subclass instead of creating a new instance for each request. So using an instance variable is the same as using a class variable. Not only that but remember I said Mongrel is multi-threaded? That means your one HttpHandler instance will be called simultanously on lots o’ threads. Start reading and writing instance variables and you’re asking for a quick trip to the happy laughy farm.

(The @request_notify variable above is ok because we set it once during initialization–before all the thread craziness starts–and never change it.)

I also noticed in my testing that sometimes the request_begins methods were getting called several times in a row before the process methods for the same requests. Again that’s because it’s multi-threaded. But just think how it messes with your instance variables. Even if you guard the reads and writes with a Mutex, your start times would get all mixed up because you’re keeping track of a bunch of simultanous start times in a single variable.

When Mongrel calls request_begins the only parameter it passes in is the request parameters. It doesn’t even pass the request because the request hasn’t finished initializing yet. I needed a way to associate the start time with the request and pull it out later so I could see how much time elapsed. Since the parameters passed to request_begins are a reference to the request’s @params variable, any changes made to the parameters will be carried along with the request. So here’s what I did:

  # Define a const for performance
  HANDLER_START_TIME = "X-Handler-Start-Time".freeze

  def request_begins(params)
    params[HANDLER_START_TIME] = Time.now
  end

This code adds a custom key to the parameters hash, and in the value it stores the start time. As the comment mentions, I froze the name of the handler to get slightly better performance. I stole that trick from the Mongrel source code.

Later on, when my process method is called with the request as one of the parameters, I can pull the start time out of request.params:

  def process(request, response)
    elapsed_time = Time.now -
      request.params[HANDLER_START_TIME]
    # I'll add more in a sec
  end

You put the time in the coconut

The next question: how do I put the time in the content of a page? The page has already been rendered by the time my handler gets called, so I can’t set an @elapsed_time variable and render it in my view.

Here’s the approach I took: in my views I output a string of characters that the BenchmarkFilter looks for. If it finds those characters it replace them with the elapsed time. If not, it does nothing.

I wanted it also to have no effect at all if it’s running without the BenchmarkFilter handler. Let me rephrase that with lots of negatives for clarity: I didn’t want it not to have no effect if it didn’t run without having no BenchmarkFilter handler.

My first idea was to create a partial file called _response_time.html.erb in a “shared” folder. The partial’s contents are:

<span id="response_time"><span>

In the handler I look for this in the response string and replace it with something like:

<span id="response_time">(0.15 seconds)<span>

Wherever I want to show the response time in a page I do this:

<%= render :partial => "shared/response_time" %>

If you want to put the output in an HTML comment you can do that too, so it’s not shown on the page but can be inspected by looking at the source.

Why no worky?

I ran it and it seemed to work. Most of the time. But it was cranky. Sometimes it worked and sometimes not. I debugged it and noticed why. Do you know? See if you can guess from these hints:

Hint #1

The HTTP headers had already been rendered. (By the RailsHandler) before my handler is called. Think before you go to hint #2.

Hint #2

One of the HTTP headers is called Content-Length.

Hint #3

I was changing the length of the content without changing the Content-Length header.

As the voice in the “create a weblog in 15 minutes” video would say: “wup!”

Kevin Spacey * 15

I started down the path of trying to fix the Content-Length header on the fly but that got frustrating so I tried something easier: change my partial so it outputs exactly the same number of characters whether the BenchmarkFilter class replaces it or not.

I decided to format my output like this:

  (0.15 seconds)

That takes 14 characters. Add another one in case it’s 10.00 seconds or more (yikes) to make 15. So I changed the span in the partial to have 15 spaces in it:

  <span id="response_time">               </span>

Then I wrote the BenchmarkFilter code so it always uses 15 characters. If it’s <9.99 seconds I insert an extra space between the number and the word “seconds.” The browser eats that space anyway so it looks fine. And if it’s >99.99 seconds, God forbid, I just output “(>100  secs)” (again there’s an extra space in there to make it 15 chars), which looks like “(>100 secs)” in the page.

Handling RJS responses

I decided I also wanted to update the response time partial when executing an RJS response. Here’s how the RJS looks:

page.replace "response_time",
  '<span id="response_time">               </span>'

(This works but I’m not happy with it. I shouldn’t repeat the special string. I really ought to move this to a constant in the application helper.)

With an RJS the HTTP Handler sees JavaScript, not HTML, in the response. So instead of:

<span id="response_time">               </span>

the output has all the special characters quoted, so the handler has to search for:

\74span id=\\"response_time\\"\76               \74/span\76

I also noticed that on newer versions of Rails, at least on my friend’s Mac, the response was in Unicode:

\\u003Cspan id=\\"response_time\\"\\u003E               \\u003C/span\\u003E

Anyway, if you want to see the hairy details you can look at the code.

Installing the handler

If you want to run Mongrel and tell it to execute a particular Ruby file at startup, one way is to use mongrel_rails with the -S option:

mongrel_rails start -S lib/http_handlers/benchmark_filter.rb

(I decided to put the handler file in lib/http_handlers).

At the end of the file, after closing out the class, you have to call the “uri” function to tell Mongrel to install the handler:

# This calls Mongrel::Configurator::uri.  The first
# argument is a prefix.  Passing "/" means we want
# this handler to be executed for all requests.
uri "/", :handler => BenchmarkFilter.new

How it looks

So you want to see what the output looks like? Here’s a picture:

Ta-da!

Obviously this won’t work with other servers than Mongrel, so if you want to support any HTTP server you’ll have to figure out another technique or use an around_filter and lose the waiting-in-the-queue time in the output.

And it won’t work with static pages that you cache in something like nginx.

You can download the code here. Feel free to customize it. But remember: don’t change the string length when doing the substitution.

4/11/09 Update: Rack

Now that Rack is on the scene, it wouldn’t be too hard to rewrite this thing to work with any web server, including Phusion Passenger.

But I haven’t rewritten it. If you feel like rewriting it, go for it–and then post a comment below so people can find it.

Are you ready?

Here goes:

Every teenage boy has a Zombie Escape Plan.

That’s what my niece told me. She was serious. And she thought it was just as weird as I do. (She doesn’t have one.)

Here’s how she found out about it. One day she was listening as two male friends of hers were comparing zombie escape plans. This was new to her. “Does every guy have a zombie escape plan?” she asked them.

“Well duh,” they both said, dead serious.

So she ran an unscientific survey of her teenage male friends to find out if it was true. And guess what.

It was.

Every boy she asked said yes, naturally he has a zombie escape plan.

By now I was busting up. I told her well, at least I have a fire escape ladder in my 2-story house. I could use that as my zombie escape plan too. Her dad (my brother-in-law) said no: that’s lame. A fire escape plan does not serve as a zombie escape plan. As evidence he pointed to his own zombie escape plan: he will dance a jig. Because everyone knows a zombie cannot resist dancing a jig if he sees someone else doing it. But it doesn’t make a very good fire escape plan.

My niece said her dad was right. One boy in her survey said his zombie escape plan involved climbing up on the roof, which is usually not a good idea in a fire. At this point her younger brother, who’s also a teenager, piped up and said that after all, his zombie escape plan is to use a flame thrower. And that never makes a very good fire escape plan.

When I got home I googled it and found that there is even a web site dedicated just to this (side note: what did we ever do before the web?): http://www.zombieescapeplan.com. Except it’s made by a girl. So I guess at least that’s good because when the zombies attack some of the girls will be prepared.

I need a good zombie escape plan. What’s yours? Please comment below so I can get some good ideas.

If you’re like me, when you started coding in Ruby last year you found the “case” statement intriguing. After years of writing in C++ and C# it was hard for you to remember Ruby’s case syntax because it can do so much more than switch statements in those languages.

So you wrote these notes to yourself as you discovered its capabilities. Except you’re not that much like me so you didn’t. But I did. I hope you find them useful.

switch/case syntaxes
(remember: Ruby uses "case" and "when"
where others use "switch" and "case"):

# Basically if/elsif/else (notice there's nothing
# after the word "case"):
[variable = ] case
when bool_condition
  statements
when bool_condition
  statements
else # the else clause is optional
  statements
end
# If you assigned 'variable =' before the case,
# the variable now has the value of the
# last-executed statement--or nil if there was
# no match.  variable=if/elsif/else does this too.

# It's common for the "else" to be a 1-line
# statement even when the cases are multi-line:
[variable = ] case
when bool_condition
  statements
when bool_condition
  statements
else statement
end

# Case on an expression:
[variable = ] case expression
when nil
  statements execute if the expr was nil
when Type1 [ , Type2 ] # e.g. Symbol, String
  statements execute if the expr
  resulted in Type1 or Type2 etc.
when value1 [ , value2 ]
  statements execute if the expr
  equals value1 or value2 etc.
when /regexp1/ [ , /regexp2/ ]
  statements execute if the expr
  matches regexp1 or regexp 2 etc.
when min1..max1 [ , min2..max2 ]
  statements execute if the expr is in the range
  from min1 to max1 or min2 to max2 etc.
  (use 3 dots min...max to go up to max-1)
else
  statements
end

# When using case on an expression you can mix &
# match different types of expressions. E.g.,
[variable =] case expression
when nil, /regexp/, Type
  statements execute when the expression
  is nil or matches the regexp or results in Type
when min..max, /regexp2/
  statements execute when the expression is
  in the range from min to max or matches regexp2
end

# You can combine matches into an array and
# precede it with an asterisk. This is useful when
# the matches are defined at runtime, not when
# writing the code. The array can contain a
# combination of match expressions
# (strings, nil, regexp, ranges, etc.)
[variable =] case expression
when *array_1
  statements execute when the expression matches one
  of the elements of array_1
when *array_2
  statements execute when the expression matches one
  of the elements of array_2
end

# Compact syntax with 'then':
[variable =] case expression
when something then statement
when something then statement
else statement
end

# Compact syntax with semicolons:
[variable =] case expression
when something; statement
when something; statement
else statement # no semicolon required
end

# Compact syntax with colons
# (no longer supported in Ruby 1.9)
[variable =] case expression
when something: statement
when something: statement
else statement # no colon required
end

# 1-line syntax:
[variable = ] case expr when {Type|value}
  statements
end

# Formatting: it's common to indent the "when"
# clauses and it's also common not to:
case
  when
  when
  else
end

case
when
when
else
end

I’m a software engineer living with my wife and three kids in the San Francisco Bay Area.

I plan to blog about stuff like software development, things that strike me as funny, etc. The technology that currently has my interest is Ruby on Rails and Flex but that may change over time. For most of my career in software I’ve been working on Windows, and I’m also a fan of most Microsoft developer technology (WPF, ASP.NET, and so on.)