Metrics for Rails

Everyone thinks they write good code -- it's just part of human nature. You can't do something every day and not secretly suspect that you're good at it. Self-delusion is a powerful thing so you need to use metrics to take a hard look at your code.

On my current project, we've just added a daily metrics build (run every day at midnight by CruiseControl.rb) that takes a look at our code in three ways:
  • Code coverage with Rcov
  • Cyclomatic complexity with Saikuro
  • And um..., Flogging with Flog
Rcov is a code coverage tool that can be used with Rails Rcov to add a bunch of rake tasks to your build so you can figure out which lines of code are run by your tests... and which are not.

Saikuro computes cyclomatic complexity which "measures the number of linearly independent paths through a program's source code." Methods with more paths are harder to understand/debug/modify.

And Flog is cyclomatic complexity with an attitude. It scores ruby methods with an "ABC metric: Assignments, Branches, Calls, with particular attention placed on calls."

Why do we use both Saikuro and Flog? Well Flog catches Ruby specific complexities that cyclomatic complexity doesn't (for instance, calls to eval are given particular weight) and it picks up methods that Saikuro misses (we use metaprogramming to define a fair amount of methods and Saikuro seems to miss anything not defined with a 'def'). But Flog outputs a flog score which isn't very familiar to most developers, while cyclomatic complexity is a relatively well understood computer science term.

Also flog has the scariest looking software website ever. I can't believe those guys have the chutzpa to put such pictures on a site that advertises them as consultants for hire.

So how did our code look? Pretty good -- 96% code coverage overall but some methods need testing love. And some of our code had pretty high cyclomatic complexity or Flog numbers so we'll need to write some developer tasks to fix the problems uncovered. (Every iteration we work on a few developer tasks in addition to the development, bug, and production tasks.)

But the metrics build isn't done, in the future I'd like to:
  • Find duplicate code with CPD.
  • Use Heckle to see if our tests are any good. (Heckle changes your code (not permanently) and then re-runs your tests -- if they don't fail then you've got bad tests)
  • Fail the build on bad metrics numbers.
  • Figure out how to integrate the results Panopticode-style with all the cool visualizations it offers.

Comments

Anonymous said…
Thanks Jake. Nice post.
September 19, 2007 at 9:39 AM
Ed said…
Great article and points, and you're right, the website is downright creepy. Here's a new website test: "Offspring test: if you wouldn't show your website to your (or your neighbors') children, it probably needs a redesign."
September 26, 2007 at 11:54 AM
Unknown said…
This comment has been removed by a blog administrator.
October 17, 2007 at 7:57 PM
John said…
would love to see an update to this post to know what you're using today
April 1, 2011 at 1:03 PM
Post a Comment

Popular posts from this blog

What's a Good Flog Score?

I've been using Flog to measure the complexity of my Ruby code and I was wondering how others interpret Flog numbers. I'll go first with my opinions: Score of Means 0-10 Awesome 11-20 Good enough 21-40 Might need refactoring 41-60 Possible to justify 61-100 Danger 100-200 Whoop, whoop, whoop 200 + Someone please think of the children (note: these are scores for an individual method) The 20 - 60 range is interesting to me. Sometimes the complexity of what you're doing will justify the score and other times the method is in desperate need of refactoring. The highest Flog score I've seen in the wild was in the 300's. What's you're personal high and what do you think of my rankings? In other news, I've redesigned the look of my blog so if you only see my post through a reader you might want to stop on by, take a look, and tell me what you think.
Read more

SICP Wasn’t Written for You

The number of software luminaries who sing the praises of “ Structure and Interpretation of Computer Programs ” (referred to as SICP) is such a long list that you might think only a crazy person would take issue with it. However, to ignore SICP’s problems and continue to blindly recommend it seems just as crazy. SICP was the textbook for MIT’s introductory programming class and was a bit of a departure from other into to computer science textbooks at the time.  Wikipedia sums it up nicely:  “Before SICP, the introductory courses were almost always filled with learning the details of some programming language, while SICP focuses on finding general patterns from specific problems and building software tools that embody each pattern.”  Which sounds awesome, but does essentially say that abstract principles will be introduced before the nuts and bolts of a language.  If you think about that for a minute, you may see where the problems will be. When I was training to be a teacher I took a b
Read more

Point Inside a Polygon in Ruby

Recently my team needed to find out if a point was inside a given polygon and, as usual, we found some code on the internet that did what we wanted. And, as usual, there wasn't much explanation as to what it was doing. Here's the code (after we converted it to Ruby): 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 def contains_point? (point) c = false i = -1 j = self .size - 1 while (i += 1 ) < self .size if (( self [i].y <= point.y && point.y < self [j].y) || ( self [j].y <= point.y && point.y < self [i].y)) if (point.x < ( self [j].x - self [i].x) * (point.y - self [i].y) / ( self [j].y - self [i].y) + self [i].x) c = !c end end j = i end return c end end So, you know, it works and I'm grateful to somebody for posting it but we had no idea how it works. So we wrapped 2 tests around it (one where the point is not in the polygon and another where it is
Read more