Jake Scruggs

writes ruby/wears crazy shirts

Every time I create a new rails project I usually put off writing tasks to analyze the code's quality 'cause it takes time and time is, you know, finite. So I've decided to extract some code into a rails plugin which I call metric_fu.

It's a bunch of rake tasks that produce reports on code coverage (using Rcov), cyclomatic complexity (using Saikuro), flog scores (using Flog), and rails stats (using 'rake stats'). It knows if it's being run inside a CruiseControl.rb build and puts the output in the Custom Build Artifacts folder so when you view a build you see this:

Cruise control build page

The coverage report is your standard rcov report:
Rcov output

Flog output is thrown into an html file:
Flog output

At the end metric_fu calculates the average flog score per method:
Flog average score
You might want to check out my previous posts on what to do with a Flog report: The Method Hit List and When You Should Ignore Metrics

Saikuro's output is the same as always:
Cyclomatic complexity output
(I changed the warning and error levels for this pic -- more on how I did that later)

And 'rake stats' is always useful:
Stats output

So how do you get all these reports?
1. install Flog
sudo gem install flog

2. install rcov
sudo gem install rcov

3. install metric_fu
ruby script/plugin install \
http://metric-fu.rubyforge.org/svn/tags/REL_0_5_1/metric_fu/
(in the base of your rails app)

4. rake metrics:all

Which should work fine if you have standard Rails testing and you like my defaults. But what if you use a combination of RSpec and stock Rails testing? Then you can insert this into your Rakefile:


namespace :metrics do
TEST_PATHS_FOR_RCOV = ['spec/**/*_spec.rb', 'test/**/*_test.rb']
end
The namespace isn't strictly necessary, but I like it for intentional purposes. Multiple paths are useful if, like on my last project, you need to be specific about which tests to run as some tests go after external services (and the people who manage them get cranky if you hammer 'em a lot).

If you also want Rcov to sort by lines of code (loc) and have more aggressive cyclomatic complexity settings then do this:

namespace :metrics do
TEST_PATHS_FOR_RCOV = ['spec/**/*_spec.rb', 'test/**/*_test.rb']
RCOV_OPTIONS = { "--sort" => "loc" }
SAIKURO_OPTIONS = { "--warn_cyclo" => "3", "--error_cyclo" => "4" }
end

That's it -- hope you find it useful, lemme know if you find a bug, and check out the project home page at:
http://metric-fu.rubyforge.org

Oh, and thanks to all my co-workers who helped write the original code, in its various forms, that became this plugin.

Update 9/22/2008 - metric_fu is now a gem, on GitHub, and is useful for any Ruby Project. Check the home page for current information.

13 comments:

Chad Pytel said...

Looks Great. Is there a way to do this so it can just be installed on the CI machine, instead of having to be installed right in the rails app?

Jake Scruggs said...

I don't know how people are going to use this plugin. The potential user may not use cc.rb and just want to occasionally generate some reports. metric_fu puts the reports in a 'metrics' folder when not run inside a cc.rb build (it looks for the presence of ENV['CC_BUILD_ARTIFACTS'] to determine if it's being run by cc.rb).

Ben Burkert said...

wow, I could have really used this 6 months ago :)

I attempted a something similar, it was a mix between metric_fu and autotest. Called it autometric: http://benburkert.com/2007/11/9/introducing-autometric

It was a hack, and the performance hit on my system wasn't worth the realtime-iness of the metrics. If metric_fu is better performance wise, it might be worth taking another go at it.

rahim said...

Hi - thanks for this, looks really useful.

Like Chad, I'm interested to know whether this can be set up on the CI machine without polluting the rails apps themselves?

Sean said...

Same here. I'd like to avoid having to install gems on the dev machines if they're not needed. Maybe you and this guy can work together:

CC.rb Plugins

Jake Scruggs said...

It's a plugin without an init.rb so it really doesn't 'pollute' the application. Essentially it's a few rake tasks and Saikuro bundled up in a convenient package. If, however, you can't stand having it in your app, you could check out http://kwala.rubyforge.org

Sean said...

What's the best way to get CC.rb to run 2 rake tasks in succession? The default 'rake' makes sure that the migrations are run (somewhere in that hierarchy), but metrics:all does not. Whenever we post a new migration (we're early in development), the build fails every test because the DB is out of sync.

Jake Scruggs said...

Good point, Sean -- I didn't think about that. You'll have to create a new rake task and have it do the migration first like so:
task :my_metrics => ["db:migrate", "metrics:all"]

Then you can just call 'my_metrics' in the cruise control config file.

Sean said...

Simple enough. Thanks!

carpeliam said...

Hey Jake, this looks great. Unfortunately I'm on a Windows machine, so your calls to "sh '...' |ok, response| ..." are dying. Is there a Windows (or even better, OS-agnostic) equivalent, or perhaps an sh gem of sorts for Windows that you know of?

JohnnyBusca said...

I added a report for reek. Anybody interested?
http://silkandspinach.net/2008/09/23/reek-a-code-smells-detector-for-ruby/

Should I add it in github?

Jake Scruggs said...

I'm interested. By all means fork metric_fu on github, add it in, and send me a pull request. Thanks taking the time to write reek - I'll have to check it out.

Vishnu S Iyengar said...

hi jake, have you tried using it with jruby so far? It doesn't seem to work out of the box here, so we're digging into it. But we were wondering if you had tried it.