Jake Scruggs

writes ruby/wears crazy shirts

What's new:

  • Merged in Grant McInnes' work on creating yaml output for all metrics to aid harvesting by other tools
  • Supporting Flog 2.1.0
  • Supporting Reek 1.0.0
  • Removed dependency on Rails Env for 3.months.ago (for churn report), now using chronic gem ("3 months ago").
  • Almost all code is out of Rakefiles now and so is more easily testable
  • Metrics inherit from a refactored Generator now. New metrics generators just have to implement "emit", "analyze", "to_h" and inherit from Generator. They also must have a template. See the flay generator and template for a simple implementation.
  • You now define the metrics you wish to run in the configuration and then run "metrics:all". No other metrics task is exposed by default.

Full documentation, as always, is at: http://metric-fu.rubyforge.org/

Let the Google group (http://groups.google.com/group/metric_fu) know about any bugs you find.

Apr 29, 2009

Women and Code

So I've been kind of heads down trying to get metric_fu ready for Rails Conf, so I had only vaguely heard about the recent controversy over Matt Aimonetti's recent presentation "CouchDB: Perform like a pr0n star." When someone did mention it to me I thought of David Heinemeier Hansson's famous "F--- You" slide or Zed Shaw's rants or any given bit of profanity/edginess at Rails conferences and shrugged it off as a bit of a bad idea.

Earlier today we were having our weekly team meeting when the topic of the controversy came up and my friend Leah was quite upset by it. And I realized that she has to deal with a lot of crap everyday being the only female coder in our office, at user groups, in conferences. We've had experiences where clients treated her in ways they would never have treated a man. And that sucks.

I've read Matt's post about the experience and I believe he didn't mean any harm, but I don't think you can show pictures of mostly naked ladies in a presentation when the female developer community is such a visibly discriminated against minority.

I really should have done this a long time ago, but you know how it goes.

Anyway, if you're using metric_fu and have a question or want to discuss it's direction/development you can now here:

http://groups.google.com/group/metric_fu

There's some big changes on the horizon for metric_fu -- If you want to get a preview, head over to github and take a look at the to_yaml branch of metric_fu. I've been pulling in Grant McInnes changes to be able to serialize the output of all the metrics. This means that there will be a yaml file created after each run which can be used by other projects. A number of people have asked for this feature so they can do analysis of their metric data.

However, this will probably break some uses of MetricFu as it is an almost complete re-write. What's cool about this branch (besides the output) is that almost nothing is in rakefiles anymore and so is more easily testable. If you use metric_fu and have an opinion on this, or anything else, join the group and make yourself heard.

I spent the weekend at Citcon (Continuous Integration and Testing Conference) in Minneapolis. Citcon (which is pronounced Kitcon -- because it sounds cooler) is an open spaces conference which means that there are no scheduled talks. The people who show up propose ideas and then they vote on what they want to see. It sounds like it would never work but it actually works very well. For instance, Toby Tripp proposed a session on testing smells and it got enough votes to get a session but when I showed up there were only 6 people. Which kinda bummed me out until we started talking, then I realized that we had exactly the right 6 people and another 20, 50, or 100 half interested dudes typing away on their computers really wouldn't have added to the talk. As it was the talk was intimate and to the point.

We came up with the following smells (a smell is a sign that something might be wrong -- Fowler used this metaphor to great effect in his seminal book "Refactoring"):

Testing Smells

Requires Supervision:
A test is not automatic -- it needs user input to function. Someone has to watch the test and enter data periodically.
Solution: Don't do that.

Refused Bequest:
A mega setup that some tests ignore, some use, and most rely on but only for a few lines. When any individual test fails it's hard to parse the setup to see what's going on.
Solution: In RSpec you can use nested describes/contexts to have mini setups for each situation. Or you can write private method setups that are more focused. Or you can create multiple test files for different contexts.

Many Assertions:
When a test tests way too much its failure is often hard to track down.
Solution: break the test up into smaller atomic chunks (test cases) that zero in on one or two things a method should do.

Large Test File:
Things can be hard to find in a large test file and its documentation value is lost.
Solutions: It could be the class under test is too big and needs to be broken up into smaller discrete bits. Or you may have redundant tests that should be removed. Maybe you need a lot of setup and could move to a data or factory driven test. Or you could extract testcases into separate files (or describe blocks) each with a specific purpose to organize things.

External Data:
A test that needs external data (fixtures, files, helper classes, etc.) in order to run can be hard to understand as the reader must file hop to get a clear picture.
Solution: Avoid this, if possible, by using standardized Object Mothers, mocks/stubs, and removing redundant tests.

Excess inline setup:
Too much setup in every test means the reader has to skip most of the test to get to the meat.
Solution: Extract to setup methods or different test files. Focus and organize your tests.

No Assertions:
A test that has no assertions only exists to lie about code coverage.
Solution: Don't do that. Delete the test or add an assertion (mock expectations count as assertions).

Testing the Framework:
If you're using a framework you pretty much have to assume it works. Testing it along with your other tests only clutters things up.
Solution: Delete the framework tests.

Test Envy:
A test that is really testing some other class than the one it is supposed to. A common example of this is testing model validations in the controller when they should be tested in the model test.
Solution: Move the test or delete it if it is redundant.

Brittle Tests:
Tests the break all the time when small things change undermine the importance of test breakage. Examples are testing that a greeting is inside an "h1" in a view or the exact wording of text on the screen.
Solution: Perhaps you don't really need to assert that. Or it might be a sign of a deeper problem with your code when it's tests are fragile.

Order Dependent Tests:
Tests that need other tests to run before them really screw things up when things get moved around.
Solution: Make sure your tests clean up after themselves or are reasonably isolated from the outside world.

Long Running Tests:
When the feedback loop of code, test, code gets too long it can encourage bad behavior.
Solution: Spend time reducing the time of your tests. Buy faster machines. Mock out expensive operations (such as database calls).

Mock Happy:
A test that needs 17 mocks to run can be quite fragile, resistant to change, and hard to understand.
Solution: This could be a sign that the code you are testing has too many dependencies. Or you could introduce concrete collaborators. Mock/Stub out that which you don't have access to or is expensive, but the rest can be real models. Yes this makes your tests less unit-y, but it will make them more robust and easier to understand

Not Idempotent:
Fancy words meaning your test changes the state of something permanently and thereby introduces the possibility of order dependent tests.
Solution: Clean up after yourself.

Overly Dry Tests:
DRY (Don't Repeat Yourself) is a good idea, but pulling out all repetition can lead to some very hard to understand tests.
Solution: Some moistness is alright if it helps the reader understand the intent of the test.

Code Run Only by Tests:
Dead code (never touched in production) is very confusing to the developer trying to understand the system. Tested dead code doubly so.
Solution: Delete the test and the code. You'll still have it in source control if you ever need it in the future.

Commented Code in the Test:
Commented out assertions or setup may mean someone took a quick path to making the test pass.
Solution: Investigate if the code should be there. If there's no reason for it, delete it.

My new talk "What's the Right Level of Testing?" got accepted for the Agile 2009 conference happening late August in Chicago. I'm pretty excited to be going -- the last time I was there it was called XPAU (eXtreme Programming Agile Universe).

So yesterday Leah and I were working on SudokuScanner -- a cool little site that will, when it's released, allow you to upload images of Sudoku boards and get a solution. All of a sudden, our mocks and stubs went totally batshit. One minute she and I were happily writing some tests, making them pass, refactoring, checking in, etc... and then, we got a bunch of failures. OK, no problem -- we must have busted something with our latest change. The failure was a mock expectations error: UserMailer expected to receive a call to deliver_activation but did not receive it. So we went over to the failing spec and... There was no expectation on UserMailer anywhere in the spec.

That's bad.

Ran the spec by itself and it passes.

Yep - not so good.

Reverse the order that the specs run in (spec --reverse) and a different number of specs fail, but still with the same expectation failure. And of course these new specs all pass on their own. And have no reference to UserMailer anywhere in them. I searched the project and only one spec has one expectation on UserMailer.deliver_activation, but now I have 10 failing specs that are all failing because of that expectation.

So it appears the mocking framework isn't cleaning up after itself -- Yipes!

After a bunch of exploring/messing about I found myself in the spec_helper.rb file and I noticed something weird: The line 'config.mock_with :mocha' that should not be commented out is commented out. Which is weird because we're using Mocha as our mocking framework. So I uncomment it and run all the specs through rake. A billion errors all saying some variation of "method stub! does not exist" or "method should_receive does not exist."

Got it.

So it seems we are using RSpec's built in mocking framework on this project, but somebody started using Mocha too. Full disclosure: That somebody is probably me. I've been bouncing between a bunch of projects lately and some of them use Mocha and some use RSpec's mocking framework. I probably task switched over and started using Mocha without thinking about it and it just worked. I have Mocha installed as a gem on my machine and maybe that's enough, or maybe one of our plugins or gems requires it, but it's there apparently in the load path so everything was fine for awhile. Of course mocking frameworks do some pretty intense meta-craziness under the covers so it was bound to happen that using both would lead to one screwing up the other's functionality like, say, breaking Mocha's ability to clean up after itself. Which lead to our pile of nutty failures.

Luckily changing over to from Mocha to RSpec's mocking framework is not hard so we were able to purge the mocha syntax from the project and all was made well.

I could write a test that looks at all of my test files for Mocha syntax... Or is there a way to un-require Mocha explicitly so this doesn't happen again?

Day Five of the Craftsmanship Swap:

Last day!

Today started off with some more work on a soon to be released 8th Light project that will hopefully make them some money. Colin and I had a good time blowing up the app and making sure a notification email gets sent out using Exception Notifier. Later on, at a retrospective, Paul asked me if I thought it would make any money and I said that I had no idea but it was worth it even if all they got out of it was some greenfield experience for Eric and Colin. Oftentimes when you're a new developer you get told what to do. But this project, although managed by Micah, has mostly been programmed by Eric and Colin. Which is awesome. It's a real app that will go into production and charge credit cards, but it's small and simple enough that they could write every line. There's a lot of 10 year veterans of the industry who've never had that sort of start to finish experience and I think it's cool they're getting it.

In the late morning/afternoon I worked on Fresnel -- a Ruby/Limelight desktop client for Lighthouse. The idea is to fix some of the ui issues but also to enable an offline mode which would sync up to the server when connected to the internet (this feature has not yet been implemented).

And now some photos of 8th Light:

The exterior.
8th Light Exterior

The lake right next door.
8th Light Outside 5

From left: Paul, Eric, and Colin
8th Light War Room 1

Lonely Colin early in the morning in the war room.
8th Light War Room 3

Long shot over the stairs into the meeting area (where they host user's groups).
8th Light Interior 2

Where people sit in the meeting area mentioned above.
8th Light Interior 1

Thanks again to all involved at 8th Light and Obtiva for letting me have this wonderful opportunity to see how another small consultancy works toward software craftsmanship.

Day Four of the Craftsmanship Swap:

Today started with Eric Smith and I working an iPhone version of Conway’s Game of Life trying to be as test first as possible. It was... Painful. I've been spoiled by my time with Ruby into expecting first class testing/mocking/stubbing. Even working with a guy who's given talks on how to test drive iPhone apps there was still a lot of wasted time scratching our heads, hand writing mocks/stubs, and f-ing around with clicking and dragging lines from one box to another.

We got interrupted mid-morning by some production issues in a large Rails application 8th Light maintains/ develops. We had to write a script to find some bad data and fix it so I got to use one of my favorite tricks to mark the progress of a long running process:

10.times do
sleep 1
print "."
STDOUT.flush
end

If you don't put in the "STDOUT.flush" all the dots come out altogether at the end, but with the flush you get a nice dot to tell you every time you've completed one loop. Copy and paste the dots into TextMate and check the column number to know how many you've done for extra long processes.

After we straightened out the productions problems we had to go back to iPhone development. Sigh. We spent about 3-4 hours working on this application and we accomplished just about nothing. We were trying to get a button to repeatedly step forward the game using an NSTimer, but by the end of the day we were still having no luck. We did get the button to stop crashing the app altogether, so I guess that's something.

Btw, Eric's personal iPhone has a face plate that is 40% shattered.

Day Three of the Craftsmanship Swap:

Micah and Paul had to head out of town for an Iteration Planning Meeting/Demo so it was just Colin Jones, Eric Meyer, and I in the office. We decided to do some tri-ping pong development on an internal product 8th Light is planning to release soon. So Eric would write a test/spec, I'd make it pass and write another spec, then Colin would make my spec pass and write one for Eric and the whole process would repeat. An enjoyable way to work made easier by their 24inch Cinema Displays and wireless keyboards.

I've noticed that everybody shakes hands at 8th Light. A lot. When someone arrives in the morning the new person tends to shake the hands of all who are already there. The same thing happens when someone leaves for the day. I asked Colin and Eric about the origins of this but they had no answers for me. It seems kind of weird, but it's also sort of a cool affirmation of camaraderie. It may just be better than the grunt I give people when they leave for the day.

At some point I found myself arguing for these lines of Ruby today:

return false unless coupon = Coupon.find_by_token(session[:coupon_token])
return coupon.active?

Instead of a 5-6 lines of if/else code. Have I become the very person I despised when I got into Ruby? That guy who can't rest until lines of code have been reduced to unreadable terseness? Even inject doesn't look so bad to me anymore...

Of course, if we were using a current version of Rails we could do this:

!!Coupon.find_by_token(session[:coupon_token]).try(:active?)

Now that's teh hotness.

Day Two of the Craftsmanship Swap:

The day started off with a bit of work on slim for C with Doug Bradbury. Slim is a replacement for Fit which underlies Fitnesse and you need to write one for every language you plan to use FitNesse on. I gave it my best shot , but then Uncle Bob (Bob Martin - father of 8th Light founder Micah Martin) and James Grenning showed up. Great to see those guys -- I don't think I've seen James since my days of apprenticeship at Object Mentor. They are crazy better at C than I am so they worked with Doug while I paired with Micah on drving that ui through Fitnesse (which I talked about yesterday). We made some pretty good progress but he had some meetings to attend so I got to work with 8th Light's newest apprentice: Colin Jones. Guess what he was working on?

Tic Tac Toe.

That takes me back. 5 years ago almost to the month, Micah had me write a Tic Tac Toe program to see if I had the skills to become an Object Mentor apprentice. Well Colin cleared the same bar and now he's moved on to 3 dimensional Tic Tac Toe. Think of a Rubick's cube with X's and O's inside the cubes. Three in a row still wins, but in any direction. Turns out there's about a trillion different outcomes to a 3D cube so brute force solving is not going to work unless you have some time. Looking 4 moves ahead took about a third of a second, and 5 moves took 6 seconds. Looking at all possible moves and then choosing ones that lead to victory would take insane amounts of time. So we brainstormed some alternatives. You could let the computer chug away and build a massive database of all possible solutions -- not super practical. Or you could approach it like chess: Have some pre-defined moves for the early game and a way of evaluating positions for the mid to late game. That way you can limit the depth of your search and still make ok decisions. We talked for awhile about what positions are valuable, but along the way we realized that 3D Tic Tac Toe is a terrible game. If you move first and take the center of the cube you're going to win in about 5 moves. If you don't take the center of the cube your opponent will and they'll take slightly longer to win -- but win they will. Draw it out for yourself and see. When Micah got back from his meeting we chatted about this and agreed that although there's merit in learning how to tell a machine to play 3D Tic Tac Toe, some of the fun of doing it is spoiled by the game itself. So Colin's switched over to 4x4 Tic Tac Toe (or is it Tic Tac Toe Tim?) for now but may return to the third dimension sometime in the future.

Btw, Micah's cell phone ring tone is "Damn it feels good to be a gangster." It does indeed.

Day One of the Craftsmanship Swap:

Today Micah and I paired on test driving a user interface through FitNesse. The UI in question is a JRuby rich client application that had reached the point where they wanted to have some acceptance tests that fire it up, click around, and assert that things are where they should be. We used DRb to stow away a reference to the application when it started up and then we could make the application dance. If you haven't used DRb it's yet another deceptively simple and powerful Ruby tool. Check this out:

~ $ irb
>> require 'drb'
=> true
>> object = "hello"
=> "hello"
>> DRb.start_service('druby://localhost:9001',object)
=> #
>> ref = DRbObject.new(nil, 'druby://localhost:9001')
=> #
>> ref.size
=> 5
>> ref.to_s
=> "hello"

Stick any object in and then another Ruby process can pick it up and do stuff with it. Neat.


I noticed 8th Light has the same desk space problem as Obtiva in that we have an open workspace, but people tend to settle into one space. And then they let all their personal junk pile up. Where do you keep all the personal products (toothpaste, dumbbells), mementos (awards, action figures, ninja swords (interesting side note: Every 8th Lighter got an engraved ninja sword for Christmas)), and other detritus (old specifications, note cards) when people don't really have a desk of their own? When I went up to Atomic Object a few weeks ago (I gave a talk at their XP West user's group) they had a bunch of cubby holes, like you might see in a school, big enough to toss a backpack into. At least it got the backpacks off the floor. Personal space is hard when you're committed to pairing. I've seen situations where developers had their own desk in addition to working in a 'war room,' but that can be tricky as when you're not used to the war room it can be tempting to sneak off to your desk so you can 'get some work done.' Phone calls can also be a problem. I love the pairing and the war room environment and there's no way I would have learned as much as I have without them, but they do present some unique problems.

I finished up the day having a good time ping-pong pairing with Eric Smith on a large rails app. It's funny how quickly you can pick up the spirit of an application and contribute when conventions are followed and there's an expert engaging you at every turn. Sure, the first 30-45 minutes I was asking a lot of questions but after that I was adding something to the process -- it really does surprise me every time.

This morning instead of taking a train to downtown Chicago I'll be driving to the northern suburb of Libertyville and working at 8th Light for a week. Obtiva (my company) is participating in a craftsman swap so 8th Light will be sending someone down to Obtiva whilst I work with 8th Light. This should be pretty interesting as the last time I worked with some of the 8th Lighters I was an apprentice at Object Mentor.