Jake Scruggs

writes ruby/wears crazy shirts

One of the awesome things about the business we're in is that you can ask people to actually do the thing you're hiring them to do IN THE INTERVIEW! You can't really ask a banker to do some banking during an interview, but it's relatively easy to set up a computer with a coding problem and ask a potential hire to work through it. Having recently gone through a job search, I though I'd discuss my thoughts on coding problems as part of the interview process.

The first time I ever wrote code for an interview was at ThoughtWorks. It was a 3 day at home coding problem. There were always 3 different questions and each could be knocked out in a few hours. The candidate could choose which one they wanted to solve and upon submission, the solution was looked at by at least 2 different developers. The great part about this is that we could filter out a lot of candidates before bringing them in to the intensive day long interview. The downside of a take home test is that you never really know if they got some help. Which is why in this type of interview it's important to go over their solution in person.

One coding interview I did was 2 developers and I working on a Rails plugin. It seemed like a good idea, but I don't create Rails sites from scratch every day so there was a fair bit of fumbling around hooking up the database and trying to remember what generators to use and so on. When we finally got to the writing a plugin part I was a bit flustered and short on time. We made some progress but I don't think either of us were satisfied with the outcome. This brings me to an important point: You will accomplish a lot less in an hour than you think you might. This is because the interviewee is using an unfamiliar machine, text editor, doesn't have his or her custom bash aliases available, and is being watched. And questioned. All this will slow anyone down by 50%. So if it would take you 30 minutes to solve a given problem from scratch then it's about right for an hour long pairing interview.

At Obtiva, Colin and I paired on some truly horrible code that he had inherited. Oh the stories we could tell you about that code base... But I'm not sure how I feel about these "work with me" interviews if they are going to last less than 3-4 hours. Actually, a day would be a better time allotment for this type of interview. The reason is that solving a problem in an existing code base requires a lot of backstory. If you only have an hour or two it's hard to grok the domain and you may just be testing the candidate for familiarity with the subject matter instead of pure programming talent. It would be easy to miss a good candidate or hire someone who just happens to know the domain.

Speaking of domain, you should try as hard as possibly to eliminate it from your coding problem. I once did an interview with a firm where the challenge was to build a highly simplified version of something from their domain. The problem was that it really wasn't simplified enough. After reading a page and a half describing the system my head was spinning with new terms and concepts (I wasn't familiar at all with their field). They had helpfully started me out with some stub classes and some failing cucumber specs but when piled onto a shaky foundational understanding it actually hurt more than it helped. I had to figure out the domain, understand the class stubs (and any hints they may have been providing), describe what I was doing to the interviewer, understand what the cucumber tests were asking for, and navigate an unfamiliar workstation. It turned into a bit of a train wreck. Basically I did what I do when I don't understand something: I went slow and tested everything. But I ran out of time before I could even get one of the cucumber scenarios passing.

You should try to structure the problem so that it has an easily achievable first step. This enables the interviewee to get his or her confidence up to try the trickier parts and also provides an easy hint you can give them if the are struggling. You might say: "Hey, why don't you just try and get X working." X being the easy part. You'll get better results when people are relaxed.

At Backstop we have a couple of coding questions, but the central theme is that the domain is either something everyone is totally familiar with or we can teach them in a few minutes before the coding starts. We also make sure that the candidate pairs with different programmers to see how they interact with different people. Now pairing while interviewing is not like regular pairing, of course. When I run the interview, I approach pairing with an interviewee like I'm a novice developer who is a domain expert. I'll answer any questions they have about the domain but I don't help much with the coding part. I always allow access to any documentation they want to look up and of course I have irb open and ready for experimentation.

For me the pairing interview is just a way to have an engaging conversation about code. If someone doesn't solve the problem but does so in an interesting and thoughtful way then I consider it a success. This also means that an unreadable working solution is considered a failure.

Having actual programing be a part of your interview process will take a some thought, but all that time is well worth the effort. I can't tell you how many times someone has sailed through the talking parts of the process only to crash and burn in the coding section. Hiring someone who is great at talking about code but terrible at actually coding is one of the worst things you can do.

I'll be giving an all new talk entitled "The Necessity and Implementation of Speedy Tests" at Ruby Kaigi in Japan! Ruby Kaigi is being held August 27-29th, 2010 in Tsukuba, Ibaraki, Japan. Very exciting!

Want to see the above talk but can't make it to Japan? I'll be giving the same (ish) talk at Ruby Midwest in Kansas City, MO. Ruby Midwest will be held July 16-17, 2010.

Stop on by and say hello -- I'll be the guy in the crazy shirt.

May 16, 2010

Why I Left Obtiva

Recently I ended my nearly 2 year relationship with Obtiva and why I left deserves a few words.

The issues I had with Obtiva were mostly just issues I have with consulting:

  • You only work at places messed up enough that they need to hire consultants
  • You leave before you get to spend time with code you wrote so it's hard to learn long term lessons
  • The most stable gigs that pay the bills for a consultancy tend to be the worst programming experiences (Large corporations with way more money than sense)
  • You have to track every hour of your time
  • Time off has to be negotiated with two companies instead of one
  • Learning a whole new bunch of names. personalities, and organizational structures every 6 months
Now don't get me wrong, consulting with ThoughtWorks and then Obtiva made me a much better programmer and was an amazing learning experience but I felt that, in order to grow as a programmer, I needed to work for a product company and learn the lessons that only living with a code base long term can teach you. Consulting is a great drinking-from-a-fire-hose experience that every developer should consider, but it has its limitations. So I joined Backstop Solutions and I write Ruby on Rails sites for hedge funds and the people who use them (note: we sell products to hedge funds, I don't actually work for a hedge fund).

Why Backstop? Well, it's big enough (60ish people) that they've worked out most of the kinks of being a business (direct deposit, 401k, health insurance, etc.) but small enough that you can get to know everyone and (hopefully) help steer the company. Their product is strong and growing, but there's some new products in the works too. They do Agile but would like to do it better (I've been brought on to help with that in addition to my coding responsibilities). But the best part was that while everyone is smart and nice, it's oddly unknown in the Ruby community -- and I can help with that.

Obtiva was a great experience. During my time there I moved from being just a coder to being more of a leader, started speaking at conferences, got to participate in many wonderful geekfests, helped mentor software apprentices (which I once was), and learned a ton from all the programmers I interacted with.

To sum up I'll quote from my company-wide resignation email: "Mommy and daddy still love each other. Except we're going to see other people. And not live together."

So recently at work I was asked help to make the company more "Agile." Well I'm a developer first and process wonk second so I responded with my usual "How many of the 12 practices are you really following?" Which was met with a lot blank stares. Turns out the classic XP practices are not so easy to find on the internet anymore. It also turns out that the word "Agile" has been so successful that lots of people don't know that XP stands for Extreme Programming. Now I'll be the first to admit that "Extreme Programming" is a colossally stupid name, but what I like about XP and the original 12 practices is that they were controversial and easy to evaluate: Either you were doing them or you weren't. What I don't like about "Agile" is that it's so broad and defined in such a "hand wavy" way that pretty much everyone can fool themselves into thinking that they already are fully Agile.

So to fix this and help my company start measuring their "Agility" better, I've decided to list out classic (They've changed over time -- but not for me. And get off my lawn!) 12 XP processes. And my highly opinionated view of each of them. After that I'll tell you how to score your "Agility."

* Planning Game
Get requirements from customer, form them into (short) stories, estimate the time it will take, do them, measure velocity, examine failed estimates for clues as to how to estimate better, and repeat. A story should be a placeholder for a conversation -- This only works if the customer (or surrogate) is highly available to the developers and the developers take advantage of that availability by checking in frequently during the development process. Note: Stories that take longer than a day significantly undermine many of the other parts of XP.

* Small Releases
Iterations should be 1-2 weeks (depending on how painful/expensive it is to organize an iteration). Releases should happen after every iteration if the application is easy to deploy (web apps).

* Metaphor
Nobody does this. Extra points if you do.

* Simple Design
This means different things at different levels. When kicking off a green field project a week of requirements gathering and design is good. For an individual story, a few minutes to an hour is good. Then you start writing code. Important: Design as you go. Stopping to spend a few hours with a whiteboard is allowed and encouraged. Do the simplest thing that could possible work until it becomes apparent that it will not. Then redesign. The idea is that at the beginning of the project you know the least about it -- spread the design across the life of the project so you can make design decisions with more knowledge.

* Testing
Test first. Coverage above 80% for Java, 90% for Ruby (less exception checking in Ruby means easier to cover). Non-brittle, atomic, fast tests (unit tests should run under 5 min. Actually under 1 minute is best, but most think that's impossible (spoiler: Its possible)).

* Refactoring
Short: Red, Green, Refactor! Long: Write a failing test, make it pass, refactor the code. This also means when you wander into a messy place you leave it a little better than you found it.

* Pair Programming
Above 80% of the time. Seriously. Tasks that seem like you can't pair on them often mean that you aren't using the right pairing techniques. Pairing is a skill: Get good at it.

* Collective Code Ownership
Anyone can change any code. Yes, they should consult whoever is knowledgeable about said code, but there are no parts of the code that only developer X is allowed to touch.

* Continuous Integration
Every check-in to the remote repository triggers a comprehensive suite of unit tests. The developers have to care if the build fails. For example: When the build fails, no one checks in until it is fixed. Also there should be some social ridicule for the build breaker (have to wear a stupid hat for the rest of the day, an obnoxious build breaker trophy placed on their desk, they have to buy donuts for the team, etc.). This all assumes that the build is reliable and that a failure means there is a real problem.

* 40-hour Week
Now called "Sustainable Pace" because "40-hour Week" tends to scare managers and end of project pushes are sometimes necessary even in XP. This should be no more than 2 weeks, have a clearly defined point at which the long hours will stop, and happen no more than twice a year. Actually, if it happens twice a year it means your estimation skills need to get better.

* On-site Customer
Doesn't happen often for non-internal projects. Usually customer surrogates are used. These surrogates need to take pains to talk to real users often (not just the managers of the users) and be highly available to developers to answer questions.

* Coding Standards
Doesn't matter what they are but the developers need to come to a consensus and stick to it. Consensus does not mean everyone nods yes in the meeting but ignores the standards when they feel like it.

So how do I score my Agility? Take the number of practices that you actually do, divide by 12, and then multiple by 100 -- that's your percent Agile. Do not give yourself a point if you don't fully implement a practice. Have Continuous Integration but some of the tests fail randomly? No point. You pair "when necessary" which ends up being 40% of the time. No point! Run iterations but don't estimate your stories? No Point!!

This is take a hard look at yourself in the mirror time, not some hippie love fest.