Jake Scruggs

writes ruby/wears crazy shirts

Jan 25, 2007

Testing vs. Speed

So I tend to be on the more extreme side of the testing debate. On my current project I helped institute extensive use of RSpec and Selenium Remote Control. RSpec allows you to test the models, view, and controllers in isolation from each other through the use of a built in mocking framework (to mock out ActiveRecord objects and Stub out call to finders) and some ideas borrowed from Zen Test. Selenium Remote Control allows you to drive a browser (your choice) from Ruby (or most other langs). Which is cool because we were going to have lots of Javascript in this project to control the Google Maps features. The RSpec tests went well: Easy to write once you got a handle on their syntax, and they run real damn fast they don’t hit the database for the view and controller tests (their opinion is that model tests should still hit the db b/c models are so tied to the db it would be silly to test them without it). For the SeleniumRC tests we used RSpec as the harness and created page objects to query the page. Here’s a typical example:



context "Having not logged in" do

specify "going to the create_event page redirects to the login page
and upon logging in redirects back to the create_event page" do
navigate_to.home_page
home_page = get_home_page
home_page.go_to_create_event_page
login_page = get_login_page
login_page. login "Username" “Test”
get_create_event_page
end

end

The context specifies a given condition (having not logged in) but the text is not executed. The specify is a “specification” or what most people think of as a test. We have a navigator object that knows how to get around the app, a page_mother object that can get page objects, and page objects that know how to do and get things on a given page. So, to take the spec line by line:


  navigate_to.home_page

Send the brower go to the right url.


  home_page = get_home_page


Get the home_page object and run some assertions to check to see if the browser is really on the home_page.


  home_page.go_to_create_event_page

The home_page object knows how to click a link that takes it to the create_event_page


  login_page = get_login_page

The get_login_page method gets the login_page object and makes sure we are indeed on the login_page (we’ve been re-directed there as only logged in users can create events).


  login_page.login "Username" “Test” 

The login_page knows how to log in with a given username and password.


  get_create_event_page

And finally the get_create_event_page method makes sure we are on the right page.

All tests are run in an instance_evel of a page_mother instance that has an instance of navigator returned by navigate_to.

Sound complex? Well it was. This was one of the most well tested apps I’ve ever worked on and, as it turned out, that was totally not what the client wanted.

Crap.

We spent a lot of time writing and running these tests because:

Google maps is hard to test – you have to write some fun javascript on your page to keep trac of any markers you place on the page because the GMap object won’t give them back to you once they’ve been set on it. And other stuff like that.

Google maps takes a bunch of time to render. We had to put our timeouts at 30 seconds and still tests failed b/c the map wasn’t back by the time we made an assertion. This, of course, caused the test to run way slow as lots of pages had GMaps.

The page_mother, navigator, and page object framework we invented was very elegant but pretty heavyweight. When you created a new page you had a lot of stuff to write just to test that page.

So our test suite, while impressive, ran slow and took a bunch of time to write. Which would have been fine if the client was interested in a bug-free site at all cost. Everybody on the ThoughtWorks team had just come from projects where we worked on apps that handled money and sensitive information. You really do not want to screw up that stuff. But his project was a little social network site where nobody would die if some data got lost. What we found out after 6 weeks is that the client just wanted something out there that looked good and in a hurry so they could see if anyone was interested in this site. Luckily when Obie and Zed joined the team they were able to see how much this reliability was slowing down the team and find out that the client didn’t much care about the value of rigorous testing. So we went in and ripped out all the SeleniumRC tests.

It was a sad day for me.


I was really proud of that test suite and it had taken a lot of hard work. Much more important was the fact that I had failed as a consultant. I had let an unspoken assumption slip through which stopped the team from delivering what the client wanted.

After we stopped writing and running Selenium tests (we still wrote unit tests with RSpec) we could go way faster and knock out a bunch of functionality in the last two weeks. Which is awesome: The site is looking good. Obie and Zed plan to go back and put in some Selenium tests once the site becomes more stable.


Now before I get beat-up for saying testing make you go slow, I should say this: I know that in the long term testing makes you faster. Speed you get now by not testing comes at the price of fragility and buggyness and trouble refactoring later. This is true. But for now the client is willing to accept a bit of ruff edges if it means speed – And we, as consultants, have to respect that.

Jan 23, 2007

Huda's Wedding

Getting married requires paper work



A lot of paperwork:



Huda, his wife Uzma, and Huda's parents


Andy's not so sure about the other Andy.


I kiss my wife


Their first dance as married people.


My wife and some drunk guy we met.


The ladies of the family


Take more photos!

So I work in Atlantic Beach, FL and I live in Chicago. So that means I spend a bunch of time on a plane every week. But this week I was best man in a wedding… in Seattle, WA. You know, the exact opposite corner of the lower 48 and not so easy to get to, as it turns out. I had cleverly arranged to connect through Chicago so I could meet my wife at O’Hare and then fly together to the land of coffee. But, of course, my plane was delayed. I got off the plane at O’Hare at 5:12 pm to make a connection at 5:15. Now I’m a large man these days but I did manage to run from the C concourse to the B in about 6 minutes – about 2 minutes too late to get on that plane. And all the flights to Seattle that night? Full. So my lovely wife gets to spend a night alone in Seattle while I sleep alone in our bed in Chicago. Nice. Then it’s up at 4am to catch a 6am to Denver, to catch a flight to L.A., and then off to Seattle.

As I type this I’m sitting on a plane in Washington, D.C.. I’ve been on this plane for over 2 hours but we haven’t actually done anything except take a tour around the tarmac and then head back to the gate for more fuel. Awesome. We spent so much time waiting for “de-icing” that we had to go back. Everybody’s being pretty jackassian about it. The lady next too me said, with no warning, “Hey mister, your going to need to move that arm of yours!” My elbow was touching her, you see.

Here’s the breakdown of yesterday’s commute to work:

10:00am get in car to go to airport in Seattle, WA.

12:30pm get on plane.

8:30pm (EST) get off plane.

10:30pm get on plane that has been delayed for an hour.

2:00am plane actually leaves the ground.

3:40am plane lands in Jacksonville, FL.

4:30am I get to my hotel.


Tomorrow I’ll post some pictures of the wedding – I’m going to bed.

Jan 18, 2007

Be the worst

So for about 2 months I’ve been working on this project with Clint Bishop and Nick Drew. Nick was the Project Manager/Tester/Technical lead and build guy. Clint and I were developers. Now that this project is being handed off to a new team (headed by Obie Fernandez), Clint and I have been working with developers other than each other. This is always a tricky thing because you have to justify the code you’ve written while working to make it better.

Even more interesting is that Clint and I are around the same level of Ruby/Rails experience while Obie and Zed (Shaw) are superstars. Not only do they have a long history in the Object Oriented field but they have been doing Ruby professionally for a while now.

Now I’ve been on a Rails project before and I’ve been messing with RoR for more than a year so I’m no chump, but I’m definitely behind the new guys. I was working with Zed for a few days last week and he’s just crazy fast. First of all, he has this behemoth HP laptop that has a tricked-out Linux distro with a custom window manager. Yep, he found a small window manager, got the source code, and tweaked it out. Also he’s a Vim dude. When he works it’s kind of an amazing montage of fast keystrokes, flipping through virtual desktops, and code appearing at an alarmingly fast rate. Then when we switch over to me driving, I fumble around in TextMate whilst he does a good job of not shouting “Faster!” ‘Cause, lets face it, he would have every right to.

A fair number of wise men advise programmers to find situations where you are the worst programmer in the room because that way you learn better. This is true – I’m learning a lot. But it’s a bit rough on your ego.

Now I shouldn’t feel bad, there’s probably only a handful of Ruby devs out there who wouldn’t feel nervous about their abilities after working with Obie and Zed, but its hard not to. It’s like when I was a first year teacher: I worked my ass off showing up early, staying late, obsessively planning my lesions only to see some 10 year vet teacher do it better and with less effort. It’ll come – I’ll get better. But until then I’ll just have to keep asking questions and saying affirmative phrases.

So there's only one more week left before this project goes live. But how did it start? Well, oddly enough, with arts and crafts.

See those faces in the above photo? I, and a bunch of professional consultants, went to Walgreen's, purchased fashion magazines, then cut out and pasted pictures onto paper. Sounds kinda dumb, huh? Well it's not -- lemme try to justify: In our talks with the client it's often helpful to focus on who's going to use the software so that only features for your intended audience get in the code. Also, it helps keeps you focused on who you're developing for. If everyone agrees that, to pick an example, Disco Stu is going to drive the content creation of the site then you might want to think about what he would like and what would annoy him. Why call him "Disco Stu"? Because "20 something bar patron" is boring. And if you're going to be spending a 1-2 weeks discussing what Disco Stu would want in a website, then its cool to have a photo and a summary description of him (and all the other potential users) on the wall.

The whole point of the planning weeks is to get down to some atomic stories that describe features (or part of a feature). These are written on card and stuck on a nearby wall like so:


Why not put them in a computer somewhere? Well we do, but I also like to have a "big visible display" so everyone who walks in the room can see how we're doing. Cards on the left are Backlog, they move into the current week, when someone picks them up they are "in progress", then they go into "dev complete", and then to "signed off." In this picture Ryan has stolen all are signed off card to enter them into the computer. I'd like to get them all back up because the whole point of the "big visible display" is so that anyone who walks into the room can see where we are in the project, who's working on what (my name is attached with sticky note to one of the cards in the picture), and what needs to be done.


Here's the work environment:


Two pair stations. Each has a mac mini hooked up to 2 monitors, 2 keyboards, and 2 mice. So yeah, we do pair programming. This post is already too long without adding my defense of pairing, so I'll save that discussion for later. In the right corner of the picture you can see our build machine. We check into Subversion, Cruise Control runs all the unit tests (using RSpec), then Cruise runs the functional/integration tests (using Selenium Remote Control to open up browser and click around), then if all tests pass it uses Capistrano to push the latest code to a place where people in the company can see and play around with the app. If the tests fail we get electric shocks.

Btw, here's where I'm staying while on this gig:


See that boxy hotel, just right of center? That, my friends, is the mighty Sea Turtle Inn. And by mighty I mean: "Don't stay there." Nice location, crappy rooms.

Yesterday I was laid low by the perils of Amazon's new RESTYSOAPY (because everyone wants their REST to be more SOAP-like) API, but today I have triumphed. I was awful 'fraid Uber dev dudes Zed Shaw and Obie Fernandez would find a 2 second solution to the problem that had plagued me all day. They weren't here yesterday and I was posative they'd come in and spot the problem all flash-like whilst snickering at my code. But of course I was just feeling self-conscious. Obie and I spent a few hours trying to put xml into an url without success until we noticed a line in the doc saying you can call a get OR a post. Oh. All I had to do was do a post with the xml in the body and ... I ran into another problem.

The xml wasn't valid without a whole bunch of stuff in the root element of the question. It should look like this:


<QuestionForm xmlns="http://mechanicalturk.amazonaws.com/
AWSMechanicalTurkDataSchemas/2005-10-01/QuestionForm.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://mechanicalturk.amazonaws.com/
AWSMechanicalTurkDataSchemas/2005-10-01/QuestionForm.xsd
http://mechanicalturk.amazonaws.com/
AWSMechanicalTurkDataSchemas/2005-10-01/QuestionForm.xsd">

Which is nowhere I could find on the Amazon site. Obie and I actually found a website that let us enter question into a web form and then it produced the xml for us! Awesome. Now I can't find it again. The Hell? It was at Awszone.com and I swear it wasn't there yesterday and it isn't there now. Maybe it'll be back. I hope so because it was friggin' cool. Once we saw all the stuff we were missing it was cake to stick it in to our xml and post a HIT (for Human Intelligence Task). Now all we gotta do is figure out how to get our answer back.

Hey, I found it again. It's not linked to very well on the homepage of the site. Anyway, here it is.

So this project I'm on is gonna be all sorts of User-Driven, but we don't want users posting just anything (like say pictures of 9/11 on a page about a Restaurant). But how do you know a pic is "appropriate"? Computers just ain't smart enough to do your job for you when it comes to images. And screening all those photos for badness would take an army of dudes.

Well our friends at Amazon are willing to put you in touch with those dudes. For cheap. Their Mechanical Turk service lets you pragmatically post odd jobs to their vast electronic job board. So now you can take a photo on you site, then post a job saying "Hey look at this image and tell me if it's really Abraham Lincoln or just more boobies and I'll give you 5 cents." Then somebody picks up the job and does it. You can even review their work (spot check) to see how they're doing. Or offer the same job to 5 people and pay 1 cent apiece and see if they all agree. Awesome.

Except that it's not so easy to use. There's 2 problems:

One -- They wanna do both SOAP and REST. So their REST API looks kinda SOAPY. With REST I should just call a http get on a URL with some params, right? Sorta. The job needs to be formatted in xml. And they want you to use URL escaping but not xml escaping. So when I use Ruby's built in escapper: CGI.escape it escapes both url and xml reserved characters. Fine. So then I do the escaping manually and... Ruby's URI parser won't see it as a good URL (which is kinda good, because without xml escaping it's what we call "unsafe"). And now I'm stuck. All I get back is a response that tells me the job is invalid. But I don't know how it's really supposed to look.

Two -- Which leads me to my second problem with the service: Their documentation doesn't have a good End to End example. Sure there's some trivial ones. I found an example, in Ruby no less, that showed me how to do a REST call to query my account balance. A good start. But what I really need is a walk through of some hard problem that takes me all the way up to exactly what the URL should look like in said hard problem. Because right now I'm just guessing about the escaping being the issue. Maybe it's something else crazy easy that I'm missing. More I think about it, that's probably it.