• describe what I’m trying to do and why
• describe how I’d like to solve the problem, pointing to similar Watir functions
• describe how I’ve worked around the issue for now.
• ask for other ways to accomplish my goals

I look forward to being able to contribute some of these solutions to Watir if they are needed.

I still feel like I am learning Ruby, but went ahead and downloaded the Watir source code yesterday while writing the draft of this post and got the unit tests running this evening while editing it.

Look for posts on frames and dialogs in the next couple of days.

Rasta

At one of the first meetings we discussed using Excel to drive software tests. Hugh McGowan and Bret Pettichord had each implemented solutions that were not robust enough to share. Hugh and Rafael Torres took on the project and provided status reports and previews to the group. I took a look at it last Fall when I was teaching myself Ruby and implemented an example and provided some feedback. Version 0.1.1 was released this evening and more information can be found at rasta.rubyforge.org.

AWUG

The origins of the Homebrew Testers was an informal Watir interest group. At an early meeting, we started talking about forming a formal Watir Users Group in Austin. There have been multiple ideas thrown around over the months, and finally a group is forming. Bret posted the announcement on his blog this evening: Hugh will be presenting Keyword Driven Testing with Rasta at the first meeting this Wednesday.

CITCON

The first meeting of the Homebrew Testers had been announced just before CITCON last year in D/FW. I did not decide to attend that first meeting until I got to know Bret and Jason Darling better at CITCON. This year’s conference is coming up in Denver on April 4-5. It is free to attend and there will be lots of great discussions about various continuous integration and testing topics. It is organized using the open spaces format, so the sessions will be about the topics of interest to the attendees. I’m looking forward to it.

As I read through his list, I came up with 6-8 tests that he didn’t include. I started to leave a comment but as I wrote, those 6-8 tests expanded out to more than two dozen.

I was impressed with Mike’s list, and agree with Matthew that all 144 are reasonable, valid tests. Mike started from the marketing materials, specifications, and warranty that were accessible through the above website. I would rather start from the requirements document, as Matthew also mentioned, to see if the stapler that we received matches the description of the stapler that product management desired. The requirements document would also give me a better idea of what behavior would indicate a pass or fail when I perform these tests:

Stapling tests
1. Does it staple x sheets of typing paper in the manner expected? [We need to define what is expected - What is a reasonable value for x that gives us a base for other tests? How much is the staple allowed to wiggle? Can staple tips come back through some/all sheets? How secure does last sheet need to be attached to the rest?]
2. Does it staple 1 sheet of typing paper in the manner expected?
3. Does it staple 2 sheets of typing paper in the manner expected?
4. Does it staple the required maximum number of sheets of typing paper in the manner expected?
5. Can it handle more than the required maximum?
6. What is the actual maximum that it can staple in the manner expected?
7. What happens if I try to staple without any paper in the stapler?

• I would then would need to repeat tests 1-6 with other types of paper:
  Thinner – onionskin or rice paper
  Still thinner – tissue paper
  Thicker – card stock or construction paper
  Still thicker – cardboard (back of a legal pad or from a dress shirt)
• What about combinations of papers?

Tacking tests
8. Does it tack 1 sheet of typing paper to a cork board? [We probably should define what is expected here for tacking as we did above - Can the staple dent the paper/cork? Should it hold the paper up with one tack? For how long? Do two staples hold up more papers than one? Are there different expectations if you tack onto a vertical surface (wall) versus an inverted horizontal surface (ceiling)?]
9. Does it tack 2 sheets of typing paper to the cork board?
10. Does it tack the required maximum number of sheets of typing paper to the cork board?
11. Can it tack more than the required maximum?
12. What is the actual maximum that it can tack to the cork board?

• I would want to further define tests for different types/combinations of papers, other surfaces (soft/hard wood, stone, cardboard, drywall, etc), multiple staples in a test, walls vs. ceilings, et cetera based on the requirements document.

Staple tests
13. Does the stapler perform in a reasonable manner with no staples loaded?
14. Does the stapler staple correctly with only 1 staple loaded?
15. If not, what is the minimum number of staples that must be loaded for the stapler to staple correctly?
16. Does the stapler staple correctly with 210 staples loaded?
17. With more if the stapler will hold them?
18. Does the stapler staple correctly with 210 individual staples loaded?

• I could also ask the same question in regards to tacking.

Anvil tests
19. Can the anvil be set somewhere other than the curling-in or curling-out position?
20. Is the behavior of the stapler reasonable/acceptable if the anvil is set somewhere other than the curling-in or curling-out position?

My first reaction after writing up these tests was to try to reason out why Mike came up with those tests and I came up with these tests. I have only read a couple of posts on Mike’s blog, so I really have no way to get into his head. I was able to do some introspection on my tests and came up with:

• My first tests on any new product are exploratory, based on my innate sense of curiosity. I will play with every button in your front seat if you give me a ride somewhere. The first test cases that I thought of as I read through Mike’s list were based on my experience playing with staplers. (e.g., 2, 14, 15, 18, 19, 20)
• I’ve been working with a strong development team. If the product manager asked for a stapler, the coders would build a stapler. My biggest contribution in an iteration was often finding the places where the two sides were making assumptions — Did the project manager care about the color of the stapler, since he hadn’t specified?
• I’ve been working on a team that did not do much pair programming. I have not yet worked in a pair programming environment, but I have worked for a company where we did code reviews. A second set of eyes on code can find a surprising number of problems, and my experience is that they tend to happen around edge cases like many of the ones above.

Now we have at least 188 tests (if you run the stapling tests with 5 different kinds of paper, or more if you tack to different surfaces and/or with different kinds of paper). Mike’s conclusions still hold up. I doubt that it took two hours between us to come up with the these tests for a common item that has been around for almost 130 years and has no batteries, buttons, or wires. The software that I have dealt with in my career has almost always been written using the latest, greatest languages and tools to solve exciting problems in imaginative ways — “infinitely more complex than a stapler.”

While Mike looks at the overwhelming pile of tests and sees “Our job is one of discovery,” I observe that our job is more about priorities. As testers, we constantly need to decide which of these hundreds of tests to run next, while coming up with more tests around each new feature. We also need to decide which tests we can and should script to be run regularly. The upside of the job is that we also get to break things.

Now I want product management to tell me how far we want the staple to fly, so that I can start testing some staplers.

I’ve been reading the software-testing email list and various testing blogs for a few months now.  I keep seeing common themes that I want to explore further and have decided that creating my own blog will be the best way to do it.

I am looking forward to sharing my thoughts with you, but I’m even more excited about being in conversation about software testing with you and the rest of the testing community.