Mark’s Thoughts

I was catching up on Matthew Heusser’s Creative Chaos blog last week (specifically this entry) and followed a link to a post in Mike Kelly’s blog where he came up with 144 tests for a reasonably standard stapler.

As I read through his list, I came up with 6-8 tests that he didn’t include. I started to leave a comment but as I wrote, those 6-8 tests expanded out to more than two dozen.

I was impressed with Mike’s list, and agree with Matthew that all 144 are reasonable, valid tests. Mike started from the marketing materials, specifications, and warranty that were accessible through the above website. I would rather start from the requirements document, as Matthew also mentioned, to see if the stapler that we received matches the description of the stapler that product management desired. The requirements document would also give me a better idea of what behavior would indicate a pass or fail when I perform these tests:

Stapling tests
1. Does it staple x sheets of typing paper in the manner expected? [We need to define what is expected - What is a reasonable value for x that gives us a base for other tests? How much is the staple allowed to wiggle? Can staple tips come back through some/all sheets? How secure does last sheet need to be attached to the rest?]
2. Does it staple 1 sheet of typing paper in the manner expected?
3. Does it staple 2 sheets of typing paper in the manner expected?
4. Does it staple the required maximum number of sheets of typing paper in the manner expected?
5. Can it handle more than the required maximum?
6. What is the actual maximum that it can staple in the manner expected?
7. What happens if I try to staple without any paper in the stapler?

• I would then would need to repeat tests 1-6 with other types of paper:
  Thinner - onionskin or rice paper
  Still thinner - tissue paper
  Thicker - card stock or construction paper
  Still thicker - cardboard (back of a legal pad or from a dress shirt)
• What about combinations of papers?

Tacking tests
8. Does it tack 1 sheet of typing paper to a cork board? [We probably should define what is expected here for tacking as we did above - Can the staple dent the paper/cork? Should it hold the paper up with one tack? For how long? Do two staples hold up more papers than one? Are there different expectations if you tack onto a vertical surface (wall) versus an inverted horizontal surface (ceiling)?]
9. Does it tack 2 sheets of typing paper to the cork board?
10. Does it tack the required maximum number of sheets of typing paper to the cork board?
11. Can it tack more than the required maximum?
12. What is the actual maximum that it can tack to the cork board?

• I would want to further define tests for different types/combinations of papers, other surfaces (soft/hard wood, stone, cardboard, drywall, etc), multiple staples in a test, walls vs. ceilings, et cetera based on the requirements document.

Staple tests
13. Does the stapler perform in a reasonable manner with no staples loaded?
14. Does the stapler staple correctly with only 1 staple loaded?
15. If not, what is the minimum number of staples that must be loaded for the stapler to staple correctly?
16. Does the stapler staple correctly with 210 staples loaded?
17. With more if the stapler will hold them?
18. Does the stapler staple correctly with 210 individual staples loaded?

• I could also ask the same question in regards to tacking.

Anvil tests
19. Can the anvil be set somewhere other than the curling-in or curling-out position?
20. Is the behavior of the stapler reasonable/acceptable if the anvil is set somewhere other than the curling-in or curling-out position?

My first reaction after writing up these tests was to try to reason out why Mike came up with those tests and I came up with these tests. I have only read a couple of posts on Mike’s blog, so I really have no way to get into his head. I was able to do some introspection on my tests and came up with:

• My first tests on any new product are exploratory, based on my innate sense of curiosity. I will play with every button in your front seat if you give me a ride somewhere. The first test cases that I thought of as I read through Mike’s list were based on my experience playing with staplers. (e.g., 2, 14, 15, 18, 19, 20)
• I’ve been working with a strong development team. If the product manager asked for a stapler, the coders would build a stapler. My biggest contribution in an iteration was often finding the places where the two sides were making assumptions — Did the project manager care about the color of the stapler, since he hadn’t specified?
• I’ve been working on a team that did not do much pair programming. I have not yet worked in a pair programming environment, but I have worked for a company where we did code reviews. A second set of eyes on code can find a surprising number of problems, and my experience is that they tend to happen around edge cases like many of the ones above.

Now we have at least 188 tests (if you run the stapling tests with 5 different kinds of paper, or more if you tack to different surfaces and/or with different kinds of paper). Mike’s conclusions still hold up. I doubt that it took two hours between us to come up with the these tests for a common item that has been around for almost 130 years and has no batteries, buttons, or wires. The software that I have dealt with in my career has almost always been written using the latest, greatest languages and tools to solve exciting problems in imaginative ways — “infinitely more complex than a stapler.”

While Mike looks at the overwhelming pile of tests and sees “Our job is one of discovery,” I observe that our job is more about priorities. As testers, we constantly need to decide which of these hundreds of tests to run next, while coming up with more tests around each new feature. We also need to decide which tests we can and should script to be run regularly. The upside of the job is that we also get to break things.

Now I want product management to tell me how far we want the staple to fly, so that I can start testing some staplers.