Introduction to Acceptance Test Driven Development

Beginning with the End
in Mind: Driving
Development with
Acceptance Tests
Elisabeth Hendrickson
Quality Tree Software, Inc.
www.qualitytree.com
esh@qualitytree.com

Last updated November 10, 2009
This work is licensed under the Creative Commons Attribution 3.0
United States License. View a copy of this license.

Acceptance-Test Driven Development (ATDD) Cycle
0&1(/.2
3,.4#15
-21&6
-21&6
-21&6
-21&6
-21&6

Copyright © 2010 Quality Tree Software, Inc.
0&1(/.2
!$-./-- '" (

*
+ #" ,

) &" "
*
!"#$%"& !"%"#17
(Model developed with Pekka Klärck, Bas Vodde, and Craig Larman.)

ATDD: Discuss

!"#$%&'
()&*+#, !"#$%&

'(&%$)*#(&+(,
-'#".
-'#".
-'#".
-'#". Concrete
Examples with
-'#". Expectations

(Slide developed in collaboration with Pekka Klärck.)

ATDD: Develop

'*&+%","-."/,/

'()*+"&
%#$&
!"# ,-(*
.&/*/
!"#$
&
' (" )

$ %" "

%#$&
&


ATDD: Deliver

Product
Feature Feature
Feature Feature
Feature
Feature
FeatureFeature
Feature

Start with the Story

As an administrator, I want users
creating accounts to be required

to choose secure passwords so
that their accounts cannot be
hacked by someone using a
password guessing program.

Discuss
And if a user provides Who’s in the
an insecure password, room?
display an error
Product
message.
Owner,
Testers,

Developers,
and anyone
else who will
touch the
What does story.
“secure” mean
to you?
At least 6 characters with
at least one letter, one
symbol, and one number.

Capture Concrete Expectations and Examples
Password Valid? Can be
expressed as
“p@ssw0rd” Yes “Given - When -
“p@s5” No Then”
“passw0rd” No

Given a user is creating an account
“p@ssword” No
When they specify an insecure
“@#$%1234” No password
Or can be Then they see a message,
expressed in tables “Passwords must be at least 6
characters long with at least one
Or in other formats letter, one number, and one
depending on the symbol.”
Framework

Why ATDD?
Reason #1:
Drive Out Ambiguity
and Clarify
Expectations

This is not an Argument about a Bug
“Bug Triage Meeting”
The Tuesday before release.

It’s a bug.

No it’s not.
Whether or not
it’s a bug, if we
Is too. make a change
we’ll blow the
IS NOT. schedule.

IS TOO!

NOT NOT NOT!

Acceptance Tests Define Scope


A Short Digression on
ATDD-Friendly Tools

Examples of ATDD-Friendly Frameworks

• Cucumber: a Ruby-based BDD tool that
supports “Given-When-Then”
• Fitnesse: a table-driven framework that
uses a wiki for displaying and editing

tests
• Robot Framework: keyword-driven
framework that supports text or tables
• Concordion: Java-based framework for
expressing expectations in prose

Frameworks, Interfaces, and Drivers
Created in Code written during
collaboration. development in a GUI Driver
Format defined programming language (e.g. SeleniumRC)
by the determined by the
framework framework.

GUI

Public API
Test
Natural “Guts”
“Fixture”
Language
Expectations Other
interfaces

Implementation

Characteristics of ATDD-Friendly Frameworks
• Support expressing expectations in a language
and format that fits the context
• Support collaboration among the whole team
including developers, testers, & the product owner

• Connect expectations to the system under test
with a minimum of test code (“fixtures,” “libraries,”
“steps”) to leverage expectations as executable
requirements
• Play nicely with source control systems and
continuous integration
• Pluggable to support a variety of interfaces

Contrasting View: Traditional Test Automation

GUI Automated Test
Scripts:
Public API Combination of
“Guts” business-facing

expectations and
Other implementation
interfaces details.

Implementation
Written or recorded
after the fact.
Expectations are
translated, not
leveraged.

Take the Acceptance Tests…
Password Valid?
“p@ssw0rd” Yes
“p@s5” No
“passw0rd” No

Given a user is creating an account
“p@ssword” No
When they specify an insecure
“@#$%1234” No password

(Note that these Then they see a message,
expectations are “Passwords must be at least 6
implementation-agnostic characters long with at least one
and express just the letter, one number, and one
essence of the symbol.”
expectation.)

…and Write the Code

Password Valid? Test GUI
“p@ssw0rd” Yes “Fixture”
“p@s5” No Public API

“passw0rd” No
“Guts”
“p@ssword” No
“@#$%1234” No Other
interfaces

Implementation

Why ATDD?
Reason #2:
Make progress visible.

Are We There Yet?


Why ATDD?
Reason #3:
Leverage, Efficiency,
and Executable
Specifications

Traditional Approaches

Requirements
Management
System

Traceability
Matrix
Test
Technical Management
Specifications ? System

Efficiency, Reusability, Maintainability
Implementation
The tests New interface?
change? Make a
define the Just add a test
localized update to
requirements. fixture
the test fixture.

Natural Test
“Fixture” …
Language
Expectations
New Interface

Implementation

(No reconciling (Preserve valid (Leverage relevant
multiple, duplicate expectations.) expectations.)
artifacts.)

How does ATDD fit with
the rest of the process?

ATDD: Part of an Agile Testing Strategy

Support Critique

Business-

facing Acceptance Tests Exploratory Testing

Code- Reviews, inspections,
facing Unit Tests pairing, code
quality metrics

(A variation on Brian Marick’s Agile Testing Quadrants as
published in his essay “Agile Testing Directions”)

Tests are Versioned with the Code
Source Control

Natural
Language
Expectations

Test
“Fixture”
Code
Unit Tests

Production
Code

Tests Execute as Part of the Automated Build

Continuous Integration

Images courtesy Mike Clark, www.pragmaticautomation.com.
Used with permission.

Why ATDD?
Reason #4:
No more bugs. (No, I’m
not kidding. But yes,
there is a catch.)

Zero Tolerance for Bugs

!"#$%"%&'%( !"#$%"%&'%(
)'*+, )'*+,

But Not Everything is a Bug

In this context,
a BUG
is behavior that
violates the letter

or spirit of the
Product Owner’s
expectations for the
implemented story.

If the behavior does not violate expectations
related to the implemented stories, it’s an item
for the backlog.

Given all that…
Why not ATDD?

Resources
Adzic, Gojko (2009). Bridging the Communication Gap:
Specification by Example and Agile Acceptance Testing.
Neuri Limited.
Crispin, Lisa (2005). “Using Customer Tests to Drive
Development.” Methods & Tools. Summer 2005 Issue.

Available online at http://www.methodsandtools.com/
archive/archive.php?id=23
Crispin, L., & Gregory, J. (2009). Agile Testing: A Practical
Guide for Testers and Agile Teams (Addison-Wesley
Signature Series (Cohn)). New York: Addison-wesley
Professional.
Marick, Brian (2003). “Agile Testing Directions.” (An
explanation of business-facing v. code- facing tests.)
Available online at http://www.exampler.com/old-blog/
2003/08/22/#agile- testing-project-2

Introduction to Acceptance Test Driven Development

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Introduction to Acceptance Test Driven Development

Similar to Introduction to Acceptance Test Driven Development (20)

More from Elisabeth Hendrickson

More from Elisabeth Hendrickson (7)

Recently uploaded

Recently uploaded (20)

Introduction to Acceptance Test Driven Development