Discussion:
[Cucumber] Do you believe BDD scenarios' quality matter?
(too old to reply)
Gabriel Oliveira
2017-09-03 17:55:06 UTC
Permalink
Raw Message
Hi folks,

Have you been practicing BDD (Behavior-Driven Development) lately and
believe scenarios' quality matter?

I'm writing my graduate thesis on that subject and would like to invite you
all to answer a (really) small 2 questions survey about it in here:
https://goo.gl/forms/dheA8XK0asiNUVze2

If you're willing to spend a little more time to talk to me about that
topic, please remember to fill out your contact information there - or
comment on this thread here - and I will promptly get in touch :) !

Best regards,
--
*Gabriel P.A. de Oliveira*
*https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>*
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
George Dinwiddie
2017-09-13 19:26:02 UTC
Permalink
Raw Message
Gabriel,

As short as it is, I find your form impossible for me to answer. The
statement "It is important to have a standard set of criteria to
evaluate written BDD scenarios' quality" implies that there can be such
a set of criteria. I have not yet found it possible, though I have been
giving talks and counseling people on the topic for years.

What qualities do you seek in scenarios? I want them to
- Communicate between roles and constituencies in an organization
- Communicate across time and space
- Describe the intended functionality
- Provide a target for development
- Check the system for unintended regressions
- Provide clear diagnostics when things go wrong
There's probably more, but these come readily to mind.

In any event, I cannot imagine a standard set of criteria to evaluate
scenarios for these qualities without regard to the context. It's like a
standard set of criteria to evaluate Shakespeare's sonnets--which is the
best sonnet?

- George
Post by Gabriel Oliveira
Hi folks,
Have you been practicing BDD (Behavior-Driven Development) lately and
believe scenarios' quality matter?
I'm writing my graduate thesis on that subject and would like to invite
https://goo.gl/forms/dheA8XK0asiNUVze2
If you're willing to spend a little more time to talk to me about that
topic, please remember to fill out your contact information there - or
comment on this thread here - and I will promptly get in touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
/https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>/
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Gabriel Oliveira
2017-09-13 20:01:34 UTC
Permalink
Raw Message
Hi George,

It's interesting to hear that you found the questions impossible to answer
- provoking reactions from people was part of their goal.

We started looking at BDD written scenarios quality inspired by
characteristics of good requirements. While even User Stories have their
INVEST criteria to help, BDD scenarios lack such a common list of words to
guide the reviewer/writer in their job of handling BDD scenarios and what a
"good" BDD scenario is. I agree that a well-written requirement is
different from a good poem, but I find it interesting to clarify what are
the characteristics that make each one of them "good".

We seek to acquire this understanding (or at least, to reduce our lack of
understanding) by summarizing multiple people opinions taken from
interviews - the survey was just the beginning of it.

Would you be available to further talk about the subject in a skype call?

Of course, this invitation is extendable to anyone who found the matter
interesting and would like to provide their opinion :)

Best regards,
--
*Gabriel P.A. de Oliveira*
*https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>*
Post by George Dinwiddie
Gabriel,
As short as it is, I find your form impossible for me to answer. The
statement "It is important to have a standard set of criteria to evaluate
written BDD scenarios' quality" implies that there can be such a set of
criteria. I have not yet found it possible, though I have been giving talks
and counseling people on the topic for years.
What qualities do you seek in scenarios? I want them to
- Communicate between roles and constituencies in an organization
- Communicate across time and space
- Describe the intended functionality
- Provide a target for development
- Check the system for unintended regressions
- Provide clear diagnostics when things go wrong
There's probably more, but these come readily to mind.
In any event, I cannot imagine a standard set of criteria to evaluate
scenarios for these qualities without regard to the context. It's like a
standard set of criteria to evaluate Shakespeare's sonnets--which is the
best sonnet?
- George
Post by Gabriel Oliveira
Hi folks,
Have you been practicing BDD (Behavior-Driven Development) lately and
believe scenarios' quality matter?
I'm writing my graduate thesis on that subject and would like to invite
https://goo.gl/forms/dheA8XK0asiNUVze2
If you're willing to spend a little more time to talk to me about that
topic, please remember to fill out your contact information there - or
comment on this thread here - and I will promptly get in touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
/https://gpaoliveirablog.wordpress.com <https://gpaoliveirablog.wordp
ress.com>/
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
--- You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Aslak Hellesøy
2017-09-13 21:03:47 UTC
Permalink
Raw Message
Hi Gabriel,

Your form states:

"Since it is well known that executable solution requirements are one of
many potential causes of a project failure, we come to ask some questions
about this subject."

1) Are you saying that project failure can be partly caused by the fact
that specifications are executable?
2) After 12 years in this field I've never heard this - on what basis are
you claiming this is well known?

Cheers,
Aslak
Post by Gabriel Oliveira
Hi folks,
Have you been practicing BDD (Behavior-Driven Development) lately and
believe scenarios' quality matter?
I'm writing my graduate thesis on that subject and would like to invite
https://goo.gl/forms/dheA8XK0asiNUVze2
If you're willing to spend a little more time to talk to me about that
topic, please remember to fill out your contact information there - or
comment on this thread here - and I will promptly get in touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
*https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>*
--
The Cucumber logo is the intellectual property of Cucumber Ltd, a limited
company registered in Scotland, number 456793.

UK Headquarters: Cucumber Ltd, Drumsyniebeg, Lochgoilhead, Cairndow,
Argyll, PA24 8AN UK.

CONFIDENTIALITY NOTICE: The information in this e-mail is confidential and
privileged; it is intended for use solely by the individual or entity named
as the recipient hereof. Disclosure, copying, distribution, or use of the
contents of this e-mail by persons other than the intended recipient is
strictly prohibited and may violate applicable laws. If you have received
this e-mail in error, please delete the original message and notify us by
email immediately. Thank you. Cucumber Ltd.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Gabriel Oliveira
2017-09-14 06:18:06 UTC
Permalink
Raw Message
Hi Aslak,

Thanks for pointing this out - this was a mistake on our part. The intent
was to state that fact that bad requirements are one of the many causes of
project failures.

Already fixed on bit.ly/bdd_quality
Post by Aslak Hellesøy
Hi Gabriel,
"Since it is well known that executable solution requirements are one of
many potential causes of a project failure, we come to ask some questions
about this subject."
1) Are you saying that project failure can be partly caused by the fact
that specifications are executable?
2) After 12 years in this field I've never heard this - on what basis are
you claiming this is well known?
Cheers,
Aslak
Post by Gabriel Oliveira
Hi folks,
Have you been practicing BDD (Behavior-Driven Development) lately and
believe scenarios' quality matter?
I'm writing my graduate thesis on that subject and would like to invite
https://goo.gl/forms/dheA8XK0asiNUVze2
If you're willing to spend a little more time to talk to me about that
topic, please remember to fill out your contact information there - or
comment on this thread here - and I will promptly get in touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
*https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>*
The Cucumber logo is the intellectual property of Cucumber Ltd, a limited
company registered in Scotland, number 456793.
UK Headquarters: Cucumber Ltd, Drumsyniebeg, Lochgoilhead, Cairndow,
Argyll, PA24 8AN UK.
CONFIDENTIALITY NOTICE: The information in this e-mail is confidential and
privileged; it is intended for use solely by the individual or entity named
as the recipient hereof. Disclosure, copying, distribution, or use of the
contents of this e-mail by persons other than the intended recipient is
strictly prohibited and may violate applicable laws. If you have received
this e-mail in error, please delete the original message and notify us by
email immediately. Thank you. Cucumber Ltd.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
aslak hellesoy
2017-09-14 09:32:09 UTC
Permalink
Raw Message
Thanks for the clarification Gabriel.

In the past few years, after helping dozens of organisations and hundreds
of people adopt BDD I have come to the conclusion that it is
counter-productive to have 3 amigos (BAs, testers and programmers) write
executable specifications (Gherkin) together.

What I encourage instead is that they create *examples* together. Examples
are not executable specifications - they are simply conversation topics,
often jotted down on a card as a sketch or a few words. I recommend Example
Mapping (https://cucumber.io/blog/2015/12/08/example-mapping-introduction)
as a technique to create examples together.

When conversations happen around lo-fi examples like this in an example
mapping session, stakeholders tend to uncover a lot of misunderstandings
and gain new insight. In my experience, this does not happen when the
conversations happen around executable specifications (Gherkin). Instead,
conversations tend to revolve around grammar and formulation, diverting the
attention from the problem domain and the examples themselves. Writing
specifications together is also slow and tedious, and people don't enjoy
these sessions as much. When it's not enjoyable, people stop doing it
pretty quickly. Example Mapping on the other hand is energising and fun.

BDD is a variant of TDD, where programmers write code guided by failing
tests. In my experience, this workflow does not work well when the
specifications were written collaboratively. When the programmers starts
the red-green-refactor cycle with collaboratively written scenarios, they
quickly realise they need to reformulate those scenarios, for a number of
reasons. They're incomplete, they're ambiguous etc. When scenarios start
passing, the scenarios are often very different from what the diverse group
created together. This often causes frustration: "You've completely
rewritten it - this is not what we wrote together - I don't understand
this".

So instead I recommend programmers write the executable specifications,
perhaps with assistance from a tester. They use the example map as input
when they do this, and do their best to capture the spirit of the examples
and the conversations. When they're done (or better - while they're doing
it), they show the scenarios to the BAs/POs and ask for feedback: Do you
understand this? Does it reflect what we discussed during example mapping?
Then they tweak it, trying to strike a good balance between 3 qualities:
Specification, Test and Documentation.

BDD isn't a process where you can write the specification up-front - even
when it happens collaboratively. They take shape as you go, in parallel
with development. The only thing you can do up-front (before coding) is to
reduce the misunderstandings, but executable specifications isn't a good
medium for that I think.

Cheers,
Aslak

On Thu, Sep 14, 2017 at 7:18 AM, Gabriel Oliveira <
Post by Gabriel Oliveira
Hi Aslak,
Thanks for pointing this out - this was a mistake on our part. The intent
was to state that fact that bad requirements are one of the many causes of
project failures.
Already fixed on bit.ly/bdd_quality
Post by Aslak Hellesøy
Hi Gabriel,
"Since it is well known that executable solution requirements are one of
many potential causes of a project failure, we come to ask some questions
about this subject."
1) Are you saying that project failure can be partly caused by the fact
that specifications are executable?
2) After 12 years in this field I've never heard this - on what basis are
you claiming this is well known?
Cheers,
Aslak
Post by Gabriel Oliveira
Hi folks,
Have you been practicing BDD (Behavior-Driven Development) lately and
believe scenarios' quality matter?
I'm writing my graduate thesis on that subject and would like to invite
https://goo.gl/forms/dheA8XK0asiNUVze2
If you're willing to spend a little more time to talk to me about that
topic, please remember to fill out your contact information there - or
comment on this thread here - and I will promptly get in touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
*https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>*
The Cucumber logo is the intellectual property of Cucumber Ltd, a limited
company registered in Scotland, number 456793.
UK Headquarters: Cucumber Ltd, Drumsyniebeg, Lochgoilhead, Cairndow,
Argyll, PA24 8AN UK.
CONFIDENTIALITY NOTICE: The information in this e-mail is confidential
and privileged; it is intended for use solely by the individual or entity
named as the recipient hereof. Disclosure, copying, distribution, or use of
the contents of this e-mail by persons other than the intended recipient is
strictly prohibited and may violate applicable laws. If you have received
this e-mail in error, please delete the original message and notify us by
email immediately. Thank you. Cucumber Ltd.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Premdas
2017-09-14 14:15:30 UTC
Permalink
Raw Message
This ties in quite nicely with my idea/opinion that any Cucumber scenario
that has examples in it, particularly tables is in some way incomplete.

You complete the scenario by extracting the spirit/meaning of the examples
and in the process name things. When you have done this all the tables of
examples just disappear and you are left with named concepts specific to
your context.

All best

Andrew
Post by aslak hellesoy
Thanks for the clarification Gabriel.
In the past few years, after helping dozens of organisations and hundreds
of people adopt BDD I have come to the conclusion that it is
counter-productive to have 3 amigos (BAs, testers and programmers) write
executable specifications (Gherkin) together.
What I encourage instead is that they create *examples* together. Examples
are not executable specifications - they are simply conversation topics,
often jotted down on a card as a sketch or a few words. I recommend Example
Mapping (https://cucumber.io/blog/2015/12/08/example-mapping-introduction)
as a technique to create examples together.
When conversations happen around lo-fi examples like this in an example
mapping session, stakeholders tend to uncover a lot of misunderstandings
and gain new insight. In my experience, this does not happen when the
conversations happen around executable specifications (Gherkin). Instead,
conversations tend to revolve around grammar and formulation, diverting the
attention from the problem domain and the examples themselves. Writing
specifications together is also slow and tedious, and people don't enjoy
these sessions as much. When it's not enjoyable, people stop doing it
pretty quickly. Example Mapping on the other hand is energising and fun.
BDD is a variant of TDD, where programmers write code guided by failing
tests. In my experience, this workflow does not work well when the
specifications were written collaboratively. When the programmers starts
the red-green-refactor cycle with collaboratively written scenarios, they
quickly realise they need to reformulate those scenarios, for a number of
reasons. They're incomplete, they're ambiguous etc. When scenarios start
passing, the scenarios are often very different from what the diverse group
created together. This often causes frustration: "You've completely
rewritten it - this is not what we wrote together - I don't understand
this".
So instead I recommend programmers write the executable specifications,
perhaps with assistance from a tester. They use the example map as input
when they do this, and do their best to capture the spirit of the examples
and the conversations. When they're done (or better - while they're doing
it), they show the scenarios to the BAs/POs and ask for feedback: Do you
understand this? Does it reflect what we discussed during example mapping?
Specification, Test and Documentation.
BDD isn't a process where you can write the specification up-front - even
when it happens collaboratively. They take shape as you go, in parallel
with development. The only thing you can do up-front (before coding) is to
reduce the misunderstandings, but executable specifications isn't a good
medium for that I think.
Cheers,
Aslak
On Thu, Sep 14, 2017 at 7:18 AM, Gabriel Oliveira <
Post by Gabriel Oliveira
Hi Aslak,
Thanks for pointing this out - this was a mistake on our part. The intent
was to state that fact that bad requirements are one of the many causes of
project failures.
Already fixed on bit.ly/bdd_quality
Post by Aslak Hellesøy
Hi Gabriel,
"Since it is well known that executable solution requirements are one of
many potential causes of a project failure, we come to ask some questions
about this subject."
1) Are you saying that project failure can be partly caused by the fact
that specifications are executable?
2) After 12 years in this field I've never heard this - on what basis
are you claiming this is well known?
Cheers,
Aslak
On Thursday, September 7, 2017 at 2:17:45 PM UTC+1, Gabriel Oliveira
Post by Gabriel Oliveira
Hi folks,
Have you been practicing BDD (Behavior-Driven Development) lately and
believe scenarios' quality matter?
I'm writing my graduate thesis on that subject and would like to invite
https://goo.gl/forms/dheA8XK0asiNUVze2
If you're willing to spend a little more time to talk to me about that
topic, please remember to fill out your contact information there - or
comment on this thread here - and I will promptly get in touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
*https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>*
The Cucumber logo is the intellectual property of Cucumber Ltd, a
limited company registered in Scotland, number 456793.
UK Headquarters: Cucumber Ltd, Drumsyniebeg, Lochgoilhead, Cairndow,
Argyll, PA24 8AN UK.
CONFIDENTIALITY NOTICE: The information in this e-mail is confidential
and privileged; it is intended for use solely by the individual or entity
named as the recipient hereof. Disclosure, copying, distribution, or use of
the contents of this e-mail by persons other than the intended recipient is
strictly prohibited and may violate applicable laws. If you have received
this e-mail in error, please delete the original message and notify us by
email immediately. Thank you. Cucumber Ltd.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups
"Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups
"Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
George Dinwiddie
2017-09-14 15:57:15 UTC
Permalink
Raw Message
Andrew,

I'm not exactly sure what you mean by this. Can you give an example of
extracting the spirit/meaning of the examples and naming them?

- George
Post by Andrew Premdas
This ties in quite nicely with my idea/opinion that any Cucumber
scenario that has examples in it, particularly tables is in some way
incomplete.
You complete the scenario by extracting the spirit/meaning of the
examples and in the process name things. When you have done this all the
tables of examples just disappear and you are left with named concepts
specific to your context.
All best
Andrew
Thanks for the clarification Gabriel.
In the past few years, after helping dozens of organisations and
hundreds of people adopt BDD I have come to the conclusion that it
is counter-productive to have 3 amigos (BAs, testers and
programmers) write executable specifications (Gherkin) together.
What I encourage instead is that they create *examples* together.
Examples are not executable specifications - they are simply
conversation topics, often jotted down on a card as a sketch or a
few words. I recommend Example Mapping
(https://cucumber.io/blog/2015/12/08/example-mapping-introduction
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction>)
as a technique to create examples together.
When conversations happen around lo-fi examples like this in an
example mapping session, stakeholders tend to uncover a lot of
misunderstandings and gain new insight. In my experience, this does
not happen when the conversations happen around executable
specifications (Gherkin). Instead, conversations tend to revolve
around grammar and formulation, diverting the attention from the
problem domain and the examples themselves. Writing specifications
together is also slow and tedious, and people don't enjoy these
sessions as much. When it's not enjoyable, people stop doing it
pretty quickly. Example Mapping on the other hand is energising and fun.
BDD is a variant of TDD, where programmers write code guided by
failing tests. In my experience, this workflow does not work well
when the specifications were written collaboratively. When the
programmers starts the red-green-refactor cycle with collaboratively
written scenarios, they quickly realise they need to reformulate
those scenarios, for a number of reasons. They're incomplete,
they're ambiguous etc. When scenarios start passing, the scenarios
are often very different from what the diverse group created
together. This often causes frustration: "You've completely
rewritten it - this is not what we wrote together - I don't
understand this".
So instead I recommend programmers write the executable
specifications, perhaps with assistance from a tester. They use the
example map as input when they do this, and do their best to capture
the spirit of the examples and the conversations. When they're done
(or better - while they're doing it), they show the scenarios to the
BAs/POs and ask for feedback: Do you understand this? Does it
reflect what we discussed during example mapping? Then they tweak
Specification, Test and Documentation.
BDD isn't a process where you can write the specification up-front -
even when it happens collaboratively. They take shape as you go, in
parallel with development. The only thing you can do up-front
(before coding) is to reduce the misunderstandings, but executable
specifications isn't a good medium for that I think.
Cheers,
Aslak
On Thu, Sep 14, 2017 at 7:18 AM, Gabriel Oliveira
Hi Aslak,
Thanks for pointing this out - this was a mistake on our part.
The intent was to state that fact that bad requirements are one
of the many causes of project failures.
Already fixed on bit.ly/bdd_quality <http://bit.ly/bdd_quality>
On Wed, Sep 13, 2017 at 11:03 PM, Aslak Hellesøy
Hi Gabriel,
"Since it is well known that executable solution
requirements are one of many potential causes of a project
failure, we come to ask some questions about this subject."
1) Are you saying that project failure can be partly caused
by the fact that specifications are executable?
2) After 12 years in this field I've never heard this - on
what basis are you claiming this is well known?
Cheers,
Aslak
On Thursday, September 7, 2017 at 2:17:45 PM UTC+1, Gabriel
Hi folks,
Have you been practicing BDD (Behavior-Driven
Development) lately and believe scenarios' quality matter?
I'm writing my graduate thesis on that subject and would
like to invite you all to answer a (really) small 2
https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>
If you're willing to spend a little more time to talk to
me about that topic, please remember to fill out your
contact information there - or comment on this thread
here - and I will promptly get in touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
/https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>/
The Cucumber logo is the intellectual property of Cucumber
Ltd, a limited company registered in Scotland, number 456793.
UK Headquarters: Cucumber Ltd, Drumsyniebeg, Lochgoilhead,
Cairndow, Argyll, PA24 8AN UK.
CONFIDENTIALITY NOTICE: The information in this e-mail is
confidential and privileged; it is intended for use solely
by the individual or entity named as the recipient hereof.
Disclosure, copying, distribution, or use of the contents of
this e-mail by persons other than the intended recipient is
strictly prohibited and may violate applicable laws. If you
have received this e-mail in error, please delete the
original message and notify us by email immediately. Thank
you. Cucumber Ltd.
--
Posting rules: http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
---
You received this message because you are subscribed to the
Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
Posting rules: http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
---
You received this message because you are subscribed to the
Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
Posting rules: http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
---
You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it,
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org <http://blog.andrew.premdas.org>
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Premdas
2017-09-14 16:36:43 UTC
Permalink
Raw Message
Post by George Dinwiddie
Andrew,
I'm not exactly sure what you mean by this. Can you give an example of
extracting the spirit/meaning of the examples and naming them?
- George
Sure, lets take something everyone knows about registrations (excuse bad
syntax I don't use outlines examples or tables anymore).

Scenario Outline: Bad registrations
When I register with <account> <password>
Then I should not be register

Examples:
Account | Password
bad_account | good_password
good_account | 1234
good_account | sdfdsflsdfsdf3313
good_account | wieikjkeiiow


OK so our business peeps have come up with some examples of
account|password combinations that should fail. Now we have to take each
pair and try and extract the spirit/meaning. I'll start with the easy one

good_account | 1234

This is probably to do with password length. (note all examples are by
their nature ambiguous, this is why scenario outlines and examples are bad
in my book, they don't have information to be clear about their intention)

So I'd extract a scenario

Scenario: Short password
When I register with a short password
Then I should see my password is to short

The next one is a bit more confusing. Some possibilities are

Need a capital letter
Need a special character

So here I might have to check with the business, and then I'd rinse and
repeat

From this process we end up extracting and making concrete several
important concepts

- short passwords
- passwords with special chars
- passwords without capitals

etc.

Now it might seem a stretch but you can apply this concept to every
scenario outline, and to every example group or table of examples. And if
you do, not only will you get simpler features that are easier to implement
and work with, you'll also get most of the 'names' you need to write the
code to implement the particular behaviour you are specifiying (yay!)

HTH

Andrew
Post by George Dinwiddie
Post by Andrew Premdas
This ties in quite nicely with my idea/opinion that any Cucumber scenario
that has examples in it, particularly tables is in some way incomplete.
You complete the scenario by extracting the spirit/meaning of the
examples and in the process name things. When you have done this all the
tables of examples just disappear and you are left with named concepts
specific to your context.
All best
Andrew
Thanks for the clarification Gabriel.
In the past few years, after helping dozens of organisations and
hundreds of people adopt BDD I have come to the conclusion that it
is counter-productive to have 3 amigos (BAs, testers and
programmers) write executable specifications (Gherkin) together.
What I encourage instead is that they create *examples* together.
Examples are not executable specifications - they are simply
conversation topics, often jotted down on a card as a sketch or a
few words. I recommend Example Mapping
(https://cucumber.io/blog/2015/12/08/example-mapping-introduction
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction>)
as a technique to create examples together.
When conversations happen around lo-fi examples like this in an
example mapping session, stakeholders tend to uncover a lot of
misunderstandings and gain new insight. In my experience, this does
not happen when the conversations happen around executable
specifications (Gherkin). Instead, conversations tend to revolve
around grammar and formulation, diverting the attention from the
problem domain and the examples themselves. Writing specifications
together is also slow and tedious, and people don't enjoy these
sessions as much. When it's not enjoyable, people stop doing it
pretty quickly. Example Mapping on the other hand is energising and
fun.
BDD is a variant of TDD, where programmers write code guided by
failing tests. In my experience, this workflow does not work well
when the specifications were written collaboratively. When the
programmers starts the red-green-refactor cycle with collaboratively
written scenarios, they quickly realise they need to reformulate
those scenarios, for a number of reasons. They're incomplete,
they're ambiguous etc. When scenarios start passing, the scenarios
are often very different from what the diverse group created
together. This often causes frustration: "You've completely
rewritten it - this is not what we wrote together - I don't
understand this".
So instead I recommend programmers write the executable
specifications, perhaps with assistance from a tester. They use the
example map as input when they do this, and do their best to capture
the spirit of the examples and the conversations. When they're done
(or better - while they're doing it), they show the scenarios to the
BAs/POs and ask for feedback: Do you understand this? Does it
reflect what we discussed during example mapping? Then they tweak
Specification, Test and Documentation.
BDD isn't a process where you can write the specification up-front -
even when it happens collaboratively. They take shape as you go, in
parallel with development. The only thing you can do up-front
(before coding) is to reduce the misunderstandings, but executable
specifications isn't a good medium for that I think.
Cheers,
Aslak
On Thu, Sep 14, 2017 at 7:18 AM, Gabriel Oliveira
Hi Aslak,
Thanks for pointing this out - this was a mistake on our part.
The intent was to state that fact that bad requirements are one
of the many causes of project failures.
Already fixed on bit.ly/bdd_quality <http://bit.ly/bdd_quality>
On Wed, Sep 13, 2017 at 11:03 PM, Aslak HellesÞy
Hi Gabriel,
"Since it is well known that executable solution
requirements are one of many potential causes of a project
failure, we come to ask some questions about this subject."
1) Are you saying that project failure can be partly caused
by the fact that specifications are executable?
2) After 12 years in this field I've never heard this - on
what basis are you claiming this is well known?
Cheers,
Aslak
On Thursday, September 7, 2017 at 2:17:45 PM UTC+1, Gabriel
Hi folks,
Have you been practicing BDD (Behavior-Driven
Development) lately and believe scenarios' quality matter?
I'm writing my graduate thesis on that subject and would
like to invite you all to answer a (really) small 2
https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>
If you're willing to spend a little more time to talk to
me about that topic, please remember to fill out your
contact information there - or comment on this thread
here - and I will promptly get in touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
/https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>/
The Cucumber logo is the intellectual property of Cucumber
Ltd, a limited company registered in Scotland, number 456793.
UK Headquarters: Cucumber Ltd, Drumsyniebeg, Lochgoilhead,
Cairndow, Argyll, PA24 8AN UK.
CONFIDENTIALITY NOTICE: The information in this e-mail is
confidential and privileged; it is intended for use solely
by the individual or entity named as the recipient hereof.
Disclosure, copying, distribution, or use of the contents of
this e-mail by persons other than the intended recipient is
strictly prohibited and may violate applicable laws. If you
have received this e-mail in error, please delete the
original message and notify us by email immediately. Thank
you. Cucumber Ltd.
-- Posting rules: http://cukes.info/posting-rule
s.html
<http://cukes.info/posting-rules.html>
---
You received this message because you are subscribed to the
Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
-- Posting rules: http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
---
You received this message because you are subscribed to the
Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
-- Posting rules: http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
---
You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it,
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org <http://blog.andrew.premdas.org>
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups
"Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
--- You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
George Dinwiddie
2017-09-14 17:37:11 UTC
Permalink
Raw Message
Nice example, Andrew. I have found that when using tables for password
rules I needed a <reason> column to explain which rule was being tested.
That's a sure sign to me that a different scenario is beneficial.

I was at a client recently and they were delighted that they could use
Scenario Outlines rather than repeating the scenario. The issue had to
do with different legal requirements in different states. I don't
remember the details, so these made up examples will have to suffice.

Scenario Outline: State requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then there is no option to remove the Secondary Named Insured

Examples:
| state |
| AL |
| AK |

Scenario Outline: No state requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then the Agent can remove the Secondary Named Insured

Examples:
| state |
| SC |
| SD |

As I recall, there were other state-oriented scenarios with value
limits, but I can't recall them well enough to generate examples.

I can certainly see how the above could be written as individual
scenarios, but it was both more convenient and less error prone to
implement it with tables.

- George
Post by George Dinwiddie
Andrew,
I'm not exactly sure what you mean by this. Can you give an example
of extracting the spirit/meaning of the examples and naming them?
 - George
Sure, lets take something everyone knows about registrations (excuse bad
syntax I don't use outlines examples or tables anymore).
Scenario Outline: Bad registrations
  When I register with <account> <password>
  Then I should not be register
   Account | Password
   bad_account | good_password
   good_account | 1234
   good_account |  sdfdsflsdfsdf3313
   good_account | wieikjkeiiow
OK so our business peeps have come up with some examples of
account|password combinations that should fail. Now we have to take each
pair and try and extract the spirit/meaning. I'll start with the easy one
good_account | 1234
This is probably to do with password length. (note all examples are by
their nature ambiguous, this is why scenario outlines and examples are
bad in my book, they don't have information to be clear about their
intention)
So I'd extract a scenario
Scenario: Short password
  When I register with a short password
  Then I should see my password is to short
The next one is a bit more confusing. Some possibilities are
Need a capital letter
Need a special character
So here I might have to check with the business, and then I'd rinse and
repeat
From this process we end up extracting and making concrete several
important concepts
- short passwords
- passwords with special chars
- passwords without capitals
etc.
Now it might seem a stretch but you can apply this concept to every
scenario outline, and to every example group or table of examples. And
if you do, not only will you get simpler features that are easier to
implement and work with, you'll also get most of the 'names' you need to
write the code to implement the particular behaviour you are specifiying
(yay!)
HTH
Andrew
This ties in quite nicely with my idea/opinion that any Cucumber
scenario that has examples in it, particularly tables is in some
way incomplete.
You complete the scenario by extracting the spirit/meaning of
the examples and in the process name things. When you have done
this all the tables of examples just disappear and you are left
with named concepts specific to your context.
All best
Andrew
On 14 September 2017 at 10:32, aslak hellesoy
    Thanks for the clarification Gabriel.
    In the past few years, after helping dozens of
organisations and
    hundreds of people adopt BDD I have come to the conclusion
that it
    is counter-productive to have 3 amigos (BAs, testers and
    programmers) write executable specifications (Gherkin)
together.
    What I encourage instead is that they create *examples*
together.
    Examples are not executable specifications - they are simply
    conversation topics, often jotted down on a card as a
sketch or a
    few words. I recommend Example Mapping
(https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction>
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction>>)
    as a technique to create examples together.
    When conversations happen around lo-fi examples like this in an
    example mapping session, stakeholders tend to uncover a lot of
    misunderstandings and gain new insight. In my experience,
this does
    not happen when the conversations happen around executable
    specifications (Gherkin). Instead, conversations tend to
revolve
    around grammar and formulation, diverting the attention
from the
    problem domain and the examples themselves. Writing
specifications
    together is also slow and tedious, and people don't enjoy these
    sessions as much. When it's not enjoyable, people stop doing it
    pretty quickly. Example Mapping on the other hand is
energising and fun.
    BDD is a variant of TDD, where programmers write code guided by
    failing tests. In my experience, this workflow does not
work well
    when the specifications were written collaboratively. When the
    programmers starts the red-green-refactor cycle with
collaboratively
    written scenarios, they quickly realise they need to
reformulate
    those scenarios, for a number of reasons. They're incomplete,
    they're ambiguous etc. When scenarios start passing, the
scenarios
    are often very different from what the diverse group created
    together. This often causes frustration: "You've completely
    rewritten it - this is not what we wrote together - I don't
    understand this".
    So instead I recommend programmers write the executable
    specifications, perhaps with assistance from a tester. They
use the
    example map as input when they do this, and do their best
to capture
    the spirit of the examples and the conversations. When
they're done
    (or better - while they're doing it), they show the
scenarios to the
    BAs/POs and ask for feedback: Do you understand this? Does it
    reflect what we discussed during example mapping? Then they
tweak
    Specification, Test and Documentation.
    BDD isn't a process where you can write the specification
up-front -
    even when it happens collaboratively. They take shape as
you go, in
    parallel with development. The only thing you can do up-front
    (before coding) is to reduce the misunderstandings, but
executable
    specifications isn't a good medium for that I think.
    Cheers,
    Aslak
    On Thu, Sep 14, 2017 at 7:18 AM, Gabriel Oliveira
        Hi Aslak,
        Thanks for pointing this out - this was a mistake on
our part.
        The intent was to state that fact that bad requirements
are one
        of the many causes of project failures.
        Already fixed on bit.ly/bdd_quality
<http://bit.ly/bdd_quality> <http://bit.ly/bdd_quality>
        On Wed, Sep 13, 2017 at 11:03 PM, Aslak Hellesøy
            Hi Gabriel,
            "Since it is well known that executable solution
            requirements are one of many potential causes of a
project
            failure, we come to ask some questions about this
subject."
            1) Are you saying that project failure can be
partly caused
            by the fact that specifications are executable?
            2) After 12 years in this field I've never heard
this - on
            what basis are you claiming this is well known?
            Cheers,
            Aslak
            On Thursday, September 7, 2017 at 2:17:45 PM UTC+1,
Gabriel
                Hi folks,
                Have you been practicing BDD (Behavior-Driven
                Development) lately and believe scenarios'
quality matter?
                I'm writing my graduate thesis on that subject
and would
                like to invite you all to answer a (really) small 2
https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>
                <https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>>
                If you're willing to spend a little more time
to talk to
                me about that topic, please remember to fill
out your
                contact information there - or comment on this
thread
                here - and I will promptly get in touch :) !
                Best regards,
                --
                *Gabriel P.A. de Oliveira*
                /https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>
                <https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>>/
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Chuck van der Linden
2017-09-15 16:37:29 UTC
Permalink
Raw Message
On Thursday, September 14, 2017 at 10:37:18 AM UTC-7, George Dinwiddie
Post by George Dinwiddie
Nice example, Andrew. I have found that when using tables for password
rules I needed a <reason> column to explain which rule was being tested.
That's a sure sign to me that a different scenario is beneficial.
I was at a client recently and they were delighted that they could use
Scenario Outlines rather than repeating the scenario. The issue had to
do with different legal requirements in different states. I don't
remember the details, so these made up examples will have to suffice.
Scenario Outline: State requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then there is no option to remove the Secondary Named Insured
| state |
| AL |
| AK |
Scenario Outline: No state requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then the Agent can remove the Secondary Named Insured
| state |
| SC |
| SD |
As I recall, there were other state-oriented scenarios with value
limits, but I can't recall them well enough to generate examples.
I can certainly see how the above could be written as individual
scenarios, but it was both more convenient and less error prone to
implement it with tables.
- George
Rather than an special 'reason' column this is a situation where I would
suggest you make use of the fact that scenario outlines can accept more
than a single set of examples, and that the Examples: keyword can have
other text after it. e,g

Scenario Outline: Users are not allowed to use weak or 'bad' passwords when
changing their password

steps..

Examples: Password contains username
example table

Examples: Password too short
example table

Examples: Password lacks complexity
example table

Examples: reusing prior or recent password
example table
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Premdas
2017-09-16 15:31:44 UTC
Permalink
Raw Message
I find it quite interesting to expand scenarios like this and then look
into refactoring them with the aim to uncover business concepts rather than
make my scenarios concise.

In this example it seems that there is a simple concept

in some states you can remove the secondary named insured
in other states you can't

What you examples here are doing is wasting large amounts of run time to
repeat something that does not need repeating.

As far as the UI is concerned we only need 2 scenarios

1. A scenario that shows there is an option to remove the secondary named
insured
2. A scenario that shows that option isn't present

As for testing whether a particular state is a removing state or not, you
can do that in a unit test that runs 100 times faster.

Now you only have two states in your examples, but it would be very easy to
add more and if you apply that across multiple scenarios in a large
application you can see why people end up with cukes that have huge run
times.

For a better example of taking apart a more complex scenario have a look at
an ancient blog post of mine
http://pages.andrew.premdas.org/2011/06/27/composable-features-and-tables.html

All best
Post by George Dinwiddie
Nice example, Andrew. I have found that when using tables for password
rules I needed a <reason> column to explain which rule was being tested.
That's a sure sign to me that a different scenario is beneficial.
I was at a client recently and they were delighted that they could use
Scenario Outlines rather than repeating the scenario. The issue had to do
with different legal requirements in different states. I don't remember the
details, so these made up examples will have to suffice.
Scenario Outline: State requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then there is no option to remove the Secondary Named Insured
| state |
| AL |
| AK |
Scenario Outline: No state requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then the Agent can remove the Secondary Named Insured
| state |
| SC |
| SD |
As I recall, there were other state-oriented scenarios with value limits,
but I can't recall them well enough to generate examples.
I can certainly see how the above could be written as individual
scenarios, but it was both more convenient and less error prone to
implement it with tables.
- George
Post by George Dinwiddie
Andrew,
I'm not exactly sure what you mean by this. Can you give an example
of extracting the spirit/meaning of the examples and naming them?
- George
Sure, lets take something everyone knows about registrations (excuse bad
syntax I don't use outlines examples or tables anymore).
Scenario Outline: Bad registrations
When I register with <account> <password>
Then I should not be register
Account | Password
bad_account | good_password
good_account | 1234
good_account | sdfdsflsdfsdf3313
good_account | wieikjkeiiow
OK so our business peeps have come up with some examples of
account|password combinations that should fail. Now we have to take each
pair and try and extract the spirit/meaning. I'll start with the easy one
good_account | 1234
This is probably to do with password length. (note all examples are by
their nature ambiguous, this is why scenario outlines and examples are bad
in my book, they don't have information to be clear about their intention)
So I'd extract a scenario
Scenario: Short password
When I register with a short password
Then I should see my password is to short
The next one is a bit more confusing. Some possibilities are
Need a capital letter
Need a special character
So here I might have to check with the business, and then I'd rinse and
repeat
From this process we end up extracting and making concrete several
important concepts
- short passwords
- passwords with special chars
- passwords without capitals
etc.
Now it might seem a stretch but you can apply this concept to every
scenario outline, and to every example group or table of examples. And if
you do, not only will you get simpler features that are easier to implement
and work with, you'll also get most of the 'names' you need to write the
code to implement the particular behaviour you are specifiying (yay!)
HTH
Andrew
This ties in quite nicely with my idea/opinion that any Cucumber
scenario that has examples in it, particularly tables is in some
way incomplete.
You complete the scenario by extracting the spirit/meaning of
the examples and in the process name things. When you have done
this all the tables of examples just disappear and you are left
with named concepts specific to your context.
All best
Andrew
On 14 September 2017 at 10:32, aslak hellesoy
Thanks for the clarification Gabriel.
In the past few years, after helping dozens of
organisations and
hundreds of people adopt BDD I have come to the conclusion
that it
is counter-productive to have 3 amigos (BAs, testers and
programmers) write executable specifications (Gherkin)
together.
What I encourage instead is that they create *examples*
together.
Examples are not executable specifications - they are simply
conversation topics, often jotted down on a card as a
sketch or a
few words. I recommend Example Mapping
(https://cucumber.io/blog/2015
/12/08/example-mapping-introduction <https://cucumber.io/blog/2015
/12/08/example-mapping-introduction>
<https://cucumber.io/blog/2015
/12/08/example-mapping-introduction <https://cucumber.io/blog/2015
/12/08/example-mapping-introduction>>)
as a technique to create examples together.
When conversations happen around lo-fi examples like this in
an
example mapping session, stakeholders tend to uncover a lot
of
misunderstandings and gain new insight. In my experience,
this does
not happen when the conversations happen around executable
specifications (Gherkin). Instead, conversations tend to
revolve
around grammar and formulation, diverting the attention
from the
problem domain and the examples themselves. Writing
specifications
together is also slow and tedious, and people don't enjoy
these
sessions as much. When it's not enjoyable, people stop doing
it
pretty quickly. Example Mapping on the other hand is
energising and fun.
BDD is a variant of TDD, where programmers write code guided
by
failing tests. In my experience, this workflow does not
work well
when the specifications were written collaboratively. When
the
programmers starts the red-green-refactor cycle with
collaboratively
written scenarios, they quickly realise they need to
reformulate
those scenarios, for a number of reasons. They're incomplete,
they're ambiguous etc. When scenarios start passing, the
scenarios
are often very different from what the diverse group created
together. This often causes frustration: "You've completely
rewritten it - this is not what we wrote together - I don't
understand this".
So instead I recommend programmers write the executable
specifications, perhaps with assistance from a tester. They
use the
example map as input when they do this, and do their best
to capture
the spirit of the examples and the conversations. When
they're done
(or better - while they're doing it), they show the
scenarios to the
BAs/POs and ask for feedback: Do you understand this? Does it
reflect what we discussed during example mapping? Then they
tweak
Specification, Test and Documentation.
BDD isn't a process where you can write the specification
up-front -
even when it happens collaboratively. They take shape as
you go, in
parallel with development. The only thing you can do up-front
(before coding) is to reduce the misunderstandings, but
executable
specifications isn't a good medium for that I think.
Cheers,
Aslak
On Thu, Sep 14, 2017 at 7:18 AM, Gabriel Oliveira
Hi Aslak,
Thanks for pointing this out - this was a mistake on
our part.
The intent was to state that fact that bad requirements
are one
of the many causes of project failures.
Already fixed on bit.ly/bdd_quality
<http://bit.ly/bdd_quality> <http://bit.ly/bdd_quality>
On Wed, Sep 13, 2017 at 11:03 PM, Aslak HellesÞy
Hi Gabriel,
"Since it is well known that executable solution
requirements are one of many potential causes of a
project
failure, we come to ask some questions about this
subject."
1) Are you saying that project failure can be
partly caused
by the fact that specifications are executable?
2) After 12 years in this field I've never heard
this - on
what basis are you claiming this is well known?
Cheers,
Aslak
On Thursday, September 7, 2017 at 2:17:45 PM UTC+1,
Gabriel
Hi folks,
Have you been practicing BDD (Behavior-Driven
Development) lately and believe scenarios'
quality matter?
I'm writing my graduate thesis on that subject
and would
like to invite you all to answer a (really)
small 2
https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>
<https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>>
If you're willing to spend a little more time
to talk to
me about that topic, please remember to fill
out your
contact information there - or comment on this
thread
here - and I will promptly get in touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
/https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>
<https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>>/
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
--- You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
George Dinwiddie
2017-09-17 16:38:28 UTC
Permalink
Raw Message
Andrew,
Post by Andrew Premdas
I find it quite interesting to expand scenarios like this and then look
into refactoring them with the aim to uncover business concepts rather
than make my scenarios concise.
In this example it seems that there is a simple concept
in some states you can remove the secondary named insured
in other states you can't
What you examples here are doing is wasting large amounts of run time to
repeat something that does not need repeating.
As far as the UI is concerned we only need 2 scenarios
1. A scenario that shows there is an option to remove the secondary
named insured
2. A scenario that shows that option isn't present
Were we only concerned with machines, I would agree with you. But we're
also concerned with people. And some of those people are focused on
business risk rather than technical risk. If the scenarios do not make
plain the things those people care about, then the scenarios are not
doing their job. In this example, the concern is less about making the
scenarios concise than it is about making the example clearly express
the actual needs, as perceived by the Three Amigos and the
constituencies they represent.
Post by Andrew Premdas
As for testing whether a particular state is a removing state or not,
you can do that in a unit test that runs 100 times faster. >
Now you only have two states in your examples, but it would be very easy
to add more and if you apply that across multiple scenarios in a large
application you can see why people end up with cukes that have huge run
times.
And those unit tests might as well be greek to the business people. They
would be back to testing all of these things by hand.

Scenarios don't have to be tied to the UI, and don't have to run slowly.
Given the nature of this particular project (a portal that integrates a
number of back-end systems) I suspect they will be, for the most part,
but I've encouraged them to organize their javascript so that some can
be tested without deploying to a server.
Post by Andrew Premdas
For a better example of taking apart a more complex scenario have a look
at an ancient blog post of mine
http://pages.andrew.premdas.org/2011/06/27/composable-features-and-tables.html
In the original formulation of that, I notice that this feature is
checking two distinct and non-interacting sets of rules. One is the
rules about taxes, and one is the rules about shipping costs. I would
divide the feature file into separate features for each of these sets of
rules before doing anything else.

cheers,
George
Post by Andrew Premdas
All best
Nice example, Andrew. I have found that when using tables for
password rules I needed a <reason> column to explain which rule was
being tested. That's a sure sign to me that a different scenario is
beneficial.
I was at a client recently and they were delighted that they could
use Scenario Outlines rather than repeating the scenario. The issue
had to do with different legal requirements in different states. I
don't remember the details, so these made up examples will have to
suffice.
Scenario Outline: State requirements for Spouse as SNI
  Given the Primary Named Insured resides in <state>
  And the Secondary Named Insured has a relationship of "Spouse"
  When the Agent views the policy
  Then there is no option to remove the Secondary Named Insured
    | state |
    | AL    |
    | AK    |
Scenario Outline: No state requirements for Spouse as SNI
  Given the Primary Named Insured resides in <state>
  And the Secondary Named Insured has a relationship of "Spouse"
  When the Agent views the policy
  Then the Agent can remove the Secondary Named Insured
    | state |
    | SC    |
    | SD    |
As I recall, there were other state-oriented scenarios with value
limits, but I can't recall them well enough to generate examples.
I can certainly see how the above could be written as individual
scenarios, but it was both more convenient and less error prone to
implement it with tables.
 - George
On 14 September 2017 at 16:57, George Dinwiddie
    Andrew,
    I'm not exactly sure what you mean by this. Can you give an
example
    of extracting the spirit/meaning of the examples and naming
them?
      - George
Sure, lets take something everyone knows about registrations
(excuse bad syntax I don't use outlines examples or tables anymore).
Scenario Outline: Bad registrations
   When I register with <account> <password>
   Then I should not be register
    Account | Password
    bad_account | good_password
    good_account | 1234
    good_account |  sdfdsflsdfsdf3313
    good_account | wieikjkeiiow
OK so our business peeps have come up with some examples of
account|password combinations that should fail. Now we have to
take each pair and try and extract the spirit/meaning. I'll
start with the easy one
good_account | 1234
This is probably to do with password length. (note all examples
are by their nature ambiguous, this is why scenario outlines and
examples are bad in my book, they don't have information to be
clear about their intention)
So I'd extract a scenario
Scenario: Short password
   When I register with a short password
   Then I should see my password is to short
The next one is a bit more confusing. Some possibilities are
Need a capital letter
Need a special character
So here I might have to check with the business, and then I'd
rinse and repeat
 From this process we end up extracting and making concrete
several important concepts
- short passwords
- passwords with special chars
- passwords without capitals
etc.
Now it might seem a stretch but you can apply this concept to
every scenario outline, and to every example group or table of
examples. And if you do, not only will you get simpler features
that are easier to implement and work with, you'll also get most
of the 'names' you need to write the code to implement the
particular behaviour you are specifiying (yay!)
HTH
Andrew
        This ties in quite nicely with my idea/opinion that any
Cucumber
        scenario that has examples in it, particularly tables
is in some
        way incomplete.
        You complete the scenario by extracting the
spirit/meaning of
        the examples and in the process name things. When you
have done
        this all the tables of examples just disappear and you
are left
        with named concepts specific to your context.
        All best
        Andrew
        On 14 September 2017 at 10:32, aslak hellesoy
             Thanks for the clarification Gabriel.
             In the past few years, after helping dozens of
        organisations and
             hundreds of people adopt BDD I have come to the
conclusion
        that it
             is counter-productive to have 3 amigos (BAs,
testers and
             programmers) write executable specifications (Gherkin)
        together.
             What I encourage instead is that they create
*examples*
        together.
             Examples are not executable specifications - they
are simply
             conversation topics, often jotted down on a card as a
        sketch or a
             few words. I recommend Example Mapping
(https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction>
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction>>
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction>
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction>>>)
             as a technique to create examples together.
             When conversations happen around lo-fi examples
like this in an
             example mapping session, stakeholders tend to
uncover a lot of
             misunderstandings and gain new insight. In my
experience,
        this does
             not happen when the conversations happen around
executable
             specifications (Gherkin). Instead, conversations
tend to
        revolve
             around grammar and formulation, diverting the
attention
        from the
             problem domain and the examples themselves. Writing
        specifications
             together is also slow and tedious, and people
don't enjoy these
             sessions as much. When it's not enjoyable, people
stop doing it
             pretty quickly. Example Mapping on the other hand is
        energising and fun.
             BDD is a variant of TDD, where programmers write
code guided by
             failing tests. In my experience, this workflow
does not
        work well
             when the specifications were written
collaboratively. When the
             programmers starts the red-green-refactor cycle with
        collaboratively
             written scenarios, they quickly realise they need to
        reformulate
             those scenarios, for a number of reasons. They're
incomplete,
             they're ambiguous etc. When scenarios start
passing, the
        scenarios
             are often very different from what the diverse
group created
             together. This often causes frustration: "You've
completely
             rewritten it - this is not what we wrote together
- I don't
             understand this".
             So instead I recommend programmers write the
executable
             specifications, perhaps with assistance from a
tester. They
        use the
             example map as input when they do this, and do
their best
        to capture
             the spirit of the examples and the conversations. When
        they're done
             (or better - while they're doing it), they show the
        scenarios to the
             BAs/POs and ask for feedback: Do you understand
this? Does it
             reflect what we discussed during example mapping?
Then they
        tweak
             it, trying to strike a good balance between 3
             Specification, Test and Documentation.
             BDD isn't a process where you can write the
specification
        up-front -
             even when it happens collaboratively. They take
shape as
        you go, in
             parallel with development. The only thing you can
do up-front
             (before coding) is to reduce the
misunderstandings, but
        executable
             specifications isn't a good medium for that I think.
             Cheers,
             Aslak
             On Thu, Sep 14, 2017 at 7:18 AM, Gabriel Oliveira
                 Hi Aslak,
                 Thanks for pointing this out - this was a
mistake on
        our part.
                 The intent was to state that fact that bad
requirements
        are one
                 of the many causes of project failures.
                 Already fixed on bit.ly/bdd_quality
<http://bit.ly/bdd_quality>
        <http://bit.ly/bdd_quality> <http://bit.ly/bdd_quality>
                 On Wed, Sep 13, 2017 at 11:03 PM, Aslak Hellesøy
                     Hi Gabriel,
                     "Since it is well known that executable
solution
                     requirements are one of many potential
causes of a
        project
                     failure, we come to ask some questions
about this
        subject."
                     1) Are you saying that project failure can be
        partly caused
                     by the fact that specifications are
executable?
                     2) After 12 years in this field I've never
heard
        this - on
                     what basis are you claiming this is well
known?
                     Cheers,
                     Aslak
                     On Thursday, September 7, 2017 at 2:17:45
PM UTC+1,
        Gabriel
                         Hi folks,
                         Have you been practicing BDD
(Behavior-Driven
                         Development) lately and believe scenarios'
        quality matter?
                         I'm writing my graduate thesis on that
subject
        and would
                         like to invite you all to answer a
(really) small 2
https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>
        <https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>>
<https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>
        <https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>>>
                         If you're willing to spend a little
more time
        to talk to
                         me about that topic, please remember
to fill
        out your
                         contact information there - or comment
on this
        thread
                         here - and I will promptly get in
touch :) !
                         Best regards,
                         --
                         *Gabriel P.A. de Oliveira*
                         /https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>
        <https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>>
                         <https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>
        <https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>>>/
--
 ----------------------------------------------------------------------
  * George Dinwiddie * http://blog.gdinwiddie.com
  Software Development http://www.idiacomputing.com
  Consultant and Coach http://www.agilemaryland.org
 ----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
--- You received this message because you are subscribed to the
Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it,
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org <http://blog.andrew.premdas.org>
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Premdas
2017-09-18 01:31:03 UTC
Permalink
Raw Message
Post by George Dinwiddie
Andrew,
Post by Andrew Premdas
I find it quite interesting to expand scenarios like this and then look
into refactoring them with the aim to uncover business concepts rather than
make my scenarios concise.
In this example it seems that there is a simple concept
in some states you can remove the secondary named insured
in other states you can't
What you examples here are doing is wasting large amounts of run time to
repeat something that does not need repeating.
As far as the UI is concerned we only need 2 scenarios
1. A scenario that shows there is an option to remove the secondary named
insured
2. A scenario that shows that option isn't present
Were we only concerned with machines, I would agree with you. But we're
also concerned with people. And some of those people are focused on
business risk rather than technical risk. If the scenarios do not make
plain the things those people care about, then the scenarios are not doing
their job. In this example, the concern is less about making the scenarios
concise than it is about making the example clearly express the actual
needs, as perceived by the Three Amigos and the constituencies they
represent.
I think you are trying to hard to defend the scenario here. Clearly the
scenario is about secondary named insured, not about which state employs
which particular rules. If you insist on munging the two things together
then I can't help you. Often business wants to do this. My personal style
is to challenge this and force the separation, but I understand in certain
business contexts this might not be the politically correct thing to do.
However I don't think that should colour this discussion.

What you seem to be saying is that when we discuss this scenario outside of
a business context we should accept and prefer a scenario which convolutes
two separate contexts over two separate tests that separate and clarify
these contexts.

I'm quite happy for you to say that you will live with the scenario that
deals with both contexts because educating your business users is to
challlenging. My whole point is that the stuff you get from business IS NOT
the canonical source it is a flawed expression that has to be examined,
questioned and translated to become the code that tests your application.
Post by George Dinwiddie
Post by Andrew Premdas
As for testing whether a particular state is a removing state or not, you
can do that in a unit test that runs 100 times faster. >
Now you only have two states in your examples, but it would be very easy
to add more and if you apply that across multiple scenarios in a large
application you can see why people end up with cukes that have huge run
times.
And those unit tests might as well be greek to the business people. They
would be back to testing all of these things by hand.
I have no concern that my unit tests or my integration tests are greek to
business people. That is expected. The only thing my tests MAY have to do
is communicate to business people that their intentions are being met. If
business people are reading the output of Cucumber scenarios and using them
to determine whether their application functions or not then they are
idiots or completely misinformed. A cucumber scenario in itself going green
is completely meaningless, as every scenario will go green with no code.
Cucumber scenarios are just a way for business people to contribute to the
drive for development. Their executions proves nothing.

No matter how I implement my scenarios they and their output will never be
understood by business users in the same way that the code of my
application won't be understood by business users. My implemented scenarios
are code and their output is not for business users it is for the
developers who will have to build open the provided functionality. This
idea that you can use the output of cucumber scenarios to judge the health
of an application baffles me. If your a business person and you want to
judge the health of an application look at the application not a wall of
green text from cucumber scenarios.
Post by George Dinwiddie
Scenarios don't have to be tied to the UI, and don't have to run slowly.
Given the nature of this particular project (a portal that integrates a
number of back-end systems) I suspect they will be, for the most part, but
I've encouraged them to organize their javascript so that some can be
tested without deploying to a server.
Scenarios will always run slowly as they are by definition abstract
descriptions of the behaviour of an application described via its user
interface. The fact that I can run several hundred scenarios over my
application in 3 minutes in a single process on my laptop with a browser
doing the javascript does not effect this. I could run 10 times the number
of unit test in a few seconds IF I wrote them as well as I write my
scenarios. Wasting scenario runtime to test/exercise things that have
already been exercised is really detrimental to the overall effectiveness
of using Cucumber to drive development.

All best

Andrew
Post by George Dinwiddie
Post by Andrew Premdas
For a better example of taking apart a more complex scenario have a look
at an ancient blog post of mine http://pages.andrew.premdas.or
g/2011/06/27/composable-features-and-tables.html
In the original formulation of that, I notice that this feature is
checking two distinct and non-interacting sets of rules. One is the rules
about taxes, and one is the rules about shipping costs. I would divide the
feature file into separate features for each of these sets of rules before
doing anything else.
cheers,
George
Post by Andrew Premdas
All best
Nice example, Andrew. I have found that when using tables for
password rules I needed a <reason> column to explain which rule was
being tested. That's a sure sign to me that a different scenario is
beneficial.
I was at a client recently and they were delighted that they could
use Scenario Outlines rather than repeating the scenario. The issue
had to do with different legal requirements in different states. I
don't remember the details, so these made up examples will have to
suffice.
Scenario Outline: State requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then there is no option to remove the Secondary Named Insured
| state |
| AL |
| AK |
Scenario Outline: No state requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then the Agent can remove the Secondary Named Insured
| state |
| SC |
| SD |
As I recall, there were other state-oriented scenarios with value
limits, but I can't recall them well enough to generate examples.
I can certainly see how the above could be written as individual
scenarios, but it was both more convenient and less error prone to
implement it with tables.
- George
On 14 September 2017 at 16:57, George Dinwiddie
Andrew,
I'm not exactly sure what you mean by this. Can you give an example
of extracting the spirit/meaning of the examples and naming them?
- George
Sure, lets take something everyone knows about registrations
(excuse bad syntax I don't use outlines examples or tables anymore).
Scenario Outline: Bad registrations
When I register with <account> <password>
Then I should not be register
Account | Password
bad_account | good_password
good_account | 1234
good_account | sdfdsflsdfsdf3313
good_account | wieikjkeiiow
OK so our business peeps have come up with some examples of
account|password combinations that should fail. Now we have to
take each pair and try and extract the spirit/meaning. I'll
start with the easy one
good_account | 1234
This is probably to do with password length. (note all examples
are by their nature ambiguous, this is why scenario outlines and
examples are bad in my book, they don't have information to be
clear about their intention)
So I'd extract a scenario
Scenario: Short password
When I register with a short password
Then I should see my password is to short
The next one is a bit more confusing. Some possibilities are
Need a capital letter
Need a special character
So here I might have to check with the business, and then I'd
rinse and repeat
From this process we end up extracting and making concrete
several important concepts
- short passwords
- passwords with special chars
- passwords without capitals
etc.
Now it might seem a stretch but you can apply this concept to
every scenario outline, and to every example group or table of
examples. And if you do, not only will you get simpler features
that are easier to implement and work with, you'll also get most
of the 'names' you need to write the code to implement the
particular behaviour you are specifiying (yay!)
HTH
Andrew
This ties in quite nicely with my idea/opinion that any Cucumber
scenario that has examples in it, particularly tables is in some
way incomplete.
You complete the scenario by extracting the
spirit/meaning of
the examples and in the process name things. When you have done
this all the tables of examples just disappear and you are left
with named concepts specific to your context.
All best
Andrew
On 14 September 2017 at 10:32, aslak hellesoy
Thanks for the clarification Gabriel.
In the past few years, after helping dozens of
organisations and
hundreds of people adopt BDD I have come to the conclusion
that it
is counter-productive to have 3 amigos (BAs, testers and
programmers) write executable specifications (Gherkin)
together.
What I encourage instead is that they create *examples*
together.
Examples are not executable specifications - they
are simply
conversation topics, often jotted down on a card as a
sketch or a
few words. I recommend Example Mapping
(https://cucumber.io/blog/2015
/12/08/example-mapping-introduction <https://cucumber.io/blog/2015
/12/08/example-mapping-introduction>
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction>>
<https://cucumber.io/blog/2015
/12/08/example-mapping-introduction <https://cucumber.io/blog/2015
/12/08/example-mapping-introduction>
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction>>>)
as a technique to create examples together.
When conversations happen around lo-fi examples
like this in an
example mapping session, stakeholders tend to
uncover a lot of
misunderstandings and gain new insight. In my experience,
this does
not happen when the conversations happen around executable
specifications (Gherkin). Instead, conversations tend to
revolve
around grammar and formulation, diverting the attention
from the
problem domain and the examples themselves. Writing
specifications
together is also slow and tedious, and people
don't enjoy these
sessions as much. When it's not enjoyable, people
stop doing it
pretty quickly. Example Mapping on the other hand is
energising and fun.
BDD is a variant of TDD, where programmers write
code guided by
failing tests. In my experience, this workflow does not
work well
when the specifications were written
collaboratively. When the
programmers starts the red-green-refactor cycle with
collaboratively
written scenarios, they quickly realise they need to
reformulate
those scenarios, for a number of reasons. They're
incomplete,
they're ambiguous etc. When scenarios start passing, the
scenarios
are often very different from what the diverse
group created
together. This often causes frustration: "You've completely
rewritten it - this is not what we wrote together - I don't
understand this".
So instead I recommend programmers write the executable
specifications, perhaps with assistance from a tester. They
use the
example map as input when they do this, and do their best
to capture
the spirit of the examples and the conversations. When
they're done
(or better - while they're doing it), they show the
scenarios to the
BAs/POs and ask for feedback: Do you understand
this? Does it
reflect what we discussed during example mapping? Then they
tweak
Specification, Test and Documentation.
BDD isn't a process where you can write the specification
up-front -
even when it happens collaboratively. They take shape as
you go, in
parallel with development. The only thing you can
do up-front
(before coding) is to reduce the
misunderstandings, but
executable
specifications isn't a good medium for that I think.
Cheers,
Aslak
On Thu, Sep 14, 2017 at 7:18 AM, Gabriel Oliveira
Hi Aslak,
Thanks for pointing this out - this was a mistake on
our part.
The intent was to state that fact that bad requirements
are one
of the many causes of project failures.
Already fixed on bit.ly/bdd_quality
<http://bit.ly/bdd_quality>
<http://bit.ly/bdd_quality> <http://bit.ly/bdd_quality>
On Wed, Sep 13, 2017 at 11:03 PM, Aslak HellesÞy
Hi Gabriel,
"Since it is well known that executable solution
requirements are one of many potential causes of a
project
failure, we come to ask some questions about this
subject."
1) Are you saying that project failure can be
partly caused
by the fact that specifications are executable?
2) After 12 years in this field I've never heard
this - on
what basis are you claiming this is well known?
Cheers,
Aslak
On Thursday, September 7, 2017 at 2:17:45 PM UTC+1,
Gabriel
Hi folks,
Have you been practicing BDD (Behavior-Driven
Development) lately and believe scenarios'
quality matter?
I'm writing my graduate thesis on that subject
and would
like to invite you all to answer a
(really) small 2
https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>
<https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>>
<https://goo.gl/forms/dheA8XK
0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>
<https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>>>
If you're willing to spend a little more time
to talk to
me about that topic, please remember to fill
out your
contact information there - or comment on this
thread
here - and I will promptly get in touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
/https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>
<https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>>
<https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>
<https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>>>/
-- -----------------------------------------------------------
-----------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------
------------
-- Posting rules: http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
--- You received this message because you are subscribed to the
Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it,
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org <http://blog.andrew.premdas.org>
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
--- You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Steve Tooke
2017-09-18 08:16:37 UTC
Permalink
Raw Message
As for testing whether a particular state is a removing state or not, you can do that in a unit test that runs 100 times faster. >
Now you only have two states in your examples, but it would be very easy to add more and if you apply that across multiple scenarios in a large application you can see why people end up with cukes that have huge run times.
And those unit tests might as well be greek to the business people. They would be back to testing all of these things by hand.
I have no concern that my unit tests or my integration tests are greek to business people. That is expected. The only thing my tests MAY have to do is communicate to business people that their intentions are being met. If business people are reading the output of Cucumber scenarios and using them to determine whether their application functions or not then they are idiots or completely misinformed. A cucumber scenario in itself going green is completely meaningless, as every scenario will go green with no code. Cucumber scenarios are just a way for business people to contribute to the drive for development. Their executions proves nothing.
No matter how I implement my scenarios they and their output will never be understood by business users in the same way that the code of my application won't be understood by business users. My implemented scenarios are code and their output is not for business users it is for the developers who will have to build open the provided functionality. This idea that you can use the output of cucumber scenarios to judge the health of an application baffles me. If your a business person and you want to judge the health of an application look at the application not a wall of green text from cucumber scenarios.
If the output of Cucumber is meaningless to the people who require the software why bother going to the effort of automating a Gherkin scenario? Why add that layer that translate the Gherkin to code? You say their value is to contribute to driving the development. Couldn’t you achieve that simply by using concrete examples in conversations and then translating them to a RSpec or JUnit, etc. Why are we writing in English (or other natural language) if these things aren’t meant to be read by people. If they are only meant for developers code is a less ambiguous to way to specify things.

Cucumber & Gherkin has 3 values:

1) Executable specifications based on concrete examples – this is how we are able to consider the business rules and discuss whether they meet our (users’) needs, and because we specify software before we write it they fit naturally into a TDD process
2) Automated tests – as software is delivered, executable specs form part of our regression suite. We learn quickly if new changes have effected earlier work, and we can decide if this is expected or not
3) Documentation - these executable specifications, written in (almost) natural language, can document what our software does. This is valuable to people on the team and people outside of the team

There’s always a balance that needs to be struck between these three values, and its down to each team to strike the right balance for themselves. I do think though that the hardest to balance are automated tests and documentation. Automated tests push us to brevity and the minimum required as we get concerned with run length. Documentation screams for concrete examples, for richness to interest a reader.
Scenarios don't have to be tied to the UI, and don't have to run slowly. Given the nature of this particular project (a portal that integrates a number of back-end systems) I suspect they will be, for the most part, but I've encouraged them to organize their javascript so that some can be tested without deploying to a server.
Scenarios will always run slowly as they are by definition abstract descriptions of the behaviour of an application described via its user interface. The fact that I can run several hundred scenarios over my application in 3 minutes in a single process on my laptop with a browser doing the javascript does not effect this. I could run 10 times the number of unit test in a few seconds IF I wrote them as well as I write my scenarios. Wasting scenario runtime to test/exercise things that have already been exercised is really detrimental to the overall effectiveness of using Cucumber to drive development.
I disagree with your definition that scenarios must run against a user interface. Scenarios are concerned with being business readable. Some of them will run end-to-end, but they don’t need to.

I wrote about this before: http://tooky.co.uk/cucumber-and-full-stack-testing/ <http://tooky.co.uk/cucumber-and-full-stack-testing/> as did Seb Rose: http://claysnow.co.uk/the-testing-iceberg/ <http://claysnow.co.uk/the-testing-iceberg/>.

cheers,
Steve
All best
Andrew
For a better example of taking apart a more complex scenario have a look at an ancient blog post of mine http://pages.andrew.premdas.org/2011/06/27/composable-features-and-tables.html <http://pages.andrew.premdas.org/2011/06/27/composable-features-and-tables.html>
In the original formulation of that, I notice that this feature is checking two distinct and non-interacting sets of rules. One is the rules about taxes, and one is the rules about shipping costs. I would divide the feature file into separate features for each of these sets of rules before doing anything else.
cheers,
George
All best
Nice example, Andrew. I have found that when using tables for
password rules I needed a <reason> column to explain which rule was
being tested. That's a sure sign to me that a different scenario is
beneficial.
I was at a client recently and they were delighted that they could
use Scenario Outlines rather than repeating the scenario. The issue
had to do with different legal requirements in different states. I
don't remember the details, so these made up examples will have to
suffice.
Scenario Outline: State requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then there is no option to remove the Secondary Named Insured
| state |
| AL |
| AK |
Scenario Outline: No state requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then the Agent can remove the Secondary Named Insured
| state |
| SC |
| SD |
As I recall, there were other state-oriented scenarios with value
limits, but I can't recall them well enough to generate examples.
I can certainly see how the above could be written as individual
scenarios, but it was both more convenient and less error prone to
implement it with tables.
- George
On 14 September 2017 at 16:57, George Dinwiddie
Andrew,
I'm not exactly sure what you mean by this. Can you give an example
of extracting the spirit/meaning of the examples and naming them?
- George
Sure, lets take something everyone knows about registrations
(excuse bad syntax I don't use outlines examples or tables anymore).
Scenario Outline: Bad registrations
When I register with <account> <password>
Then I should not be register
Account | Password
bad_account | good_password
good_account | 1234
good_account | sdfdsflsdfsdf3313
good_account | wieikjkeiiow
OK so our business peeps have come up with some examples of
account|password combinations that should fail. Now we have to
take each pair and try and extract the spirit/meaning. I'll
start with the easy one
good_account | 1234
This is probably to do with password length. (note all examples
are by their nature ambiguous, this is why scenario outlines and
examples are bad in my book, they don't have information to be
clear about their intention)
So I'd extract a scenario
Scenario: Short password
When I register with a short password
Then I should see my password is to short
The next one is a bit more confusing. Some possibilities are
Need a capital letter
Need a special character
So here I might have to check with the business, and then I'd
rinse and repeat
From this process we end up extracting and making concrete
several important concepts
- short passwords
- passwords with special chars
- passwords without capitals
etc.
Now it might seem a stretch but you can apply this concept to
every scenario outline, and to every example group or table of
examples. And if you do, not only will you get simpler features
that are easier to implement and work with, you'll also get most
of the 'names' you need to write the code to implement the
particular behaviour you are specifiying (yay!)
HTH
Andrew
This ties in quite nicely with my idea/opinion that any Cucumber
scenario that has examples in it, particularly tables is in some
way incomplete.
You complete the scenario by extracting the
spirit/meaning of
the examples and in the process name things. When you have done
this all the tables of examples just disappear and you are left
with named concepts specific to your context.
All best
Andrew
On 14 September 2017 at 10:32, aslak hellesoy
Thanks for the clarification Gabriel.
In the past few years, after helping dozens of
organisations and
hundreds of people adopt BDD I have come to the conclusion
that it
is counter-productive to have 3 amigos (BAs, testers and
programmers) write executable specifications (Gherkin)
together.
What I encourage instead is that they create *examples*
together.
Examples are not executable specifications - they are simply
conversation topics, often jotted down on a card as a
sketch or a
few words. I recommend Example Mapping
(https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction> <https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction>>
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction> <https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction>>>
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction> <https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction>>
<https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction> <https://cucumber.io/blog/2015/12/08/example-mapping-introduction <https://cucumber.io/blog/2015/12/08/example-mapping-introduction>>>>)
as a technique to create examples together.
When conversations happen around lo-fi examples
like this in an
example mapping session, stakeholders tend to
uncover a lot of
misunderstandings and gain new insight. In my experience,
this does
not happen when the conversations happen around executable
specifications (Gherkin). Instead, conversations tend to
revolve
around grammar and formulation, diverting the attention
from the
problem domain and the examples themselves. Writing
specifications
together is also slow and tedious, and people
don't enjoy these
sessions as much. When it's not enjoyable, people
stop doing it
pretty quickly. Example Mapping on the other hand is
energising and fun.
BDD is a variant of TDD, where programmers write
code guided by
failing tests. In my experience, this workflow does not
work well
when the specifications were written
collaboratively. When the
programmers starts the red-green-refactor cycle with
collaboratively
written scenarios, they quickly realise they need to
reformulate
those scenarios, for a number of reasons. They're incomplete,
they're ambiguous etc. When scenarios start passing, the
scenarios
are often very different from what the diverse group created
together. This often causes frustration: "You've completely
rewritten it - this is not what we wrote together - I don't
understand this".
So instead I recommend programmers write the executable
specifications, perhaps with assistance from a tester. They
use the
example map as input when they do this, and do their best
to capture
the spirit of the examples and the conversations. When
they're done
(or better - while they're doing it), they show the
scenarios to the
BAs/POs and ask for feedback: Do you understand this? Does it
reflect what we discussed during example mapping? Then they
tweak
Specification, Test and Documentation.
BDD isn't a process where you can write the specification
up-front -
even when it happens collaboratively. They take shape as
you go, in
parallel with development. The only thing you can do up-front
(before coding) is to reduce the
misunderstandings, but
executable
specifications isn't a good medium for that I think.
Cheers,
Aslak
On Thu, Sep 14, 2017 at 7:18 AM, Gabriel Oliveira
Hi Aslak,
Thanks for pointing this out - this was a mistake on
our part.
The intent was to state that fact that bad requirements
are one
of the many causes of project failures.
Already fixed on bit.ly/bdd_quality <http://bit.ly/bdd_quality>
<http://bit.ly/bdd_quality <http://bit.ly/bdd_quality>>
<http://bit.ly/bdd_quality <http://bit.ly/bdd_quality>> <http://bit.ly/bdd_quality <http://bit.ly/bdd_quality>>
On Wed, Sep 13, 2017 at 11:03 PM, Aslak HellesÞy
Hi Gabriel,
"Since it is well known that executable solution
requirements are one of many potential causes of a
project
failure, we come to ask some questions about this
subject."
1) Are you saying that project failure can be
partly caused
by the fact that specifications are executable?
2) After 12 years in this field I've never heard
this - on
what basis are you claiming this is well known?
Cheers,
Aslak
On Thursday, September 7, 2017 at 2:17:45 PM UTC+1,
Gabriel
Hi folks,
Have you been practicing BDD
(Behavior-Driven
Development) lately and believe scenarios'
quality matter?
I'm writing my graduate thesis on that subject
and would
like to invite you all to answer a
(really) small 2
https://goo.gl/forms/dheA8XK0asiNUVze2 <https://goo.gl/forms/dheA8XK0asiNUVze2>
<https://goo.gl/forms/dheA8XK0asiNUVze2 <https://goo.gl/forms/dheA8XK0asiNUVze2>>
<https://goo.gl/forms/dheA8XK0asiNUVze2 <https://goo.gl/forms/dheA8XK0asiNUVze2>
<https://goo.gl/forms/dheA8XK0asiNUVze2 <https://goo.gl/forms/dheA8XK0asiNUVze2>>>
<https://goo.gl/forms/dheA8XK0asiNUVze2 <https://goo.gl/forms/dheA8XK0asiNUVze2>
<https://goo.gl/forms/dheA8XK0asiNUVze2 <https://goo.gl/forms/dheA8XK0asiNUVze2>>
<https://goo.gl/forms/dheA8XK0asiNUVze2 <https://goo.gl/forms/dheA8XK0asiNUVze2>
<https://goo.gl/forms/dheA8XK0asiNUVze2 <https://goo.gl/forms/dheA8XK0asiNUVze2>>>>
If you're willing to spend a little more time
to talk to
me about that topic, please remember to fill
out your
contact information there - or comment on this
thread
here - and I will promptly get in touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
/https://gpaoliveirablog.wordpress.com <https://gpaoliveirablog.wordpress.com/>
<https://gpaoliveirablog.wordpress.com <https://gpaoliveirablog.wordpress.com/>>
<https://gpaoliveirablog.wordpress.com <https://gpaoliveirablog.wordpress.com/>
<https://gpaoliveirablog.wordpress.com <https://gpaoliveirablog.wordpress.com/>>>
<https://gpaoliveirablog.wordpress.com <https://gpaoliveirablog.wordpress.com/>
<https://gpaoliveirablog.wordpress.com <https://gpaoliveirablog.wordpress.com/>>
<https://gpaoliveirablog.wordpress.com <https://gpaoliveirablog.wordpress.com/>
<https://gpaoliveirablog.wordpress.com <https://gpaoliveirablog.wordpress.com/>>>>/
-- ----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com <http://blog.gdinwiddie.com/>
Software Development http://www.idiacomputing.com <http://www.idiacomputing.com/>
Consultant and Coach http://www.agilemaryland.org <http://www.agilemaryland.org/>
----------------------------------------------------------------------
-- Posting rules: http://cukes.info/posting-rules.html <http://cukes.info/posting-rules.html>
<http://cukes.info/posting-rules.html <http://cukes.info/posting-rules.html>>
--- You received this message because you are subscribed to the
Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it,
For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>
<https://groups.google.com/d/optout <https://groups.google.com/d/optout>>.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org <http://blog.andrew.premdas.org/> <http://blog.andrew.premdas.org <http://blog.andrew.premdas.org/>>
--
Posting rules: http://cukes.info/posting-rules.html <http://cukes.info/posting-rules.html>
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com <http://blog.gdinwiddie.com/>
Software Development http://www.idiacomputing.com <http://www.idiacomputing.com/>
Consultant and Coach http://www.agilemaryland.org <http://www.agilemaryland.org/>
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html <http://cukes.info/posting-rules.html>
--- You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org <http://blog.andrew.premdas.org/>
--
Posting rules: http://cukes.info/posting-rules.html <http://cukes.info/posting-rules.html>
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Premdas
2017-09-16 15:48:08 UTC
Permalink
Raw Message
Post by Chuck van der Linden
Rather than an special 'reason' column this is a situation where I would
suggest you make use of the fact that scenario outlines can accept more
than a single set of examples, and that the Examples: keyword can have
other text after it. e,g
Scenario Outline: Users are not allowed to use weak or 'bad' passwords
when changing their password
steps..
Examples: Password contains username
example table
Examples: Password too short
example table
Examples: Password lacks complexity
example table
Examples: reusing prior or recent password
example table
--
This has two flaws from my perspective

1. Its harder to read and debug.

2. It still keeps the example data in the feature file. This is the really
important point.

I much prefer to extract examples out of scenarios getting them from either
step definition helpers or the application itself (if possible). The thing
with example data in scenarios is that when the rules change the scenario
has to change. Lets take a short password as an example.

Lets say we write

Scenario Outline:
...

Example: short password
email | password
... | 1234567890

and we have a password rule that says passwords must be greater than 10
chars. Now when the business rule changes to passwords can be 8 chars
(because the users are complaining) then we have to modify the sample data.

Consider the alternative

Scenario:
When I login with a short password
...

Now in our step def we can have

When "I login with a short password" do
login as: @i, password: short_password
end

and somewhere we can have

def short_password
# here we go to the application, and get the minimum password length
# and then return a shorter password
end

Now when the rule changes we don't have to modify any of our features and
scenarios, the cost of changing this business rule has been dramatically
reduced.

Again you can extend this idea to every piece of example data in a
scenario.


Example data in implemented scenario == development debt (there is still
work to do)

All best

Andrew
Post by Chuck van der Linden
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups
"Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
George Dinwiddie
2017-09-17 16:03:31 UTC
Permalink
Raw Message
Andrew,

It's possible to hide too much, and lose the advantages of concrete
examples.
Rather than an special 'reason' column this is a situation where  I
would suggest you make use of the fact that scenario outlines can
keyword can have other text after it.  e,g
<snip>
This has two flaws from my perspective
1. Its harder to read and debug.
2.  It still keeps the example data in the feature file. This is the
really important point.
I much prefer to extract examples out of scenarios getting them from
either step definition helpers or the application itself (if possible).
The thing with example data in scenarios is that when the rules change
the scenario has to change. Lets take a short password as an example.
Lets say we write
   ...
   Example: short password
      email | password
      ...       |  1234567890
and we have a password rule that says passwords must be greater than 10
chars. Now when the business rule changes to passwords can be 8 chars
(because the users are complaining) then we have to modify the sample data.
When the rule changes, it seems reasonable to modify the examples that
illustrate the rule.
Consider the alternative
   When I login with a short password
   ...
Now in our step def we can have
When "I login with a short password" do
end
and somewhere we can have
def short_password
  # here we go to the application, and get the minimum password length
  # and then return a shorter password
end
This is too far, in my opinion. With this, the test now depends on the
application for the threshold to check. If the application reports this
wrong, the test is wrong. We're back to a single point of failure.

Back in the early 2000s, there was a bookkeeper learning programming who
showed up on the XP list for awhile. It often seemed that he had only
one lens for viewing the world, and that was double-entry bookkeeping.
Even though he was often dismissed for sounding like a Johnny One-Note
on the list, I came to view this as a profound insight about the nature
of test-driven development. The tests might be wrong, but as long as
they are an INDEPENDENT and different representation of what's desired,
they are likely to catch errors. The error might be in the test or
application, and a human can figure that out.

Once we start using the application to tell us what the answer should
be, or even calculate the desired answer using the same algorithm, we've
lost that quality.
Now when the rule changes we don't have to modify any of our features
and scenarios, the cost of changing this business rule has been
dramatically reduced.
Again you can extend this idea to every piece of example data in a
scenario.
I've seen this taken too far. In a recent discussion at a client, they
were going round and round on developing a particular scenario. They
were developing something like this:

Scenario: Driver One information must match Customer Application
Given the Customer Application has a different field values than
Prefill data
When the user lands on the driver screen
Then Driver One information displays the field values from the
Customer Application

People kept getting confused about what it meant. Essentially, the
scenario is restating the rule rather than provide an example.

I listened to this for awhile, and suggested they listen to the language
they were using to explain it to each other, and write the scenario
using that language. They ended up with something like this:

Scenario: When prefilling applicant data, data on the application
takes precedence
Given the Customer Application has an Occupation of "Lawyer"
And the Prefill returns an occupation of "Musician"
When the user lands on the driver screen
Then Driver One information displays an Occupation of "Lawyer"
And Driver One Prefill information displays an Occupation of "Musician"

Now this scenario could, IMO, be improved. The name "Prefill" is poorly
chosen. Using a verb as a name is generally confusing. It was, however,
the commonly used name, and therefore not easily changed.

Using concrete examples for a particular field and particular values in
that field, though, communicated much more clearly than the more
abstract descriptions. And that is precisely the advantage that accrues
from using examples to illustrate the rules.

And, bringing this back to the subject line of this thread, this
illustrates the difficulty in assessing the quality of a BDD scenario.
There is no one exact point that can be pronounced "good quality." Human
judgement is better than any mnemonic.

- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Chuck van der Linden
2017-09-18 16:57:21 UTC
Permalink
Raw Message
Post by George Dinwiddie
Andrew,
It's possible to hide too much, and lose the advantages of concrete
examples.
<javascript:>
Post by Chuck van der Linden
Rather than an special 'reason' column this is a situation where I
would suggest you make use of the fact that scenario outlines can
keyword can have other text after it. e,g
<snip>
Post by Chuck van der Linden
This has two flaws from my perspective
1. Its harder to read and debug.
2. It still keeps the example data in the feature file. This is the
really important point.
I much prefer to extract examples out of scenarios getting them from
either step definition helpers or the application itself (if possible).
The thing with example data in scenarios is that when the rules change
the scenario has to change.
I'm with George on this one. IMHO a large purpose of the examples is to
describe the business logic with specific examples (because just words are
often too ambiguous). I'm all for hiding implementation detail in the
steps, things like what button to click, which field to fill in.. stuff
that might all change without the business logic being any different. But
I'm strongly against burying any of the details of the expected behavior
due to the business rules down in steps. The eventual result of seeking to
simplify and make more generic the step wording is to muddy the waters
around specifics that are important for all to understand. The other
things that happens with that approach, as your wording becomes more and
more generalized is you head towards what I call the 'one ring' scenario.

Given our application and a user
When the user uses our application
Then everything works the way we expect it to.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Premdas
2017-09-18 08:47:54 UTC
Permalink
Raw Message
Post by Andrew Premdas
Post by George Dinwiddie
Post by Andrew Premdas
As for testing whether a particular state is a removing state or not,
you can do that in a unit test that runs 100 times faster. >
Now you only have two states in your examples, but it would be very easy
to add more and if you apply that across multiple scenarios in a large
application you can see why people end up with cukes that have huge run
times.
And those unit tests might as well be greek to the business people. They
would be back to testing all of these things by hand.
I have no concern that my unit tests or my integration tests are greek to
business people. That is expected. The only thing my tests MAY have to do
is communicate to business people that their intentions are being met. If
business people are reading the output of Cucumber scenarios and using them
to determine whether their application functions or not then they are
idiots or completely misinformed. A cucumber scenario in itself going green
is completely meaningless, as every scenario will go green with no code.
Cucumber scenarios are just a way for business people to contribute to the
drive for development. Their executions proves nothing.
No matter how I implement my scenarios they and their output will never be
understood by business users in the same way that the code of my
application won't be understood by business users. My implemented scenarios
are code and their output is not for business users it is for the
developers who will have to build open the provided functionality. This
idea that you can use the output of cucumber scenarios to judge the health
of an application baffles me. If your a business person and you want to
judge the health of an application look at the application not a wall of
green text from cucumber scenarios.
If the output of Cucumber is meaningless to the people who require the
software why bother going to the effort of automating a Gherkin scenario?
Why add that layer that translate the Gherkin to code? You say their value
is to contribute to driving the development. Couldn’t you achieve that
simply by using concrete examples in conversations and then translating
them to a RSpec or JUnit, etc. Why are we writing in English (or other
natural language) if these things aren’t meant to be read by people. If
they are only meant for developers code is a less ambiguous to way to
specify things.
Because it drives the development of the the functionality that the
business people want. The automation provides regression so that the next
piece of functionality can be developed. We write scenarios to facilitate
communication between devs and business so the devs can have an
understanding of WHAT is wanted and WHY its important before they start
building stiff. But once a scenario has been written and is in use then I
see it having much less functionality for the business people and much more
functionality for the developers. This transition from a scenario being a
driver of development, to scenario being code is key. And a necessary part
of making that transition is extracting meaning from examples.
Post by Andrew Premdas
Cucumber & Gherkin has 3 values
1) Executable specifications based on concrete examples – this is how we
are able to consider the business rules and discuss whether they meet our
(users’) needs, and because we specify software before we write it they fit
naturally into a TDD process
2) Automated tests – as software is delivered, executable specs form part
of our regression suite. We learn quickly if new changes have effected
earlier work, and we can decide if this is expected or not
3) Documentation - these executable specifications, written in (almost)
natural language, can document what our software does. This is valuable to
people on the team and people outside of the team
There’s always a balance that needs to be struck between these three
values, and its down to each team to strike the right balance for
themselves. I do think though that the hardest to balance are automated
tests and documentation. Automated tests push us to brevity and the minimum
required as we get concerned with run length. Documentation screams for
concrete examples, for richness to interest a reader.
I don't really disagree with this, but I feel you are not appreciating the
lifespan of the scenario. In its creation 1) is paramount, but after its
implementation 1) is almost irrelevant. 2) doesn't even exist before a
scenario is implemented after its implementation 2) is paramount. 3) is a
sort of constant presence.

As I'm talking about what it takes to implement and live with scenarios 1)
has much less value at this point in the scenarios lifecycle. Surely it
makes more sense for a business person to experience the functionality of
something by interacting with its implementation (remember at this point in
time the functionality is implemented) rather than be reading a bit of
Gherkin.

Finally the original point of the thred is about exploring the idea
specifications based on concrete examples are not sufficient to implement
business rules. The examples need their meaning extracted so we can
explicitly state the intention of each rule and give it a name. If you
don't do this you are left with ambiguity and overly complex scenarios
which hinder future development.
Post by Andrew Premdas
Scenarios don't have to be tied to the UI, and don't have to run slowly.
Post by George Dinwiddie
Given the nature of this particular project (a portal that integrates a
number of back-end systems) I suspect they will be, for the most part, but
I've encouraged them to organize their javascript so that some can be
tested without deploying to a server.
Scenarios will always run slowly as they are by definition abstract
descriptions of the behaviour of an application described via its user
interface. The fact that I can run several hundred scenarios over my
application in 3 minutes in a single process on my laptop with a browser
doing the javascript does not effect this. I could run 10 times the number
of unit test in a few seconds IF I wrote them as well as I write my
scenarios. Wasting scenario runtime to test/exercise things that have
already been exercised is really detrimental to the overall effectiveness
of using Cucumber to drive development.
I disagree with your definition that scenarios must run against a user
interface. Scenarios are concerned with being business readable. Some of
them will run end-to-end, but they don’t need to.
I never made the assertion that scenarios must run against a user
interface. I made the assertion that scenarios will always run slower than
unit tests (assuming they are equally well written) because scenarios
execute larger chunks of functionality. (I hope we are agreed that using
Gherkin to write unit tests is a pretty big waste of time ;))

Anyhow good talking to you. Hope the above has cleared some things up.

All best

Andrew
Post by Andrew Premdas
I wrote about this before: http://tooky.co.uk/cucumber-and-full-stack-
testing/ as did Seb Rose: http://claysnow.co.uk/the-testing-iceberg/.
cheers,
Steve
All best
Andrew
Post by George Dinwiddie
Post by Andrew Premdas
For a better example of taking apart a more complex scenario have a look
at an ancient blog post of mine http://pages.andrew.
premdas.org/2011/06/27/composable-features-and-tables.html
In the original formulation of that, I notice that this feature is
checking two distinct and non-interacting sets of rules. One is the rules
about taxes, and one is the rules about shipping costs. I would divide the
feature file into separate features for each of these sets of rules before
doing anything else.
cheers,
George
Post by Andrew Premdas
All best
Nice example, Andrew. I have found that when using tables for
password rules I needed a <reason> column to explain which rule was
being tested. That's a sure sign to me that a different scenario is
beneficial.
I was at a client recently and they were delighted that they could
use Scenario Outlines rather than repeating the scenario. The issue
had to do with different legal requirements in different states. I
don't remember the details, so these made up examples will have to
suffice.
Scenario Outline: State requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then there is no option to remove the Secondary Named Insured
| state |
| AL |
| AK |
Scenario Outline: No state requirements for Spouse as SNI
Given the Primary Named Insured resides in <state>
And the Secondary Named Insured has a relationship of "Spouse"
When the Agent views the policy
Then the Agent can remove the Secondary Named Insured
| state |
| SC |
| SD |
As I recall, there were other state-oriented scenarios with value
limits, but I can't recall them well enough to generate examples.
I can certainly see how the above could be written as individual
scenarios, but it was both more convenient and less error prone to
implement it with tables.
- George
On 14 September 2017 at 16:57, George Dinwiddie
Andrew,
I'm not exactly sure what you mean by this. Can you give an
example
of extracting the spirit/meaning of the examples and naming
them?
- George
Sure, lets take something everyone knows about registrations
(excuse bad syntax I don't use outlines examples or tables
anymore).
Scenario Outline: Bad registrations
When I register with <account> <password>
Then I should not be register
Account | Password
bad_account | good_password
good_account | 1234
good_account | sdfdsflsdfsdf3313
good_account | wieikjkeiiow
OK so our business peeps have come up with some examples of
account|password combinations that should fail. Now we have to
take each pair and try and extract the spirit/meaning. I'll
start with the easy one
good_account | 1234
This is probably to do with password length. (note all examples
are by their nature ambiguous, this is why scenario outlines and
examples are bad in my book, they don't have information to be
clear about their intention)
So I'd extract a scenario
Scenario: Short password
When I register with a short password
Then I should see my password is to short
The next one is a bit more confusing. Some possibilities are
Need a capital letter
Need a special character
So here I might have to check with the business, and then I'd
rinse and repeat
From this process we end up extracting and making concrete
several important concepts
- short passwords
- passwords with special chars
- passwords without capitals
etc.
Now it might seem a stretch but you can apply this concept to
every scenario outline, and to every example group or table of
examples. And if you do, not only will you get simpler features
that are easier to implement and work with, you'll also get most
of the 'names' you need to write the code to implement the
particular behaviour you are specifiying (yay!)
HTH
Andrew
This ties in quite nicely with my idea/opinion that any
Cucumber
scenario that has examples in it, particularly tables
is in some
way incomplete.
You complete the scenario by extracting the
spirit/meaning of
the examples and in the process name things. When you
have done
this all the tables of examples just disappear and you
are left
with named concepts specific to your context.
All best
Andrew
On 14 September 2017 at 10:32, aslak hellesoy
Thanks for the clarification Gabriel.
In the past few years, after helping dozens of
organisations and
hundreds of people adopt BDD I have come to the
conclusion
that it
is counter-productive to have 3 amigos (BAs,
testers and
programmers) write executable specifications
(Gherkin)
together.
What I encourage instead is that they create
*examples*
together.
Examples are not executable specifications - they
are simply
conversation topics, often jotted down on a card
as a
sketch or a
few words. I recommend Example Mapping
(https://cucumber.io/blog/
2015/12/08/example-mapping-introduction <https://
cucumber.io/blog/2015/12/08/example-mapping-introduction>
<https://cucumber.io/blog/2015/12/08/example-mapping-
introduction <https://cucumber.io/blog/2015/12/08/
example-mapping-introduction>>
<https://cucumber.io/blog/
2015/12/08/example-mapping-introduction <https://
cucumber.io/blog/2015/12/08/example-mapping-introduction>
<https://cucumber.io/blog/2015/12/08/example-mapping-
introduction <https://cucumber.io/blog/2015/12/08/
example-mapping-introduction>>>)
as a technique to create examples together.
When conversations happen around lo-fi examples
like this in an
example mapping session, stakeholders tend to
uncover a lot of
misunderstandings and gain new insight. In my
experience,
this does
not happen when the conversations happen around
executable
specifications (Gherkin). Instead, conversations
tend to
revolve
around grammar and formulation, diverting the
attention
from the
problem domain and the examples themselves.
Writing
specifications
together is also slow and tedious, and people
don't enjoy these
sessions as much. When it's not enjoyable, people
stop doing it
pretty quickly. Example Mapping on the other hand
is
energising and fun.
BDD is a variant of TDD, where programmers write
code guided by
failing tests. In my experience, this workflow
does not
work well
when the specifications were written
collaboratively. When the
programmers starts the red-green-refactor cycle
with
collaboratively
written scenarios, they quickly realise they need
to
reformulate
those scenarios, for a number of reasons. They're
incomplete,
they're ambiguous etc. When scenarios start
passing, the
scenarios
are often very different from what the diverse
group created
together. This often causes frustration: "You've
completely
rewritten it - this is not what we wrote together
- I don't
understand this".
So instead I recommend programmers write the
executable
specifications, perhaps with assistance from a
tester. They
use the
example map as input when they do this, and do
their best
to capture
the spirit of the examples and the conversations.
When
they're done
(or better - while they're doing it), they show
the
scenarios to the
BAs/POs and ask for feedback: Do you understand
this? Does it
reflect what we discussed during example mapping?
Then they
tweak
it, trying to strike a good balance between 3
Specification, Test and Documentation.
BDD isn't a process where you can write the
specification
up-front -
even when it happens collaboratively. They take
shape as
you go, in
parallel with development. The only thing you can
do up-front
(before coding) is to reduce the
misunderstandings, but
executable
specifications isn't a good medium for that I
think.
Cheers,
Aslak
On Thu, Sep 14, 2017 at 7:18 AM, Gabriel Oliveira
Hi Aslak,
Thanks for pointing this out - this was a
mistake on
our part.
The intent was to state that fact that bad
requirements
are one
of the many causes of project failures.
Already fixed on bit.ly/bdd_quality
<http://bit.ly/bdd_quality>
<http://bit.ly/bdd_quality> <http://bit.ly/bdd_quality>
On Wed, Sep 13, 2017 at 11:03 PM, Aslak
HellesÞy
Hi Gabriel,
"Since it is well known that executable
solution
requirements are one of many potential
causes of a
project
failure, we come to ask some questions
about this
subject."
1) Are you saying that project failure
can be
partly caused
by the fact that specifications are
executable?
2) After 12 years in this field I've never
heard
this - on
what basis are you claiming this is well
known?
Cheers,
Aslak
On Thursday, September 7, 2017 at 2:17:45
PM UTC+1,
Gabriel
Hi folks,
Have you been practicing BDD
(Behavior-Driven
Development) lately and believe
scenarios'
quality matter?
I'm writing my graduate thesis on that
subject
and would
like to invite you all to answer a
(really) small 2
https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>
<https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>>
<https://goo.gl/forms/dheA8XK
0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>
<https://goo.gl/forms/dheA8XK0asiNUVze2
<https://goo.gl/forms/dheA8XK0asiNUVze2>>>
If you're willing to spend a little
more time
to talk to
me about that topic, please remember
to fill
out your
contact information there - or comment
on this
thread
here - and I will promptly get in
touch :) !
Best regards,
--
*Gabriel P.A. de Oliveira*
/https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>
<https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>>
<https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>
<https://gpaoliveirablog.wordpress.com
<https://gpaoliveirablog.wordpress.com>>>/
-- -----------------------------------------------------------
-----------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------
------------
-- Posting rules: http://cukes.info/posting-rules.html
<http://cukes.info/posting-rules.html>
--- You received this message because you are subscribed to the
Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it,
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org <http://blog.andrew.premdas.org>
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
--- You received this message because you are subscribed to the Google
Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups
"Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups
"Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Premdas
2017-09-19 10:28:53 UTC
Permalink
Raw Message
Post by Chuck van der Linden
Post by George Dinwiddie
Andrew,
It's possible to hide too much, and lose the advantages of concrete
examples.
Post by Chuck van der Linden
Rather than an special 'reason' column this is a situation where I
would suggest you make use of the fact that scenario outlines can
keyword can have other text after it. e,g
<snip>
Post by Chuck van der Linden
This has two flaws from my perspective
1. Its harder to read and debug.
2. It still keeps the example data in the feature file. This is the
really important point.
I much prefer to extract examples out of scenarios getting them from
either step definition helpers or the application itself (if possible).
The thing with example data in scenarios is that when the rules change
the scenario has to change.
I'm with George on this one. IMHO a large purpose of the examples is to
describe the business logic with specific examples (because just words are
often too ambiguous). I'm all for hiding implementation detail in the
steps, things like what button to click, which field to fill in.. stuff
that might all change without the business logic being any different. But
I'm strongly against burying any of the details of the expected behavior
due to the business rules down in steps. The eventual result of seeking to
simplify and make more generic the step wording is to muddy the waters
around specifics that are important for all to understand. The other
things that happens with that approach, as your wording becomes more and
more generalized is you head towards what I call the 'one ring' scenario.
Given our application and a user
When the user uses our application
Then everything works the way we expect it to.
Of course you can take any idea, and use a bad example to dismiss it. My
point is that examples need to be translated into specific, precise wording
that names concepts and provides an understanding that is relevant to the
context and can be shared. How have you got from that to your one ring
scenario?

Extracting meaning from examples is challenging work, which is one reason
people don't do it! You have to find the underlying concept behind each
example and give it a meaningful name, this requires in in depth
understanding of the business process, establishing a consensus about the
vocabulary you are using and ensuring uniqueness in your naming. Its
precisely the opposite of wording becoming more generalized. Scenarios do
become more concise but at the same time they become more precise.
Post by Chuck van der Linden
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups
"Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
------------------------
Andrew Premdas
blog.andrew.premdas.org
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...