[Cucumber] Is BDD enough?

[John]

So I was reading a lot about Cucumber, and have been using it for a while.
I have something that bothers me and I would like to hear your opinions.
Let's say I have an app the matches two people by comparing their online
profile. My feature file would like like so:

- I can create a profile
- I can view others profile
- When match is happening I get a notification
- I can check my previous matches

etc.
This part is crystal clear and does benefit from speaking behavioral. But
what if I want to test the matching system? Is very complex system, and
there is a lot of data connections within in. For example:

*X, Y, Z, 1, 2, 3 *will match *X, Y, Z, 1, 2, 4*, but will not match *X, Y,
Z, 1 ,3 ,2*. Why? Because your profile is marked as 'Male', because your
age is different, because your don't have shared friends and other stuff.

So technically, I can try to sit down and ask myself - what the above
combination means to the stakeholders? Can I break them down into smaller
parts?
This is tough thing to do. Because we write feature file retroactively -
It's require a lot of understanding in how the system work and what's the
data correlations are. Even if you do manage to do it, steps end very
verbal, or you have to add a big Cucumber table with data into it the
feature file become monstrous and not readable.

So very complex systems that are very data oriented always felt better
working on a Data Driven unit tests framework that support reading data
from external CSV, has better support in dynamic data tables etc.
So, back to the question - is BDD enough? Is it enough to use JUST
cucumber? or Cucumber is mainly an acceptance tests tool and it should be
used with more traditional unit/component/ testing tool?

[Mark Levison]

John - no tool is ever enough. As you already note Cucumber will help with
some things and not others. My question for your DataDriven tests (Cucumber
or otherwise) to what extent can you decouple them so we avoid problems of
MxNxOxP === many many many tests? There was some very good work a few years
ago on pairwise testing, where you got most of the value for a fraction of
the tests.

In addition as most people in this community will suggest always push the
test the furtherest down the stack you can. So if you can test something
with a Unit Test go for it.

Cheers
Mark

Post by John
So I was reading a lot about Cucumber, and have been using it for a while.
I have something that bothers me and I would like to hear your opinions.
Let's say I have an app the matches two people by comparing their online
- I can create a profile
- I can view others profile
- When match is happening I get a notification
- I can check my previous matches
etc.
This part is crystal clear and does benefit from speaking behavioral. But
what if I want to test the matching system? Is very complex system, and
*X, Y, Z, 1, 2, 3 *will match *X, Y, Z, 1, 2, 4*, but will not match *X,
Y, Z, 1 ,3 ,2*. Why? Because your profile is marked as 'Male', because
your age is different, because your don't have shared friends and other
stuff.
So technically, I can try to sit down and ask myself - what the above
combination means to the stakeholders? Can I break them down into smaller
parts?
This is tough thing to do. Because we write feature file retroactively -
It's require a lot of understanding in how the system work and what's the
data correlations are. Even if you do manage to do it, steps end very
verbal, or you have to add a big Cucumber table with data into it the
feature file become monstrous and not readable.
So very complex systems that are very data oriented always felt better
working on a Data Driven unit tests framework that support reading data
from external CSV, has better support in dynamic data tables etc.
So, back to the question - is BDD enough? Is it enough to use JUST
cucumber? or Cucumber is mainly an acceptance tests tool and it should be
used with more traditional unit/component/ testing tool?
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

--
[image: headshot-square-300x300]
<http://www.flickr.com/photos/***@N00/9674877046/> *Mark Levison* | 1
[phone] | Twitter <https://twitter.com/mlevison> | LinkedIn
<http://ca.linkedin.com/in/marklevison> | Facebook
<https://www.facebook.com/agilepainrelief>
Certified ScrumMaster Training: Vancouver
<http://agilepainrelief.com/courses/vancouver> | Edmonton
<http://agilepainrelief.com/courses/edmonton> | Ottawa
<http://agilepainrelief.com/courses/ottawa> | Montreal
<http://agilepainrelief.com/courses/montreal> | Toronto
<http://agilepainrelief.com/courses/toronto>
Certified Product Owner & Private Training also available ~ Our Training
Schedule <http://agilepainrelief.com/courses/certified-scrum-agile-training>
Agile Pain Relief Consulting <http://agilepainrelief.com/> | Notes from a
Tool User <http://agilepainrelief.com/notesfromatooluser>
Proud Sponsor of Agile Tour Gatineau Ottawa <http://goagiletour.ca/> and Agile
Coach Camp Canada <http://agilecoachcampcanada.wordpress.com/>

[John]

Hi Marks,

You are right, we trying to decouple those huge tests into more precise
tests that are more white-box oriented. I was wondering if people take
those (hopefully minimal) and implement them on Cucumber, or extract those
into a different tools. It seems like so based on your initial sentence.
I'm still trying to figure out the fine line between 'things we test using
BDD and Cucumber and things we do using UnitTests/Datadriven frameworks'.

Thanks!

Post by Mark Levison
John - no tool is ever enough. As you already note Cucumber will help with
some things and not others. My question for your DataDriven tests (Cucumber
or otherwise) to what extent can you decouple them so we avoid problems of
MxNxOxP === many many many tests? There was some very good work a few years
ago on pairwise testing, where you got most of the value for a fraction of
the tests.
In addition as most people in this community will suggest always push the
test the furtherest down the stack you can. So if you can test something
with a Unit Test go for it.
Cheers
Mark

Post by John
So I was reading a lot about Cucumber, and have been using it for a
while. I have something that bothers me and I would like to hear your
opinions.
Let's say I have an app the matches two people by comparing their online
- I can create a profile
- I can view others profile
- When match is happening I get a notification
- I can check my previous matches
etc.
This part is crystal clear and does benefit from speaking behavioral. But
what if I want to test the matching system? Is very complex system, and
*X, Y, Z, 1, 2, 3 *will match *X, Y, Z, 1, 2, 4*, but will not match *X,
Y, Z, 1 ,3 ,2*. Why? Because your profile is marked as 'Male', because
your age is different, because your don't have shared friends and other
stuff.
So technically, I can try to sit down and ask myself - what the above
combination means to the stakeholders? Can I break them down into smaller
parts?
This is tough thing to do. Because we write feature file retroactively -
It's require a lot of understanding in how the system work and what's the
data correlations are. Even if you do manage to do it, steps end very
verbal, or you have to add a big Cucumber table with data into it the
feature file become monstrous and not readable.
So very complex systems that are very data oriented always felt better
working on a Data Driven unit tests framework that support reading data
from external CSV, has better support in dynamic data tables etc.
So, back to the question - is BDD enough? Is it enough to use JUST
cucumber? or Cucumber is mainly an acceptance tests tool and it should be
used with more traditional unit/component/ testing tool?
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

--
[image: headshot-square-300x300]
[phone] | Twitter <https://twitter.com/mlevison> | LinkedIn
<http://ca.linkedin.com/in/marklevison> | Facebook
<https://www.facebook.com/agilepainrelief>
Certified ScrumMaster Training: Vancouver
<http://agilepainrelief.com/courses/vancouver> | Edmonton
<http://agilepainrelief.com/courses/edmonton> | Ottawa
<http://agilepainrelief.com/courses/ottawa> | Montreal
<http://agilepainrelief.com/courses/montreal> | Toronto
<http://agilepainrelief.com/courses/toronto>
Certified Product Owner & Private Training also available ~ Our Training
Schedule
<http://agilepainrelief.com/courses/certified-scrum-agile-training>
Agile Pain Relief Consulting <http://agilepainrelief.com/> | Notes from a
Tool User <http://agilepainrelief.com/notesfromatooluser>
Proud Sponsor of Agile Tour Gatineau Ottawa <http://goagiletour.ca/> and Agile
Coach Camp Canada <http://agilecoachcampcanada.wordpress.com/>

[Tim Walker]

Post by John
So I was reading a lot about Cucumber, and have been using it for a

while. I have something that bothers me and I would like to hear your
opinions.

Post by John
Let's say I have an app the matches two people by comparing their online
- I can create a profile
- I can view others profile
- When match is happening I get a notification
- I can check my previous matches
etc.
This part is crystal clear and does benefit from speaking behavioral. But

what if I want to test the matching system? Is very complex system, and

Post by John
X, Y, Z, 1, 2, 3 will match X, Y, Z, 1, 2, 4, but will not match X, Y, Z,

1 ,3 ,2. Why? Because your profile is marked as 'Male', because your age is
different, because your don't have shared friends and other stuff.

Post by John
So technically, I can try to sit down and ask myself - what the above

combination means to the stakeholders? Can I break them down into smaller
parts?

Post by John
This is tough thing to do. Because we write feature file retroactively -

It's require a lot of understanding in how the system work and what's the
data correlations are. Even if you do manage to do it, steps end very
verbal, or you have to add a big Cucumber table with data into it the
feature file become monstrous and not readable.

If you write the feature retroactively you are most definitely NOT doing
any form of BDD.

Post by John
So very complex systems that are very data oriented always felt better

working on a Data Driven unit tests framework that support reading data
from external CSV, has better support in dynamic data tables etc.

Post by John
So, back to the question - is BDD enough? Is it enough to use JUST

cucumber? or Cucumber is mainly an acceptance tests tool and it should be
used with more traditional unit/component/ testing tool?

Post by John
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[John]

Doing it retroactively is indeed not optimal, but what is the other option?
not do any BDD and only add Feature files for new features? Doing BDD
retroactively help us to better understand the project, and will help
future feature. So we already have working code right, but now we extract
it into a human language. It's not that bad isn't it?

Post by Tim Walker
If you write the feature retroactively you are most definitely NOT doing
any form of BDD.

[aslak hellesoy]

Post by John
Doing it retroactively is indeed not optimal, but what is the other
option? not do any BDD and only add Feature files for new features? Doing
BDD retroactively help us to better understand the project, and will help
future feature. So we already have working code right, but now we extract
it into a human language. It's not that bad isn't it?

I'm not sure exactly what about your workflow you consider to be BDD.

I've tried to summarise what BDD is (to me at least) in a keynote I gave at
CukeUp! last week:
<https://skillsmatter.com/skillscasts/7361-keynote-kind-of-green> (You need
to sign up to watch, but it's free).

Aslak

Post by John

Post by Tim Walker
If you write the feature retroactively you are most definitely NOT doing
any form of BDD.

--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[John]

Thanks Aslak and George for answer my questions. The keynote was very
useful for me (Sadly, I couldn't find your Talk George).

So, I was under the impression that there is SOME benefit is writing
feature files to our exists code. It's indeed not 'behavioral driven' as
all the logic is already written, but it does help us to achieve two things:

1. Sometimes we revise or add modifications to old components. Having a
feature file (that was created retroactively) can make the the new
modifications easier to grasp by our stakeholders and help us to make
at-least the modification - behavioral driven.

2. Creating a retroactive feature files means we could write Tests for them
(which we currently don't have). This put Cucumber as a more of a
'framework' rather then a tool for collaboration as you mentioned (but at
least we have tests that test the spec).

Your feedback made me think again about writing feature file retrospective,
maybe it will not be wise even with the above example.

So, I'm still wondering how Software Development Cycle looks in a 'BDD'
environment. All stakeholders collaborate and we create a behavioral
feature files. We write the test code - it fails. We write the dev code
implementation. We run they tests and they fail again. We fix the code. We
run the tests and everything green. Up until now everything good.

But then we are still missing more 'complex' tests. Integration tests,
performance tests, data driven tests, unit tests etc. According to George
those should not be maintained in feature files. So we revert into more
unit-test oriented tools. When do we write those? after the feature driven
tests are green? during it? are the developers responsible on writing both
feature file tests and unit testing? A feature driven feature file may test
I can log into the system - a unit-tests might do the same while testing
short vs long password. Is it OK they we now have "almost" duplicated tests?

I would like to hear your experience on how Feature driven testing mix into
your local software development cycle, because I think that people around
think that BDD and Cucumber should be the source and only solution for
testings. If feature files only talk in a language everyone understand, I'm
guessing it should only solve us the need to automate somewhere else
acceptance tests?

Thanks!

[Andrew Premdas]

Have you seen the bdd cycle diagram there is a version here (
https://camo.githubusercontent.com/bea2b47618dac09fbc7f70a70ed5c5bd3194668f/687474703a2f2f692e737461636b2e696d6775722e636f6d2f6d744c414d2e706e67
)

Personally I use Cucumber to drive my development by first writing/refining
scenarios to get a high level understanding of the problem space and to
clarify language. I find this helps with one of the hardest problems in
computer science ... naming things. With good names we can start writing
code from the outside, working inwards. We start with a failing scenario
and its error message. If this is a good error message we can use it to
drive the next bit of development. For example if we were adding the
ability to add posts to a Blog we might have some step definition code that
takes us to a page where we create posts (in rails this would be something
as simple as `visit new_posts_path). Then we would get a series of tiny
bits of codes followed by errors followed by more code. e.g.

missing route new_posts_path so we add a route
missing controller PostsController so we add a PostsController
missing method New for PostsController, so we add ...

This explores the 'happy' and simple path and creates tools and
infrastructure to start exploring 'Posting' in greater detail. Generally
the more detailed you get and the more sad paths and edge cases you explore
the greater your need to use other testing tools in addition to Cucumber.
The idea that Cucumber can be the single point of truth and the only
testing tool used for an application is totally flawed. Its a very
attractive idea, and I really really wanted it to be true (many years ago)
but its just Wrong!

When the code is already wriiten Cucumber scenarios have to work with the
existing mess (this is really hard) instead of guiding you to write simple
code (which is much easier). So to use Cucumber with existing code you have
to be much more experienced with it, both from the implementation side
(which is rarely talked about) and from the scenario writing side.

HTH some

All best

Andrew

Post by John
Thanks Aslak and George for answer my questions. The keynote was very
useful for me (Sadly, I couldn't find your Talk George).
So, I was under the impression that there is SOME benefit is writing
feature files to our exists code. It's indeed not 'behavioral driven' as
1. Sometimes we revise or add modifications to old components. Having a
feature file (that was created retroactively) can make the the new
modifications easier to grasp by our stakeholders and help us to make
at-least the modification - behavioral driven.
2. Creating a retroactive feature files means we could write Tests for
them (which we currently don't have). This put Cucumber as a more of a
'framework' rather then a tool for collaboration as you mentioned (but at
least we have tests that test the spec).
Your feedback made me think again about writing feature file
retrospective, maybe it will not be wise even with the above example.
So, I'm still wondering how Software Development Cycle looks in a 'BDD'
environment. All stakeholders collaborate and we create a behavioral
feature files. We write the test code - it fails. We write the dev code
implementation. We run they tests and they fail again. We fix the code. We
run the tests and everything green. Up until now everything good.
But then we are still missing more 'complex' tests. Integration tests,
performance tests, data driven tests, unit tests etc. According to George
those should not be maintained in feature files. So we revert into more
unit-test oriented tools. When do we write those? after the feature driven
tests are green? during it? are the developers responsible on writing both
feature file tests and unit testing? A feature driven feature file may test
I can log into the system - a unit-tests might do the same while testing
short vs long password. Is it OK they we now have "almost" duplicated tests?
I would like to hear your experience on how Feature driven testing mix
into your local software development cycle, because I think that people
around think that BDD and Cucumber should be the source and only solution
for testings. If feature files only talk in a language everyone understand,
I'm guessing it should only solve us the need to automate somewhere else
acceptance tests?
Thanks!
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

--
------------------------
Andrew Premdas
blog.andrew.premdas.org

[George Dinwiddie]

John,

Post by John
Thanks Aslak and George for answer my questions. The keynote was very
useful for me (Sadly, I couldn't find your Talk George).

The talk was not recorded. It's in the book, though, and there's more
lessons in the README and following the evolution in the git commits.

Post by John
So, I was under the impression that there is SOME benefit is writing
feature files to our exists code. It's indeed not 'behavioral driven' as
1. Sometimes we revise or add modifications to old components. Having a
feature file (that was created retroactively) can make the the new
modifications easier to grasp by our stakeholders and help us to make
at-least the modification - behavioral driven.
2. Creating a retroactive feature files means we could write Tests for
them (which we currently don't have). This put Cucumber as a more of a
'framework' rather then a tool for collaboration as you mentioned (but
at least we have tests that test the spec).
Your feedback made me think again about writing feature file
retrospective, maybe it will not be wise even with the above example.
So, I'm still wondering how Software Development Cycle looks in a 'BDD'
environment. All stakeholders collaborate and we create a behavioral
feature files. We write the test code - it fails. We write the dev code
implementation. We run they tests and they fail again. We fix the code.
We run the tests and everything green. Up until now everything good.
But then we are still missing more 'complex' tests. Integration tests,
performance tests, data driven tests, unit tests etc. According to
George those should not be maintained in feature files. So we revert
into more unit-test oriented tools. When do we write those? after the
feature driven tests are green? during it?

I don't suppose you looked at the README of the github project where I
show the path I took. I talk there about the decisions I made, when and why.

Post by John
are the developers
responsible on writing both feature file tests and unit testing? A
feature driven feature file may test I can log into the system - a
unit-tests might do the same while testing short vs long password. Is it
OK they we now have "almost" duplicated tests?

I don't worry much about some duplication between features and unit
tests. It doesn't turn out to be much for me. The evolution of the
project I shared with you shows that.

Post by John
I would like to hear your experience on how Feature driven testing mix
into your local software development cycle, because I think that people
around think that BDD and Cucumber should be the source and only
solution for testings. If feature files only talk in a language everyone
understand, I'm guessing it should only solve us the need to automate
somewhere else acceptance tests?

I'm not sure what you mean by that paragraph.

- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------

[George Dinwiddie]

John,

Post by John
So I was reading a lot about Cucumber, and have been using it for a
while. I have something that bothers me and I would like to hear your
opinions.
Let's say I have an app the matches two people by comparing their online
- I can create a profile
- I can view others profile
- When match is happening I get a notification
- I can check my previous matches
etc.
This part is crystal clear and does benefit from speaking behavioral.
But what if I want to test the matching system? Is very complex system,
*X, Y, Z, 1, 2, 3 *will match *X, Y, Z, 1, 2, 4*, but will not match *X,
Y, Z, 1 ,3 ,2*. Why? Because your profile is marked as 'Male', because
your age is different, because your don't have shared friends and other
stuff.
So technically, I can try to sit down and ask myself - what the above
combination means to the stakeholders? Can I break them down into
smaller parts?
This is tough thing to do. Because we write feature file retroactively -
It's require a lot of understanding in how the system work and what's
the data correlations are. Even if you do manage to do it, steps end
very verbal, or you have to add a big Cucumber table with data into it
the feature file become monstrous and not readable.
So very complex systems that are very data oriented always felt better
working on a Data Driven unit tests framework that support reading data
from external CSV, has better support in dynamic data tables etc.
So, back to the question - is BDD enough? Is it enough to use JUST
cucumber? or Cucumber is mainly an acceptance tests tool and it should
be used with more traditional unit/component/ testing tool?

First, go read
https://cucumber.io/blog/2014/03/03/the-worlds-most-misunderstood-collaboration-tool
BDD is not in the tool; it's in the conversation. The conversation comes
first, and tries to achieve shared understanding of the business rules
you want to implement. You test that shared understanding by discussing
scenarios that illustrate those business rules. Then turn scenarios into
automated tests where it seems it will help. After that, implement the
system that implements that functionality.

As far as I can tell, you're working completely backwards. It's no
wonder that your tests don't have meaning to your stakeholders.

- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------

[John]

Thanks for the replies :)

You are right by saying that retroactively writing feature file is not a
'pure BDD'. But what other alternatives we have? we want to start doing
BDD, and new features are indeed being develop the proper way (we talk, we
write test, we implement). We do see the benefit in having our older code
also documents, and which it's not easy - we think that having older code
having feature file is still worth the extra work.

But this was not my main concern. My main concern was to learn from people
who doing BDD if they have any other testing frameworks and tools besides
Cucumber in their eco-system, and whatever they they somehow bridge BDD
feature files with those tools (using feature file for unit-testing?)

Mark hinted Cucumber it might not be so and I would like to hear where this
fine line of 'what we write feature file for' and what we use differently
cross.
Thanks!

Post by George Dinwiddie
John,

Post by John
So I was reading a lot about Cucumber, and have been using it for a
while. I have something that bothers me and I would like to hear your
opinions.
Let's say I have an app the matches two people by comparing their online
- I can create a profile
- I can view others profile
- When match is happening I get a notification
- I can check my previous matches
etc.
This part is crystal clear and does benefit from speaking behavioral.
But what if I want to test the matching system? Is very complex system,
*X, Y, Z, 1, 2, 3 *will match *X, Y, Z, 1, 2, 4*, but will not match *X,
Y, Z, 1 ,3 ,2*. Why? Because your profile is marked as 'Male', because
your age is different, because your don't have shared friends and other
stuff.
So technically, I can try to sit down and ask myself - what the above
combination means to the stakeholders? Can I break them down into
smaller parts?
This is tough thing to do. Because we write feature file retroactively -
It's require a lot of understanding in how the system work and what's
the data correlations are. Even if you do manage to do it, steps end
very verbal, or you have to add a big Cucumber table with data into it
the feature file become monstrous and not readable.
So very complex systems that are very data oriented always felt better
working on a Data Driven unit tests framework that support reading data
from external CSV, has better support in dynamic data tables etc.
So, back to the question - is BDD enough? Is it enough to use JUST
cucumber? or Cucumber is mainly an acceptance tests tool and it should
be used with more traditional unit/component/ testing tool?

First, go read
https://cucumber.io/blog/2014/03/03/the-worlds-most-misunderstood-collaboration-tool
BDD is not in the tool; it's in the conversation. The conversation comes
first, and tries to achieve shared understanding of the business rules
you want to implement. You test that shared understanding by discussing
scenarios that illustrate those business rules. Then turn scenarios into
automated tests where it seems it will help. After that, implement the
system that implements that functionality.
As far as I can tell, you're working completely backwards. It's no
wonder that your tests don't have meaning to your stakeholders.
- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------

[Mark Levison]

John writing from my phone in the back of a bouncing taxi on the way the
Scrum Gathering, please excuse errors.

Tool choices and options boil down to what your strategy.

If you what to evolve existing hard to test APIs into more testable and
human understandable interfaces then Cucumber is a great tool to engage
your stakeholders in conversations around what behaviours do you really
want from your system.

In many cases your stakeholders maybe impatient. In which use BDD for all
new changes might the way to go and simply bring the legacy system under
test as a side effect of change.

Key point Cucumber and BDD are a collaboration game. No collaboration, no
BDD.

Also remember whatever tool you use long running tests are complicated and
risky. Each should really only test one behaviour.

If the language of Cucumber doesn't work for your team, I've also seen
RobotFramework used well at clients. Risk you can do BDD but you can also
do pure functional test automation. I recommend avoiding that.

Arriving at the airport, good luck
Mark

Post by John
Thanks for the replies :)
You are right by saying that retroactively writing feature file is not a
'pure BDD'. But what other alternatives we have? we want to start doing
BDD, and new features are indeed being develop the proper way (we talk, we
write test, we implement). We do see the benefit in having our older code
also documents, and which it's not easy - we think that having older code
having feature file is still worth the extra work.
But this was not my main concern. My main concern was to learn from people
who doing BDD if they have any other testing frameworks and tools besides
Cucumber in their eco-system, and whatever they they somehow bridge BDD
feature files with those tools (using feature file for unit-testing?)
Mark hinted Cucumber it might not be so and I would like to hear where
this fine line of 'what we write feature file for' and what we use
differently cross.
Thanks!

Post by John
John,

Post by John
So I was reading a lot about Cucumber, and have been using it for a
while. I have something that bothers me and I would like to hear your
opinions.
Let's say I have an app the matches two people by comparing their

online

Post by John
- I can create a profile
- I can view others profile
- When match is happening I get a notification
- I can check my previous matches
etc.
This part is crystal clear and does benefit from speaking behavioral.
But what if I want to test the matching system? Is very complex

system,

Post by John
*X, Y, Z, 1, 2, 3 *will match *X, Y, Z, 1, 2, 4*, but will not match

*X,

Post by John
Y, Z, 1 ,3 ,2*. Why? Because your profile is marked as 'Male', because
your age is different, because your don't have shared friends and other
stuff.
So technically, I can try to sit down and ask myself - what the above
combination means to the stakeholders? Can I break them down into
smaller parts?
This is tough thing to do. Because we write feature file retroactively

-

Post by John
It's require a lot of understanding in how the system work and what's
the data correlations are. Even if you do manage to do it, steps end
very verbal, or you have to add a big Cucumber table with data into it
the feature file become monstrous and not readable.
So very complex systems that are very data oriented always felt better
working on a Data Driven unit tests framework that support reading data
from external CSV, has better support in dynamic data tables etc.
So, back to the question - is BDD enough? Is it enough to use JUST
cucumber? or Cucumber is mainly an acceptance tests tool and it should
be used with more traditional unit/component/ testing tool?

First, go read
https://cucumber.io/blog/2014/03/03/the-worlds-most-misunderstood-collaboration-tool
BDD is not in the tool; it's in the conversation. The conversation comes
first, and tries to achieve shared understanding of the business rules
you want to implement. You test that shared understanding by discussing
scenarios that illustrate those business rules. Then turn scenarios into
automated tests where it seems it will help. After that, implement the
system that implements that functionality.
As far as I can tell, you're working completely backwards. It's no
wonder that your tests don't have meaning to your stakeholders.
- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--

Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[John]

Hi Mark :)

Thanks, I bet writing on a bouncing taxi is not a small feat :)
Yes, we previously used RobotFramework. I think this is why Cucumber is
misunderstood here, because I was picked as an alternative to Robot. That's
why I'm working sort of taking a step for more BDD oriented feature files
without tons of tables with data in them.
Thanks!

Post by Mark Levison
John writing from my phone in the back of a bouncing taxi on the way the
Scrum Gathering, please excuse errors.
Tool choices and options boil down to what your strategy.
If you what to evolve existing hard to test APIs into more testable and
human understandable interfaces then Cucumber is a great tool to engage
your stakeholders in conversations around what behaviours do you really
want from your system.
In many cases your stakeholders maybe impatient. In which use BDD for all
new changes might the way to go and simply bring the legacy system under
test as a side effect of change.
Key point Cucumber and BDD are a collaboration game. No collaboration, no
BDD.
Also remember whatever tool you use long running tests are complicated and
risky. Each should really only test one behaviour.
If the language of Cucumber doesn't work for your team, I've also seen
RobotFramework used well at clients. Risk you can do BDD but you can also
do pure functional test automation. I recommend avoiding that.
Arriving at the airport, good luck
Mark

Post by John
Thanks for the replies :)
You are right by saying that retroactively writing feature file is not a
'pure BDD'. But what other alternatives we have? we want to start doing
BDD, and new features are indeed being develop the proper way (we talk, we
write test, we implement). We do see the benefit in having our older code
also documents, and which it's not easy - we think that having older code
having feature file is still worth the extra work.
But this was not my main concern. My main concern was to learn from
people who doing BDD if they have any other testing frameworks and tools
besides Cucumber in their eco-system, and whatever they they somehow bridge
BDD feature files with those tools (using feature file for unit-testing?)
Mark hinted Cucumber it might not be so and I would like to hear where
this fine line of 'what we write feature file for' and what we use
differently cross.
Thanks!

Post by John
John,

Post by John
So I was reading a lot about Cucumber, and have been using it for a
while. I have something that bothers me and I would like to hear your
opinions.
Let's say I have an app the matches two people by comparing their

online

Post by John
- I can create a profile
- I can view others profile
- When match is happening I get a notification
- I can check my previous matches
etc.
This part is crystal clear and does benefit from speaking behavioral.
But what if I want to test the matching system? Is very complex

system,

Post by John
*X, Y, Z, 1, 2, 3 *will match *X, Y, Z, 1, 2, 4*, but will not match

*X,

Post by John
Y, Z, 1 ,3 ,2*. Why? Because your profile is marked as 'Male', because
your age is different, because your don't have shared friends and

other

Post by John
stuff.
So technically, I can try to sit down and ask myself - what the above
combination means to the stakeholders? Can I break them down into
smaller parts?
This is tough thing to do. Because we write feature file retroactively

-

Post by John
It's require a lot of understanding in how the system work and what's
the data correlations are. Even if you do manage to do it, steps end
very verbal, or you have to add a big Cucumber table with data into it
the feature file become monstrous and not readable.
So very complex systems that are very data oriented always felt better
working on a Data Driven unit tests framework that support reading

data

Post by John
from external CSV, has better support in dynamic data tables etc.
So, back to the question - is BDD enough? Is it enough to use JUST
cucumber? or Cucumber is mainly an acceptance tests tool and it should
be used with more traditional unit/component/ testing tool?

First, go read
https://cucumber.io/blog/2014/03/03/the-worlds-most-misunderstood-collaboration-tool
BDD is not in the tool; it's in the conversation. The conversation comes
first, and tries to achieve shared understanding of the business rules
you want to implement. You test that shared understanding by discussing
scenarios that illustrate those business rules. Then turn scenarios into
automated tests where it seems it will help. After that, implement the
system that implements that functionality.
As far as I can tell, you're working completely backwards. It's no
wonder that your tests don't have meaning to your stakeholders.
- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--

Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[George Dinwiddie]

John,

Post by John
Thanks for the replies :)
You are right by saying that retroactively writing feature file is not a
'pure BDD'. But what other alternatives we have?

You have lots of alternatives, but I wouldn't *center* any of them on
writing feature files. The core of BDD is deciding what behavior you
(plural, collectively) want from the system. The common understanding
with the stakeholders is essential. If you can describe the behavior to
them, and they agree, then you're making progress. If they can't
understand what you mean, then perhaps you're on a level that's more
technical than business oriented.

Post by John
We want to start doing
BDD, and new features are indeed being develop the proper way (we talk,
we write test, we implement). We do see the benefit in having our older
code also documents, and which it's not easy - we think that having
older code having feature file is still worth the extra work.
But this was not my main concern. My main concern was to learn from
people who doing BDD if they have any other testing frameworks and tools
besides Cucumber in their eco-system, and whatever they they somehow
bridge BDD feature files with those tools (using feature file for
unit-testing?)
Mark hinted Cucumber it might not be so and I would like to hear where
this fine line of 'what we write feature file for' and what we use
differently cross.

I drive into unit tests (using TDD) pretty quickly. I talked about this
recently at the Agile Alliance Technical Conference. The code I talked
about is found on github (https://github.com/gdinwiddie/EquineHoroscope)
along with some "as I went" muttering to myself. The book about the talk
is on LeanPub (https://leanpub.com/EvolutionaryAnatomy/).

I do not recommend using feature files for unit testing. That path will
hurt you. Use feature files to document and verify what is important for
your stakeholders.

- George

Post by John
Thanks!
John,

Post by John
So I was reading a lot about Cucumber, and have been using it for a
while. I have something that bothers me and I would like to hear

your

Post by John
opinions.
Let's say I have an app the matches two people by comparing their

online

Post by John
- I can create a profile
- I can view others profile
- When match is happening I get a notification
- I can check my previous matches
etc.
This part is crystal clear and does benefit from speaking

behavioral.

Post by John
But what if I want to test the matching system? Is very complex

system,

Post by John
*X, Y, Z, 1, 2, 3 *will match *X, Y, Z, 1, 2, 4*, but will not

match *X,

Post by John
Y, Z, 1 ,3 ,2*. Why? Because your profile is marked as 'Male',

because

Post by John
your age is different, because your don't have shared friends and

other

Post by John
stuff.
So technically, I can try to sit down and ask myself - what the

above

Post by John
combination means to the stakeholders? Can I break them down into
smaller parts?
This is tough thing to do. Because we write feature file

retroactively -

Post by John
It's require a lot of understanding in how the system work and

what's

Post by John
the data correlations are. Even if you do manage to do it, steps

end

Post by John
very verbal, or you have to add a big Cucumber table with data

into it

Post by John
the feature file become monstrous and not readable.
So very complex systems that are very data oriented always felt

better

Post by John
working on a Data Driven unit tests framework that support

reading data

Post by John
from external CSV, has better support in dynamic data tables etc.
So, back to the question - is BDD enough? Is it enough to use JUST
cucumber? or Cucumber is mainly an acceptance tests tool and it

should

Post by John
be used with more traditional unit/component/ testing tool?

First, go read
https://cucumber.io/blog/2014/03/03/the-worlds-most-misunderstood-collaboration-tool
<https://cucumber.io/blog/2014/03/03/the-worlds-most-misunderstood-collaboration-tool>
BDD is not in the tool; it's in the conversation. The conversation comes
first, and tries to achieve shared understanding of the business rules
you want to implement. You test that shared understanding by discussing
scenarios that illustrate those business rules. Then turn scenarios into
automated tests where it seems it will help. After that, implement the
system that implements that functionality.
As far as I can tell, you're working completely backwards. It's no
wonder that your tests don't have meaning to your stakeholders.
- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------

[Chuck van der Linden]

Post by John
Thanks for the replies :)
... ...But this was not my main concern. My main concern was to learn
from people who doing BDD if they have any other testing frameworks and
tools besides Cucumber in their eco-system, and whatever they they somehow
bridge BDD feature files with those tools (using feature file for
unit-testing?)

The testing performed via any BDD tool typically is of 'acceptance' or
'integration' level. It is rarely (nor I would I argue should it ever be)
the only test automation being done by a team developing software. Most
teams wrap BDD around a TDD process used to create code units. Many
leverage other testing tools and approaches as well
(load/stress/performance, etc etc)

I'd suggest you google 'Test Automation Pyramid' and also "testing
quadrants". I'd also strongly recommend the book "Agile Testing" by
Crispin and Gregory. Also of course "The Cucumber Book" and
"Specification by Example"

And as others have pointed out (and I agree with) the primary purpose of
BDD is shared understanding. Automated tests and knowing your product is
still behaving as specified are in some senses a happy byproduct of that.

[John]

apremdas,
Thank you, this is actually a very good example to me on how a proper BDD
should look like.

David Kowis,
So I'm wondering. If Cucumber is a tool that serves a language and is not
aimed to be a testing framework, why does it has Cucumber Tables? Scenario
outlines yelling 'use parameters tests on me'.

George Dinwiddie,
Oh, I'm sorry. I was sure it was recorded for some reason, I'll have a look
on that book and the repository README file.

Chuck van der Linden,
Thank you, we already own 'The cucumber Book' - and I'll be checking the
other links you supplied.

I would like to thank anyone who contributed to the discussion. This is (at
least for me) was very educational. I think I'm starting to see the bigger
pictures here. Correct me if I'm (again) wrong, but Cucumber is not a
strict testing framework, but a tool to push your BDD develop. You combine
tests to sample your feature developing process and it helps your identify
what your missing in a cleaner way.
One your done writing your base code and the feature tests pass, it's time
to write more in-depth tests, and those probably should be written as unit
tests in a different tool. Some of you also mentioned that after the
feature is develop and green, there is not real meaning to keep running the
Cucumber tests as your no longer developing but rather fixing bugs (unless
you add a new feature ofcourse).
If it's indeed true, then a lot of the articles I was reading got Cucumber
wrong.

I'm also interesting to know how you write your cucumber tests. Do you test
things on the 'user-level' (ie, web application using selenium), or the
scope is smaller (calling your API directly)?

[Mark Levison]

Post by John
One your done writing your base code and the feature tests pass, it's time
to write more in-depth tests, and those probably should be written as unit
tests in a different tool. Some of you also mentioned that after the
feature is develop and green, there is not real meaning to keep running the
Cucumber tests as your no longer developing but rather fixing bugs (unless
you add a new feature ofcourse).

​Up until this paragraph we agreed. What would happen if your Examples
tested the core value of your product what other acceptance/integeration
tests would you need?​

I'm also interesting to know how you write your cucumber tests. Do you test

Post by John
things on the 'user-level' (ie, web application using selenium), or the
scope is smaller (calling your API directly)?

​Mostly API, with just enough GUI to prove I had the correct API hooked up
underneath.

Cheers
Mark in Arlington Virginia - missing George :-)​
--
[image: headshot-square-300x300]
<http://www.flickr.com/photos/***@N00/9674877046/> *Mark Levison* | 1
[phone] | Twitter <https://twitter.com/mlevison> | LinkedIn
<http://ca.linkedin.com/in/marklevison> | Facebook
<https://www.facebook.com/agilepainrelief>
Certified ScrumMaster Training: Vancouver
<http://agilepainrelief.com/courses/vancouver> | Edmonton
<http://agilepainrelief.com/courses/edmonton> | Ottawa
<http://agilepainrelief.com/courses/ottawa> | Montreal
<http://agilepainrelief.com/courses/montreal> | Toronto
<http://agilepainrelief.com/courses/toronto>
Certified Product Owner & Private Training also available ~ Our Training
Schedule <http://agilepainrelief.com/courses/certified-scrum-agile-training>
Agile Pain Relief Consulting <http://agilepainrelief.com/> | Notes from a
Tool User <http://agilepainrelief.com/notesfromatooluser>
Proud Sponsor of Agile Tour Gatineau Ottawa <http://goagiletour.ca/> and Agile
Coach Camp Canada <http://agilecoachcampcanada.wordpress.com/>

[John]

​What would happen if your Examples tested the core value of your product
what other acceptance/integeration tests would you need?​

Perhaps your right. It's possible the after I have my 'base'
acceptance/integration tests written and working, I can come back to add
more 'meat' or 'examples' into Cucumber. But I think it was George
mentioned that I should be careful with writing unit-tests with my
Cucumber. So as long as you stay with 'acceptance' tests, I think Cucumber
i indeed enough. The line between what we test with Cucumber versus what we
test with Unit-tests/performance tools is a line I can't draw yet. I guess
it comes with experience.

​Mostly API, with just enough GUI to prove I had the correct API hooked up

underneath.

What that be public API or something similar to unit-testing (calling
undocumented methods)?

[Chuck van der Linden]

Post by John
So I'm wondering. If Cucumber is a tool that serves a language and is not
aimed to be a testing framework, why does it has Cucumber Tables? Scenario
outlines yelling 'use parameters tests on me'.

because there are many cases where a scenario outline is the easiest way to
describe behavior as opposed to having to use 3 or 4 or more scenarios with
nearly identical steps. It's easier to follow the logic for conditional
behaviors if the differences are side by side in a table vs spread out over
multiple scenarios. a good example is something like a button that might
be enabled or disabled based on a combination of factors.

[Massimo Manca]

Post by John
So I'm wondering. If Cucumber is a tool that serves a language and
is not aimed to be a testing framework, why does it has Cucumber
Tables? Scenario outlines yelling 'use parameters tests on me'.

Because Gherkin describes requirements using examples.
But, also if I was criticized (and I understand the reasons) I said that
Gherkin syntax is good also to describe test so that non technical
persons may understand and help to write tests.

Post by John
because there are many cases where a scenario outline is the easiest
way to describe behavior as opposed to having to use 3 or 4 or more
scenarios with nearly identical steps. It's easier to follow the
logic for conditional behaviors if the differences are side by side in
a table vs spread out over multiple scenarios. a good example is
something like a button that might be enabled or disabled based on a
combination of factors.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

---
Questa e-mail Ú stata controllata per individuare virus con Avast antivirus.
https://www.avast.com/antivirus

[Andrew Premdas]

Post by Chuck van der Linden

Post by John
So I'm wondering. If Cucumber is a tool that serves a language and is not
aimed to be a testing framework, why does it has Cucumber Tables? Scenario
outlines yelling 'use parameters tests on me'.

because there are many cases where a scenario outline is the easiest way
to describe behavior as opposed to having to use 3 or 4 or more scenarios
with nearly identical steps. It's easier to follow the logic for
conditional behaviors if the differences are side by side in a table vs
spread out over multiple scenarios. a good example is something like a
button that might be enabled or disabled based on a combination of factors.

This is an interesting point. I'd say there is no need at all to use
scenario outlines and data tables when cuking. I know almost everyone does,
and that this is a somewhat controversial opinion, but there is some valid
reasoning behind it, and that alone might be of some interest.

The power of using natural language to write scenarios, is that you can
choose your level of abstraction. So no matter what the problem is that you
are describing you can always abstract upwards so that you get simpler
scenario. The art is going high enough that you get the essence of 'what'
it is you are trying do, without losing the meaning of what you are doing.
IMO most scenarios don't go high enough and are full of implementation
details (like whether a button is enabled or not). Scenario Outlines and
tables accomodate scenario writers who haven't gone high enough for
whatever reason. They put too much of the HOW in their scenarios. Outlines
and talbes makes it much easier to write scenarios, but makes the scenarios
much harder to read, implement and maintain. After many years of cuking
I've come to the conclusion that their usage is 'smelly' at best and
probably an anti-pattern.

The art of writing abstract scenarios is a difficult one. It takes alot of
thought, has very vague metrics about what a good scenario is, and is
highly context sensitive. Also happy path scenarios are often so profoundly
simple and obvious that people think they have no meaning e..g

Scenario: Sign in
When I sign in
Then I should be signed in

Alone this seems silly but in context with

Scenario: Sign in with bad password
When I sign in with a bad password
Then I should not be signed in

It starts to make more sense.

Anyhow hope thats thought provoking?

Post by Chuck van der Linden
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

--
------------------------
Andrew Premdas
blog.andrew.premdas.org

[Chuck van der Linden]

Post by Andrew Premdas

Post by Chuck van der Linden

Post by John
So I'm wondering. If Cucumber is a tool that serves a language and is
not aimed to be a testing framework, why does it has Cucumber Tables?
Scenario outlines yelling 'use parameters tests on me'.

because there are many cases where a scenario outline is the easiest way
to describe behavior as opposed to having to use 3 or 4 or more scenarios
with nearly identical steps. It's easier to follow the logic for
conditional behaviors if the differences are side by side in a table vs
spread out over multiple scenarios. a good example is something like a
button that might be enabled or disabled based on a combination of factors.

This is an interesting point. I'd say there is no need at all to use
scenario outlines and data tables when cuking. I know almost everyone does,
and that this is a somewhat controversial opinion, but there is some valid
reasoning behind it, and that alone might be of some interest.
The power of using natural language to write scenarios, is that you can
choose your level of abstraction. So no matter what the problem is that you
are describing you can always abstract upwards so that you get simpler
scenario. The art is going high enough that you get the essence of 'what'
it is you are trying do, without losing the meaning of what you are doing.
IMO most scenarios don't go high enough and are full of implementation
details (like whether a button is enabled or not). Scenario Outlines and
tables accomodate scenario writers who haven't gone high enough for
whatever reason. They put too much of the HOW in their scenarios. Outlines
and talbes makes it much easier to write scenarios, but makes the scenarios
much harder to read, implement and maintain. After many years of cuking
I've come to the conclusion that their usage is 'smelly' at best and
probably an anti-pattern.
The art of writing abstract scenarios is a difficult one. It takes alot of
thought, has very vague metrics about what a good scenario is, and is
highly context sensitive. Also happy path scenarios are often so profoundly
simple and obvious that people think they have no meaning e..g
Scenario: Sign in
When I sign in
Then I should be signed in
Alone this seems silly but in context with
Scenario: Sign in with bad password
When I sign in with a bad password
Then I should not be signed in
It starts to make more sense.
Anyhow hope thats thought provoking?

I think in the context of this discussion, we do need to consider that
'business logic' is not an implementation detail, it is rather often very
core to the behavior of the application. In my experience that logic can
get complicated, and is often best codified with a truth table or something
similar.. That's a very good use for tables in many cases

[Andrew Premdas]

Post by Chuck van der Linden

Post by Andrew Premdas

Post by Chuck van der Linden

Post by John
So I'm wondering. If Cucumber is a tool that serves a language and is
not aimed to be a testing framework, why does it has Cucumber Tables?
Scenario outlines yelling 'use parameters tests on me'.

because there are many cases where a scenario outline is the easiest way
to describe behavior as opposed to having to use 3 or 4 or more scenarios
with nearly identical steps. It's easier to follow the logic for
conditional behaviors if the differences are side by side in a table vs
spread out over multiple scenarios. a good example is something like a
button that might be enabled or disabled based on a combination of factors.

This is an interesting point. I'd say there is no need at all to use
scenario outlines and data tables when cuking. I know almost everyone does,
and that this is a somewhat controversial opinion, but there is some valid
reasoning behind it, and that alone might be of some interest.
The power of using natural language to write scenarios, is that you can
choose your level of abstraction. So no matter what the problem is that you
are describing you can always abstract upwards so that you get simpler
scenario. The art is going high enough that you get the essence of 'what'
it is you are trying do, without losing the meaning of what you are doing.
IMO most scenarios don't go high enough and are full of implementation
details (like whether a button is enabled or not). Scenario Outlines and
tables accomodate scenario writers who haven't gone high enough for
whatever reason. They put too much of the HOW in their scenarios. Outlines
and talbes makes it much easier to write scenarios, but makes the scenarios
much harder to read, implement and maintain. After many years of cuking
I've come to the conclusion that their usage is 'smelly' at best and
probably an anti-pattern.
The art of writing abstract scenarios is a difficult one. It takes alot
of thought, has very vague metrics about what a good scenario is, and is
highly context sensitive. Also happy path scenarios are often so profoundly
simple and obvious that people think they have no meaning e..g
Scenario: Sign in
When I sign in
Then I should be signed in
Alone this seems silly but in context with
Scenario: Sign in with bad password
When I sign in with a bad password
Then I should not be signed in
It starts to make more sense.
Anyhow hope thats thought provoking?

I think in the context of this discussion, we do need to consider that
'business logic' is not an implementation detail, it is rather often very
core to the behavior of the application. In my experience that logic can
get complicated, and is often best codified with a truth table or something
similar.. That's a very good use for tables in many cases

Found this in my 'drafts' hopefully its still of some value

I would argue that complex logic business logic is generally an
implementation detail. There are some things that I can find we can
generally extract from such logic, if we actually understand 'WHY?' this
logic exists. The first is the names of things, the second is are
abstrac/overall goals. In our super simple login example we can add all
sorts of 'logic' and reasoning e.g.

If a user logs in with a bad password we should not tell them that the
password is bad because this is a security risk
If a user makes 5 failed in log attempts in 15 minutes then we should
suspend their account because ...

Underlying these things are 2 overall goals

1. To ensure a user can login
2. To make sure the login process is not vulnerable to abuse

These latter two statements are very different to the first two. First of
all they are much less likely to change. Secondly they are all about 'Why'
and 'What' not 'How'. Thirdly they are abstract, perhaps even vague.


The beauty of Cucumber and scenarios is that they allow you to choose what
sort of language you use to drive your development. (Cucumber is a BDD
tool). In practice you can drive your development much more effectively
with abstract features than with precise features if certain conditions are
met. In particular you have to

1. Trust your developers (they can specify and add the precision)
2. Understand the limitations of scenarios/features/integration tests.
3. Review work properly (tests of any sort do not replace review)

Ultimately the only thing that features can do is drive your development
and give you some confidence that things are working as they were
previously when you make changes. They do not prove that something works!!
This applies no matter what language you use or however detailed your
features/scenarios are. Now using detailed language is more expensive to
implement (you end up with more code, more steps and more complexity,
tables and regex's - though this is offset in that using such language
requires less thought and insight). Detailed language is also way more
expensive to maintain and dramatically increases the cost of change.
Therefore the only reason to use it is if the detail it communicates is of
such a profound importance to the business that they are willing to
double/triple/nx the cost of implementation and maintenance to convey this
information. Finally scenarios have a high runtime cost, so the more you
have to 'prove' that something works, the greater the slowdown in your
project development.

TL/DR Use cucumber to drive development, use Unit Tests to 'prove' things
work. Write a few simple abstract scenarios rather than many concrete
complex scenarios. When implementing think about the cost of change.

All best

Andrew

Post by Chuck van der Linden
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

--
------------------------
Andrew Premdas
blog.andrew.premdas.org

[David Kowis]

Post by John
apremdas,
Thank you, this is actually a very good example to me on how a proper
BDD should look like.
David Kowis,
So I'm wondering. If Cucumber is a tool that serves a language and is
not aimed to be a testing framework, why does it has Cucumber Tables?
Scenario outlines yelling 'use parameters tests on me'.

There have been several attempts to remove Scenario Outlines from
Cucumber. They haven't quite been successful yet. I find them useful in
API testing, but that's about it. Generally I avoid them, because they
do tend to lead to a "testing" mindset instead of a "communication" mindset.

--
David Kowis

Post by John
George Dinwiddie,
Oh, I'm sorry. I was sure it was recorded for some reason, I'll have a
look on that book and the repository README file.
Chuck van der Linden,
Thank you, we already own 'The cucumber Book' - and I'll be checking the
other links you supplied.
I would like to thank anyone who contributed to the discussion. This is
(at least for me) was very educational. I think I'm starting to see the
bigger pictures here. Correct me if I'm (again) wrong, but Cucumber is
not a strict testing framework, but a tool to push your BDD develop. You
combine tests to sample your feature developing process and it helps
your identify what your missing in a cleaner way.
One your done writing your base code and the feature tests pass, it's
time to write more in-depth tests, and those probably should be written
as unit tests in a different tool. Some of you also mentioned that after
the feature is develop and green, there is not real meaning to keep
running the Cucumber tests as your no longer developing but rather
fixing bugs (unless you add a new feature ofcourse).
If it's indeed true, then a lot of the articles I was reading got
Cucumber wrong.
I'm also interesting to know how you write your cucumber tests. Do you
test things on the 'user-level' (ie, web application using selenium), or
the scope is smaller (calling your API directly)?
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

[David Kowis]

Post by John
So I was reading a lot about Cucumber, and have been using it for a
while. I have something that bothers me and I would like to hear your
opinions.
Let's say I have an app the matches two people by comparing their online
- I can create a profile
- I can view others profile
- When match is happening I get a notification
- I can check my previous matches
etc.
This part is crystal clear and does benefit from speaking behavioral.
But what if I want to test the matching system? Is very complex system,
*X, Y, Z, 1, 2, 3 *will match *X, Y, Z, 1, 2, 4*, but will not match *X,
Y, Z, 1 ,3 ,2*. Why? Because your profile is marked as 'Male', because
your age is different, because your don't have shared friends and other
stuff.
So technically, I can try to sit down and ask myself - what the above
combination means to the stakeholders? Can I break them down into
smaller parts?
This is tough thing to do. Because we write feature file retroactively -
It's require a lot of understanding in how the system work and what's
the data correlations are. Even if you do manage to do it, steps end
very verbal, or you have to add a big Cucumber table with data into it
the feature file become monstrous and not readable.
So very complex systems that are very data oriented always felt better
working on a Data Driven unit tests framework that support reading data
from external CSV, has better support in dynamic data tables etc.
So, back to the question - is BDD enough? Is it enough to use JUST
cucumber? or Cucumber is mainly an acceptance tests tool and it should
be used with more traditional unit/component/ testing tool?

Sounds like you want to do some properties based testing:

https://bitbucket.org/blob79/quickcheck

You'll want to write some scenario steps like:
Given for all types of people (male/female)
Within the age range (1-35)
... whatever is meaningful here :)

But you write your tests in terms of properties that they should
support, not the actual data, and then you use generators to produce
that data. Still fits in the cucumber realm of describing business data.

--David