[Cucumber] [Cucumber, Capybara] Ordering the sequence of tagged scenarios

[James GameSparks]

Hello all,

I have multiple .features in my test suite.



*scores.featureteams.feature*
Within those .features are multiple scenarios.

*scores.feature*
Scenario: create Score
....
Scenario: edit Score
....

*teams.feature*
Scenario: create Team
....
Scenario: edit Team
....

My features are organised and grouped by areas of our system. However, I
want to add some new scenarios within a feature (for example, that create
team scores) which would rely on other features being run first (i.e.
create the teams first in *teams.feature*):

*scores.feature*
Scenario: create Score
....
Scenario: edit Score
...

*Scenario: create Team Score....*

Because I want to add scenarios that relate more to the *scores.feature*,
that's where they should belong. But I can't run them if some of the
scenarios in the *teams.feature* haven't run first. How can I do that?
Let's also say that *scores.feature* MUST run before *teams.feature*
because of other dependencies.

I tried running the features (in sequence) with tags:

*scores.feature*
*@default*
Scenario: create Score
....
*@default*
Scenario: edit Score
...
*@teams*
Scenario: create Team Score
....

*teams.feature*

*@default*Scenario: create Team
....

*@default*Scenario: edit Team
....


then running them with:

cucumber BASE_URL=https://mygame.com RESOURCES_DIR=tests/features/resources
tests/features/scores.feature tests/features/teams.feature --tags @default
@teams
or
cucumber BASE_URL=https://mygame.com RESOURCES_DIR=tests/features/resources
tests/features/scores.feature tests/features/teams.feature --tags
@default,@teams

But this will just run all the scenarios within *scores.feature* first
(including any with @teams tag) and then run *teams.feature* afterwards.
Obviously, when it gets to the "create Team Score" scenario, it fails
because it doesn't have info it needs from the *teams.feature* being run
first.

How can I make it run through the *scores.feature*, run the original
scenarios ignoring the "create Team Scores" scenario for now, then run
through the *teams.feature*, coming back to *scores.feature* and run the
new "create Team Scores" scenario, without having to run two separate
commands such as:

--tags @default --tags ~@teams

then:

--tags @teams

[Steve Tooke]

Hi James,

Answers are inline below.

Post by James GameSparks
Hello all,
I have multiple .features in my test suite.
scores.feature
teams.feature
Within those .features are multiple scenarios.
scores.feature
Scenario: create Score
....
Scenario: edit Score
....
teams.feature
Scenario: create Team
....
Scenario: edit Team
....
scores.feature
Scenario: create Score
....
Scenario: edit Score
...
Scenario: create Team Score
....
Because I want to add scenarios that relate more to the scores.feature, that's where they should belong. But I can't run them if some of the scenarios in the teams.feature haven't run first. How can I do that? Let's also say that scores.feature MUST run before teams.feature because of other dependencies.

I would suggest that you should work really hard to make your scenarios _stateless_.

You want to get to having each scenario being able to run completely on its own, independent of any other scenarios.

Each scenario should be responsible for creating everything it needs to run - it’s context.

Use the given steps to create the context that you are interested in.

So with your examples you might expect something like:

Feature: Scores
Scenario: Create Score
Given there are no scores
When I create a score "xxx"
Then there should be a score “xxx”

Scenario: Create Score
Given a score “xxx"
When I change score “xxx” to “yyy"
Then there should be a score “yyy”

Feature: Teams
# something similar to the above

Feature: Team Scores
Given a team “abc”
When I create a team score “xxx” for “abc"
Then there should be a score “xxx” for team “abc”

By making your scenarios independent and responsible for their own context you will find that the resulting tests are much less brittle because they do not have interdependencies. You will also benefit from being about to run each one independently.

Another benefit is that you can then start to vary the context, and explore the problem in more detail.

Feature: Team scores again
Given a team score “xxx” for “abc”
When I create a team score “yyy” for “abc"
Then …. what should happen now?

If you’re scenarios all create interdependent state you can’t create new scenarios without breaking all of the other scenarios that rely on a particular data model.

Post by James GameSparks
scores.feature
@default
Scenario: create Score
....
@default
Scenario: edit Score
...
@teams
Scenario: create Team Score
....
teams.feature
@default
Scenario: create Team
....
@default
Scenario: edit Team
....
or

One of the features we are adding in Cucumber 2.0 is the ability to randomise the order that your scenarios are executed. We want to do this to help people ensure that there is no order-dependency in their scenarios. It might be worth giving it a try to help flush out these scenarios.

Cheers,
Steve
--
E: ***@boxjump.co.uk
T: [phone]
http://tooky.co.uk | http://kickstartacademy.io | https://twitter.com/tooky

[aslak hellesoy]

Post by Steve Tooke
Hi James,
Answers are inline below.

Post by James GameSparks
Hello all,
I have multiple .features in my test suite.
scores.feature
teams.feature
Within those .features are multiple scenarios.
scores.feature
Scenario: create Score
....
Scenario: edit Score
....
teams.feature
Scenario: create Team
....
Scenario: edit Team
....
My features are organised and grouped by areas of our system.

However, I want to add some new scenarios within a feature (for example,
that create team scores) which would rely on other features being run first

Post by James GameSparks
scores.feature
Scenario: create Score
....
Scenario: edit Score
...
Scenario: create Team Score
....
Because I want to add scenarios that relate more to the scores.feature,

that's where they should belong. But I can't run them if some of the
scenarios in the teams.feature haven't run first. How can I do that?
Let's also say that scores.feature MUST run before teams.feature because of
other dependencies.
I would suggest that you should work really hard to make your scenarios _stateless_.
You want to get to having each scenario being able to run completely on
its own, independent of any other scenarios.
Each scenario should be responsible for creating everything it needs to
run - it’s context.
Use the given steps to create the context that you are interested in.
Feature: Scores
Scenario: Create Score
Given there are no scores
When I create a score "xxx"
Then there should be a score “xxx”
Scenario: Create Score
Given a score “xxx"
When I change score “xxx” to “yyy"
Then there should be a score “yyy”
Feature: Teams
# something similar to the above
Feature: Team Scores
Given a team “abc”
When I create a team score “xxx” for “abc"
Then there should be a score “xxx” for team “abc”
By making your scenarios independent and responsible for their own context
you will find that the resulting tests are much less brittle because they
do not have interdependencies. You will also benefit from being about to
run each one independently.
Another benefit is that you can then start to vary the context, and
explore the problem in more detail.
Feature: Team scores again
Given a team score “xxx” for “abc”
When I create a team score “yyy” for “abc"
Then 
. what should happen now?
If you’re scenarios all create interdependent state you can’t create new
scenarios without breaking all of the other scenarios that rely on a
particular data model.

Post by James GameSparks
scores.feature
@default
Scenario: create Score
....
@default
Scenario: edit Score
...
@teams
Scenario: create Team Score
....
teams.feature
@default
Scenario: create Team
....
@default
Scenario: edit Team
....
cucumber BASE_URL=https://mygame.com

RESOURCES_DIR=tests/features/resources tests/features/scores.feature

Post by James GameSparks
or
cucumber BASE_URL=https://mygame.com

RESOURCES_DIR=tests/features/resources tests/features/scores.feature

Post by James GameSparks
But this will just run all the scenarios within scores.feature first

Obviously, when it gets to the "create Team Score" scenario, it fails
because it doesn't have info it needs from the teams.feature being run
first.

Post by James GameSparks
How can I make it run through the scores.feature, run the original

scenarios ignoring the "create Team Scores" scenario for now, then run
through the teams.feature, coming back to scores.feature and run the new
"create Team Scores" scenario, without having to run two separate commands
One of the features we are adding in Cucumber 2.0 is the ability to
randomise the order that your scenarios are executed.

I guess this is still up for debate, but I personally think Cucumber Ruby
2.0 should offer the ability to *not* randomise the order.

If we want users to adopt the idea that independent scenarios is a good
thing we need to make randomised ordering the default behaviour. RSpec
already does this.

If people want a predictable order they can specify it with --plugin
alphabetic-order or something. People could also specify custom ordering
schemes, for example a scheme that would run the scenarios most likely to
fail first.

Aslak

Post by Steve Tooke
We want to do this to help people ensure that there is no order-dependency
in their scenarios. It might be worth giving it a try to help flush out
these scenarios.
Cheers,
Steve
--
T: [phone]
http://tooky.co.uk | http://kickstartacademy.io |
https://twitter.com/tooky
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Steve Tooke]

Post by Steve Tooke
Hi James,
Answers are inline below.

Post by James GameSparks
Hello all,
I have multiple .features in my test suite.
scores.feature
teams.feature
Within those .features are multiple scenarios.
scores.feature
Scenario: create Score
....
Scenario: edit Score
....
teams.feature
Scenario: create Team
....
Scenario: edit Team
....
scores.feature
Scenario: create Score
....
Scenario: edit Score
...
Scenario: create Team Score
....
Because I want to add scenarios that relate more to the scores.feature, that's where they should belong. But I can't run them if some of the scenarios in the teams.feature haven't run first. How can I do that? Let's also say that scores.feature MUST run before teams.feature because of other dependencies.

I would suggest that you should work really hard to make your scenarios _stateless_.
You want to get to having each scenario being able to run completely on its own, independent of any other scenarios.
Each scenario should be responsible for creating everything it needs to run - it’s context.
Use the given steps to create the context that you are interested in.
Feature: Scores
Scenario: Create Score
Given there are no scores
When I create a score "xxx"
Then there should be a score “xxx”
Scenario: Create Score
Given a score “xxx"
When I change score “xxx” to “yyy"
Then there should be a score “yyy”
Feature: Teams
# something similar to the above
Feature: Team Scores
Given a team “abc”
When I create a team score “xxx” for “abc"
Then there should be a score “xxx” for team “abc”
By making your scenarios independent and responsible for their own context you will find that the resulting tests are much less brittle because they do not have interdependencies. You will also benefit from being about to run each one independently.
Another benefit is that you can then start to vary the context, and explore the problem in more detail.
Feature: Team scores again
Given a team score “xxx” for “abc”
When I create a team score “yyy” for “abc"
Then 
. what should happen now?
If you’re scenarios all create interdependent state you can’t create new scenarios without breaking all of the other scenarios that rely on a particular data model.

Post by James GameSparks
scores.feature
@default
Scenario: create Score
....
@default
Scenario: edit Score
...
@teams
Scenario: create Team Score
....
teams.feature
@default
Scenario: create Team
....
@default
Scenario: edit Team
....
or

One of the features we are adding in Cucumber 2.0 is the ability to randomise the order that your scenarios are executed.
I guess this is still up for debate, but I personally think Cucumber Ruby 2.0 should offer the ability to *not* randomise the order.
If we want users to adopt the idea that independent scenarios is a good thing we need to make randomised ordering the default behaviour. RSpec already does this.

Agreed - but I think we should wait for Cucumber-Ruby 2.1 before making it the default. We are trying really hard to make 2.0 a drop in replacement and I guess(?) there are enough order-dependent cucumber suites out there that it could be a surprise.

Steve

Post by Steve Tooke
If people want a predictable order they can specify it with --plugin alphabetic-order or something. People could also specify custom ordering schemes, for example a scheme that would run the scenarios most likely to fail first.
Aslak
We want to do this to help people ensure that there is no order-dependency in their scenarios. It might be worth giving it a try to help flush out these scenarios.

[Rob Park]

Post by aslak hellesoy

Post by Steve Tooke
Hi James,
Answers are inline below.

Post by James GameSparks
Hello all,
I have multiple .features in my test suite.
scores.feature
teams.feature
Within those .features are multiple scenarios.
scores.feature
Scenario: create Score
....
Scenario: edit Score
....
teams.feature
Scenario: create Team
....
Scenario: edit Team
....
My features are organised and grouped by areas of our system.

However, I want to add some new scenarios within a feature (for example,
that create team scores) which would rely on other features being run first

Post by James GameSparks
scores.feature
Scenario: create Score
....
Scenario: edit Score
...
Scenario: create Team Score
....
Because I want to add scenarios that relate more to the scores.feature,

that's where they should belong. But I can't run them if some of the
scenarios in the teams.feature haven't run first. How can I do that?
Let's also say that scores.feature MUST run before teams.feature because of
other dependencies.
I would suggest that you should work really hard to make your scenarios _stateless_.
You want to get to having each scenario being able to run completely on
its own, independent of any other scenarios.
Each scenario should be responsible for creating everything it needs to
run - it’s context.
Use the given steps to create the context that you are interested in.
Feature: Scores
Scenario: Create Score
Given there are no scores
When I create a score "xxx"
Then there should be a score “xxx”
Scenario: Create Score
Given a score “xxx"
When I change score “xxx” to “yyy"
Then there should be a score “yyy”
Feature: Teams
# something similar to the above
Feature: Team Scores
Given a team “abc”
When I create a team score “xxx” for “abc"
Then there should be a score “xxx” for team “abc”
By making your scenarios independent and responsible for their own
context you will find that the resulting tests are much less brittle
because they do not have interdependencies. You will also benefit from
being about to run each one independently.
Another benefit is that you can then start to vary the context, and
explore the problem in more detail.
Feature: Team scores again
Given a team score “xxx” for “abc”
When I create a team score “yyy” for “abc"
Then 
. what should happen now?
If you’re scenarios all create interdependent state you can’t create new
scenarios without breaking all of the other scenarios that rely on a
particular data model.

Post by James GameSparks
scores.feature
@default
Scenario: create Score
....
@default
Scenario: edit Score
...
@teams
Scenario: create Team Score
....
teams.feature
@default
Scenario: create Team
....
@default
Scenario: edit Team
....
cucumber BASE_URL=https://mygame.com

RESOURCES_DIR=tests/features/resources tests/features/scores.feature

Post by James GameSparks
or
cucumber BASE_URL=https://mygame.com

RESOURCES_DIR=tests/features/resources tests/features/scores.feature

Post by James GameSparks
But this will just run all the scenarios within scores.feature first

Obviously, when it gets to the "create Team Score" scenario, it fails
because it doesn't have info it needs from the teams.feature being run
first.

Post by James GameSparks
How can I make it run through the scores.feature, run the original

scenarios ignoring the "create Team Scores" scenario for now, then run
through the teams.feature, coming back to scores.feature and run the new
"create Team Scores" scenario, without having to run two separate commands
One of the features we are adding in Cucumber 2.0 is the ability to
randomise the order that your scenarios are executed.

I guess this is still up for debate, but I personally think Cucumber Ruby
2.0 should offer the ability to *not* randomise the order.
If we want users to adopt the idea that independent scenarios is a good
thing we need to make randomised ordering the default behaviour. RSpec
already does this.
Agreed - but I think we should wait for Cucumber-Ruby 2.1 before making it
the default. We are trying really hard to make 2.0 a drop in replacement
and I guess(?) there are enough order-dependent cucumber suites out there
that it could be a surprise.
Steve

2 cents:

Wouldn't 2.0 be a more appropriate place from a semver perspective to
change defaults?
And if there's a switch to make it possible that feels similar to rspec
changing it's 3.0 formatter with an allowance for rspec-legacy_formatters.

@robpark

Post by aslak hellesoy
If people want a predictable order they can specify it with --plugin
alphabetic-order or something. People could also specify custom ordering
schemes, for example a scheme that would run the scenarios most likely to
fail first.
Aslak

Post by Steve Tooke
We want to do this to help people ensure that there is no
order-dependency in their scenarios. It might be worth giving it a try to
help flush out these scenarios.

[Steve Tooke]

Post by Steve Tooke

Post by Steve Tooke
One of the features we are adding in Cucumber 2.0 is the ability to randomise the order that your scenarios are executed.
I guess this is still up for debate, but I personally think Cucumber Ruby 2.0 should offer the ability to *not* randomise the order.
If we want users to adopt the idea that independent scenarios is a good thing we need to make randomised ordering the default behaviour. RSpec already does this.

Agreed - but I think we should wait for Cucumber-Ruby 2.1 before making it the default. We are trying really hard to make 2.0 a drop in replacement and I guess(?) there are enough order-dependent cucumber suites out there that it could be a surprise.
Wouldn't 2.0 be a more appropriate place from a semver perspective to change defaults?
And if there's a switch to make it possible that feels similar to rspec changing it's 3.0 formatter with an allowance for rspec-legacy_formatters.

Normally I would agree, but there are a couple of problems with doing that here.

We want to give people a deprecation warning if they are running specs with no options that default ordering will be changing. With that warning we will need to let them know how they can use the new ordering and check if their features will run with the new. If it doesn’t we will need to let them know how to configure things to keep the current default ordering.

Unfortunately we have no way of providing random ordering on the 1.3.x versions of cucumber.

So I think we should introduce the warning in 2.0, and wait until 2.1 to change the ordering. We could wait until 3.0 to make the change? But is this a breaking change, or an update to a default?

Steve

[Rob Park]

Post by Steve Tooke

Post by aslak hellesoy

Post by Steve Tooke
One of the features we are adding in Cucumber 2.0 is the ability to
randomise the order that your scenarios are executed.

I guess this is still up for debate, but I personally think Cucumber Ruby
2.0 should offer the ability to *not* randomise the order.
If we want users to adopt the idea that independent scenarios is a good
thing we need to make randomised ordering the default behaviour. RSpec
already does this.
Agreed - but I think we should wait for Cucumber-Ruby 2.1 before making
it the default. We are trying really hard to make 2.0 a drop in replacement
and I guess(?) there are enough order-dependent cucumber suites out there
that it could be a surprise.

Wouldn't 2.0 be a more appropriate place from a semver perspective to change defaults?
And if there's a switch to make it possible that feels similar to rspec
changing it's 3.0 formatter with an allowance for rspec-legacy_formatters.
Normally I would agree, but there are a couple of problems with doing that here.
We want to give people a deprecation warning if they are running specs
with no options that default ordering will be changing. With that warning
we will need to let them know how they can use the new ordering and check
if their features will run with the new. If it doesn’t we will need to let
them know how to configure things to keep the current default ordering.

That's a really good point.

Post by Steve Tooke
Unfortunately we have no way of providing random ordering on the 1.3.x
versions of cucumber.
So I think we should introduce the warning in 2.0, and wait until 2.1 to
change the ordering. We could wait until 3.0 to make the change? But is
this a breaking change, or an update to a default?

It seems it comes down to your interpretation of "new, backwards compatible
functionality".
And yeah, I see your point that changing the default is still backwards
compatible as long as providing the extra option keeps it working as before.

Post by Steve Tooke
Steve
Thanks. Makes sense to me.

@rob

[byrnejb]

Post by Steve Tooke
Agreed - but I think we should wait for Cucumber-Ruby 2.1 before making it
the default. We are trying really hard to make 2.0 a drop in replacement
and I guess(?) there are enough order-dependent cucumber suites out there
that it could be a surprise.
Steve

Would it not make more sense, from a semantic version point of view, to
release the drop-in replacement, with deprecation, as the last release of
1.X .Y series? And then move right on 2.0.0.

[James GameSparks]

According to the documentation here:
https://github.com/cucumber/cucumber/wiki/Tags

@billing

Feature: Verify billing
@important
Scenario: Missing product description
Scenario: Several products

But this isn't representative of how it really works. If my feature looks
like this:

@main
Feature: Totals

Scenario: Create Total
...

@teams
Scenario: Create Team Total
...

and I run:
--tags @main --tags @teams

It runs BOTH scenarios, not just the LAST one.

So what is the correct behavior? Are the docs wrong?

Hello all,
I have multiple .features in my test suite.
*scores.featureteams.feature*
Within those .features are multiple scenarios.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
....
*teams.feature*
Scenario: create Team
....
Scenario: edit Team
....
My features are organised and grouped by areas of our system. However,
I want to add some new scenarios within a feature (for example, that create
team scores) which would rely on other features being run first (i.e.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
...
*Scenario: create Team Score....*
Because I want to add scenarios that relate more to the *scores.feature*,
that's where they should belong. But I can't run them if some of the
scenarios in the *teams.feature* haven't run first. How can I do that?
Let's also say that *scores.feature* MUST run before *teams.feature*
because of other dependencies.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
...
Scenario: create Team Score
....
*teams.feature*
....
....
cucumber BASE_URL=https://mygame.com
RESOURCES_DIR=tests/features/resources tests/features/scores.feature
or
cucumber BASE_URL=https://mygame.com
RESOURCES_DIR=tests/features/resources tests/features/scores.feature
But this will just run all the scenarios within *scores.feature* first
Obviously, when it gets to the "create Team Score" scenario, it fails
because it doesn't have info it needs from the *teams.feature* being run
first.
How can I make it run through the *scores.feature*, run the original
scenarios ignoring the "create Team Scores" scenario for now, then run
through the *teams.feature*, coming back to *scores.feature* and run the
new "create Team Scores" scenario, without having to run two separate

[aslak hellesoy]

On Mon, Dec 15, 2014 at 3:47 PM, James GameSparks <

Post by James GameSparks
https://github.com/cucumber/cucumber/wiki/Tags
@billing

Feature: Verify billing
@important
Scenario: Missing product description
Scenario: Several products

But this isn't representative of how it really works. If my feature looks
@main
Feature: Totals
Scenario: Create Total
...
@teams
Scenario: Create Team Total
...
It runs BOTH scenarios, not just the LAST one.

Works for me:

cucumber --tags @main --tags @teams
https://gist.githubusercontent.com/aslakhellesoy/aa9a87381d129eb29902/raw/248c24d9da10e363d36091523a6a2b89621255e7/tags.feature

Aslak

Post by James GameSparks
So what is the correct behavior? Are the docs wrong?

Hello all,
I have multiple .features in my test suite.
*scores.featureteams.feature*
Within those .features are multiple scenarios.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
....
*teams.feature*
Scenario: create Team
....
Scenario: edit Team
....
My features are organised and grouped by areas of our system. However,
I want to add some new scenarios within a feature (for example, that create
team scores) which would rely on other features being run first (i.e.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
...
*Scenario: create Team Score....*
Because I want to add scenarios that relate more to the *scores.feature*,
that's where they should belong. But I can't run them if some of the
scenarios in the *teams.feature* haven't run first. How can I do that?
Let's also say that *scores.feature* MUST run before *teams.feature*
because of other dependencies.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
...
Scenario: create Team Score
....
*teams.feature*
....
....
cucumber BASE_URL=https://mygame.com RESOURCES_DIR=tests/features/resources
@teams
or
cucumber BASE_URL=https://mygame.com RESOURCES_DIR=tests/features/resources
tests/features/scores.feature tests/features/teams.feature --tags
@default,@teams
But this will just run all the scenarios within *scores.feature* first
afterwards. Obviously, when it gets to the "create Team Score" scenario,
it fails because it doesn't have info it needs from the *teams.feature*
being run first.
How can I make it run through the *scores.feature*, run the original
scenarios ignoring the "create Team Scores" scenario for now, then run
through the *teams.feature*, coming back to *scores.feature* and run the
new "create Team Scores" scenario, without having to run two separate
--

Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[James GameSparks]

According to the documentation here:
https://github.com/cucumber/cucumber/wiki/Tags

@billing

Feature: Verify billing
@important
Scenario: Missing product description
Scenario: Several products

But this isn't representative of how it really works. If my feature looks
like this:

@main
Feature: Totals

Scenario: Create Total
...

@teams
Scenario: Create Team Total
...

and I run:
--tags @default --tags @teams

It runs BOTH scenarios, not just the LAST one.

So what is the correct behavior? Are the docs wrong?

[James GameSparks]

I want to run a set of features excluding tagged scenarios (~@teams) within
those features.
cucumber BASE_URL=https://mygame.com scores.feature teams.feature
results.feature --tags @default ~@teams

Once they are completed, I want to run some of those features again but to
exclusively run the tagged scenarios (@teams) to generate the team scores:
cucumber BASE_URL=https://mygame.com scores.feature teams.feature
results.feature --tags @teams

However, I can't do this within a single command. It must be within a
single command. I tried:

cucumber BASE_URL=https://mygame.com scores.feature teams.feature
results.feature --tags @default ~@teams --tags @teams
and
cucumber BASE_URL=https://mygame.com scores.feature teams.feature
results.feature --tags @default ~@teams scores.feature teams.feature
results.feature --tags @teams
That doesn't work.

I tried using profiles in my cucumber.yml, the first command line for one
profile (preTeams) and the second command line in another (postTeams) and
calling both profiles, one after the other.
cucumber BASE_URL=https://mygame.com -p resources -p preTeams -p postTeams
-p json_report

But still, I run into the same problem. The tests never run:
Using the resources, preTeams, postTeams and json_report profiles...
0 scenarios
0 steps
0m0.000s

How can I run those features, excluding certain tags, then run the same
tests only including those tags, all within the same command?

Post by James GameSparks
Hello all,
I have multiple .features in my test suite.
*scores.featureteams.feature*
Within those .features are multiple scenarios.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
....
*teams.feature*
Scenario: create Team
....
Scenario: edit Team
....
My features are organised and grouped by areas of our system. However,
I want to add some new scenarios within a feature (for example, that create
team scores) which would rely on other features being run first (i.e.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
...
*Scenario: create Team Score....*
Because I want to add scenarios that relate more to the *scores.feature*,
that's where they should belong. But I can't run them if some of the
scenarios in the *teams.feature* haven't run first. How can I do that?
Let's also say that *scores.feature* MUST run before *teams.feature*
because of other dependencies.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
...
Scenario: create Team Score
....
*teams.feature*
....
....
cucumber BASE_URL=https://mygame.com
RESOURCES_DIR=tests/features/resources tests/features/scores.feature
or
cucumber BASE_URL=https://mygame.com
RESOURCES_DIR=tests/features/resources tests/features/scores.feature
But this will just run all the scenarios within *scores.feature* first
Obviously, when it gets to the "create Team Score" scenario, it fails
because it doesn't have info it needs from the *teams.feature* being run
first.
How can I make it run through the *scores.feature*, run the original
scenarios ignoring the "create Team Scores" scenario for now, then run
through the *teams.feature*, coming back to *scores.feature* and run the
new "create Team Scores" scenario, without having to run two separate

['James B. Byrne' via Cukes]

Post by James GameSparks
How can I run those features, excluding certain tags, then run the
same tests only including those tags, all within the same command?

At some point the cucumber API changed so that all tag flags are
ANDed. That means --tags @teams ~@teams will return nothing for the
intersection of 'is X' and 'is not X' is null; as you have discovered
empirically. It not clear from your message why you deem it necessary
to do so. What are you trying to accomplish? Why will two successive
runs of cucumber with different tag flags checked not produce the
results you seek?
--
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:***@Harte-Lyne.ca
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: [phone]
Hamilton, Ontario fax: [phone]
Canada L8E 3C3

[James GameSparks]

Because the way my Features are structured (based on areas of the system,
not by use cases), some scenarios within my features require some
information (e.g. team data) to have been set up, which is done later in
the process. The scenarios that are related to this information are
skipped and then run again at a later stage. I know it is not the standard
when it comes to creating Cucumber scenarios, but this is what is in place.

Secondly, I can't run more than one command because I need to output the
whole test results into one .json file at the end. The -out parameter does
not allow me to append the test results to the end of an existing file.
Running separate commands will either have the results in separate output
files or the test results in the .json file overwritten by the following
tests executed by the next command if using the same output filename.

Post by James GameSparks
Hello all,
I have multiple .features in my test suite.
*scores.featureteams.feature*
Within those .features are multiple scenarios.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
....
*teams.feature*
Scenario: create Team
....
Scenario: edit Team
....
My features are organised and grouped by areas of our system. However,
I want to add some new scenarios within a feature (for example, that create
team scores) which would rely on other features being run first (i.e.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
...
*Scenario: create Team Score....*
Because I want to add scenarios that relate more to the *scores.feature*,
that's where they should belong. But I can't run them if some of the
scenarios in the *teams.feature* haven't run first. How can I do that?
Let's also say that *scores.feature* MUST run before *teams.feature*
because of other dependencies.
*scores.feature*
Scenario: create Score
....
Scenario: edit Score
...
Scenario: create Team Score
....
*teams.feature*
....
....
cucumber BASE_URL=https://mygame.com
RESOURCES_DIR=tests/features/resources tests/features/scores.feature
or
cucumber BASE_URL=https://mygame.com
RESOURCES_DIR=tests/features/resources tests/features/scores.feature
But this will just run all the scenarios within *scores.feature* first
Obviously, when it gets to the "create Team Score" scenario, it fails
because it doesn't have info it needs from the *teams.feature* being run
first.
How can I make it run through the *scores.feature*, run the original
scenarios ignoring the "create Team Scores" scenario for now, then run
through the *teams.feature*, coming back to *scores.feature* and run the
new "create Team Scores" scenario, without having to run two separate

['James B. Byrne' via Cukes]

Post by James GameSparks
Because the way my Features are structured (based on areas of the system,
not by use cases), some scenarios within my features require some
information (e.g. team data) to have been set up, which is done later in
the process. The scenarios that are related to this information are
skipped and then run again at a later stage. I know it is not the standard
when it comes to creating Cucumber scenarios, but this is what is in place.

This practice is called scenario coupling and it is gravely deprecated
by the Cucumber community. I believe that you will find it more
useful to construct some sort of setup and tear-down step-definitions
that you can call from in the Given section of the relevant scenarios
or the Before section of the feature file.

Post by James GameSparks
Secondly, I can't run more than one command because I need to
output the whole test results into one .json file at the end.
The -out parameter does not allow me to append the test results
to the end of an existing file.

But, you do not need the --out parameter at all. Use the '>' redirect
on the first run to create the file and the '>>' on the second to
append.

cucumber --tags @X --format=json feature.feature > output.json
cucumber --tags ~@X --format=json feature.feature >> output.json
--
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:***@Harte-Lyne.ca
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: [phone]
Hamilton, Ontario fax: [phone]
Canada L8E 3C3

['James B. Byrne' via Cukes]

Post by 'James B. Byrne' via Cukes
This practice is called scenario coupling and it is gravely deprecated
by the Cucumber community.

If you absolutely MUST do this and you need to run cucumber multiple
times against aggregated result sets then take a look in
./features/support/env.rb and read up about:

DatabaseCleaner.strategy = :truncation, {:except => %w[tbl-x tbl-y]}

But, if you do continue down this road you will reach a point where
your project testing with cucumber will drop off a cliff.
--
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:***@Harte-Lyne.ca
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: [phone]
Hamilton, Ontario fax: [phone]
Canada L8E 3C3

[James GameSparks]

I totally understand your points of view here.
Ultimately, I decided to split the features. It seems the best option for
now until we ever decide to base them around individual use cases.

So I've gone for *scores_preTeams.feature* and *scores_postTeams.feature*
for example.

Post by 'James B. Byrne' via Cukes

Post by 'James B. Byrne' via Cukes
This practice is called scenario coupling and it is gravely deprecated
by the Cucumber community.

If you absolutely MUST do this and you need to run cucumber multiple
times against aggregated result sets then take a look in
DatabaseCleaner.strategy = :truncation, {:except => %w[tbl-x tbl-y]}
But, if you do continue down this road you will reach a point where
your project testing with cucumber will drop off a cliff.
--
*** E-Mail is NOT a SECURE channel ***
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: [phone]
Hamilton, Ontario fax: [phone]
Canada L8E 3C3