Discussion:
[Cucumber] Excluding manual tests from my run
(too old to reply)
matthew thomas
2018-03-01 02:00:24 UTC
Permalink
Hi. I'm new here and new to Cucumber and coding so try and bear with me
please!

I am a few weeks in to creating a framework where we are using Cucumber to
deliver a web UI application. So far all is going well. I'm using Java with
page object classes where I call the methods in these page classes from my
step defs.

I have found that some features are just not realistic to automate so I am
tagging these with @Manual at the feature file level as we still want the
behavior recorded at feature file to give business exposure to it.

So now I am wanting to ensure that these tests don't get reported against
when we run the automation checks.

I have added the following to my @Before step which lives in my step defs:

@Before("~@Manual")

Sure enough this does stop these scenarios getting executed but the problem
is that the results in the run window show these steps as fails.

I've looked at using Serenity but I feel I'm too far in to go back and
refactor to use this approach now. We are in sprints so my time is already
stretched automating the new features.

Is there a quick and easy way to exclude manual tagged features from my
automated execution?
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Koen Prins
2018-03-01 07:02:36 UTC
Permalink
Not sure if it would work for your situation, but my suggestion would
always be to include tests in a run rather than exclude tests.

Either way, a test is passed unless it throws an exception so to make the
@Manual tests green in a run that will include all would be to simply not
implement anything, using an empty step method. Upside is green tests and
you can even include a sys.out saying "this test is to be ran manually".

Now if you think this will pollute your reporting, you can create a JUnit
runner to run those tests you wish to include using the tags, I found an
example here:
https://automationrhapsody.com/introduction-to-cucumber-and-bdd-with-examples/

Of course, you can use the runner to exclude tests as well but I feel that
inclusion is better than exclusion.
Post by matthew thomas
Hi. I'm new here and new to Cucumber and coding so try and bear with me
please!
I am a few weeks in to creating a framework where we are using Cucumber to
deliver a web UI application. So far all is going well. I'm using Java with
page object classes where I call the methods in these page classes from my
step defs.
I have found that some features are just not realistic to automate so I am
behavior recorded at feature file to give business exposure to it.
So now I am wanting to ensure that these tests don't get reported against
when we run the automation checks.
@Before("~@Manual")
Sure enough this does stop these scenarios getting executed but the
problem is that the results in the run window show these steps as fails.
I've looked at using Serenity but I feel I'm too far in to go back and
refactor to use this approach now. We are in sprints so my time is already
stretched automating the new features.
Is there a quick and easy way to exclude manual tagged features from my
automated execution?
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
matthew thomas
2018-03-16 00:58:00 UTC
Permalink
Thanks for this, Keon. I'm a bit slow catching up with this group!

This makes sense. I'm going to put some empty methods with a sys.out as you
recommend.
Post by Koen Prins
Not sure if it would work for your situation, but my suggestion would
always be to include tests in a run rather than exclude tests.
Either way, a test is passed unless it throws an exception so to make the
@Manual tests green in a run that will include all would be to simply not
implement anything, using an empty step method. Upside is green tests and
you can even include a sys.out saying "this test is to be ran manually".
Now if you think this will pollute your reporting, you can create a JUnit
runner to run those tests you wish to include using the tags, I found an
https://automationrhapsody.com/introduction-to-cucumber-and-bdd-with-examples/
Of course, you can use the runner to exclude tests as well but I feel that
inclusion is better than exclusion.
Post by matthew thomas
Hi. I'm new here and new to Cucumber and coding so try and bear with me
please!
I am a few weeks in to creating a framework where we are using Cucumber
to deliver a web UI application. So far all is going well. I'm using Java
with page object classes where I call the methods in these page classes
from my step defs.
I have found that some features are just not realistic to automate so I
the behavior recorded at feature file to give business exposure to it.
So now I am wanting to ensure that these tests don't get reported against
when we run the automation checks.
@Before("~@Manual")
Sure enough this does stop these scenarios getting executed but the
problem is that the results in the run window show these steps as fails.
I've looked at using Serenity but I feel I'm too far in to go back and
refactor to use this approach now. We are in sprints so my time is already
stretched automating the new features.
Is there a quick and easy way to exclude manual tagged features from my
automated execution?
--
Post by matthew thomas
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups
Post by matthew thomas
"Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to a topic in the
Google Groups "Cukes" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/cukes/QpafTFwNARM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
For more options, visit https://groups.google.com/d/optout.
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
George Dinwiddie
2018-03-16 13:24:30 UTC
Permalink
Matthew,
Post by matthew thomas
Hi. I'm new here and new to Cucumber and coding so try and bear with me
please!
I am a few weeks in to creating a framework where we are using Cucumber
to deliver a web UI application. So far all is going well. I'm using
Java with page object classes where I call the methods in these page
classes from my step defs.
I have found that some features are just not realistic to automate so I
the behavior recorded at feature file to give business exposure to it.
So now I am wanting to ensure that these tests don't get reported
against when we run the automation checks.
If I'm not mistaken, that says to run that Before block only for tests
not labeled @Manual. The @Manual tests will still be run, only without
the Before block.
Post by matthew thomas
Sure enough this does stop these scenarios getting executed but the
problem is that the results in the run window show these steps as fails.
I've looked at using Serenity but I feel I'm too far in to go back and
refactor to use this approach now. We are in sprints so my time is
already stretched automating the new features.
Is there a quick and easy way to exclude manual tagged features from my
automated execution?
I think what you want is use the --tags argument. If you're using Ruby
Cucumber, that's as simple as

cucumber --tags ~@Manual

(see https://github.com/cucumber/cucumber/wiki/Tags) The exact way to
pass command line arguments varies with other flavors of Cucumber.

- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------
--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...