[Cucumber] Applying BDD / Cucumber to large batch execution scenarios

[CS Wong]

[This question was previously asked in StackOverflow but since this is the
official Cucumber community group, I thought I'd stand a better chance here]

I'm trying to apply BDD practices to my organization using Cucumber. I work
in a bank where the nightly batch job is a huge orchestrated multi-system
flow of batch jobs running and passing data between one another.

During our tests, interactive online tests probably make up only 40-50% of
test scenarios while the rest are embedded inside the batch job. As an
example, the test scenario may be:

1. Given that my savings account has a balance of $100 as of 10PM
2. When the nightly batch is run at 11PM
3. Then at 3AM after the batch run is finished, I should come back and see
that I have an additional accrued interest of $0.001.
4. And the general ledger of the bank should have an additional entry for
accrued interest of $0.001.

So as you can see, this is an extremely asynchronous scenario. If I were to
use Cucumber to trigger it, I can probably create a step definition to
insert the $100 balance into the account by 10PM, but it will not be
realistic to use Cucumber to trigger the batch to be run at 11PM as batch
jobs are usually executed by operators using their own scheduling tools
such as Control-M. And having Cucumber then wait and listen a few hours
before verifying the accrued interest, I'm not sure if I'll run into a
timeout or not.

This is just one scenario. Batch runs are very expensive for the bank and
we always tack on as many scenarios as possible to ride on a single batch
run. We also have aging scenarios where we need to run 6 months of batch
just to check whether the final interest at the end of a fixed deposit term
is correct or not (I definitely cannot make Cucumber wait and listen for
that long, can I?)

My question is, is there any example where BDD practices were applied to
large batch scenarios such as these? How would one approach this?

Please note that isolated batch scenarios in a controlled environment are
quite possible to be implemented using Cucumber and I do have it
implemented in some of my proof of concept work. But eventually, we need to
hit a test level that has an entire end-to-end environment, typically in
SIT. In this environment, it is a criteria for multiple test scenarios to
be run in parallel, none of which have complete control over the
environment. Depending on the scope of the project, this environment may
run up to 200 applications. So customer channels such as Internet Banking
will run transactional scenarios, whiles at the core banking system,
scenarios such as interest calculation, automatic transfers etc will be
executed. There will also be accounting scenarios where a general ledger
system consolidates and balances all the accounts in the environment. To do
manual testing in this environment frequently requires at least 30-50
personnel executing transactions and checking on results.

What I am trying to do is to find a way to leverage on a BDD framework to
automate test execution and capture the results so that we do not have to
manually track them all in the environment.

Thanks!
Wong

[Aslak Hellesøy]

Post by CS Wong
[This question was previously asked in StackOverflow but since this is the
official Cucumber community group, I thought I'd stand a better chance here]
I'm trying to apply BDD practices to my organization using Cucumber. I
work in a bank where the nightly batch job is a huge orchestrated
multi-system flow of batch jobs running and passing data between one
another.

I recently coached a team at a large company in the finance sector. They
have batch jobs written in Cobol running on mainframes.

We figured out an elegant pattern to test them from Cucumber. I intend to
write a longer blog post, but the process is basically:

* Given: FTP upload a file to the mainframe. The batch job will read this
file.
* When: trigger the job by uploading a "trigger" file. Another batch job
will poll for the presence of this file and trigger your batch job.
* Then: FTP download the file generated by your batch job. Compare its
contents to expected results.

HTH,
Aslak

Post by CS Wong
During our tests, interactive online tests probably make up only 40-50% of
test scenarios while the rest are embedded inside the batch job. As an
1. Given that my savings account has a balance of $100 as of 10PM
2. When the nightly batch is run at 11PM
3. Then at 3AM after the batch run is finished, I should come back and
see that I have an additional accrued interest of $0.001.
4. And the general ledger of the bank should have an additional entry for
accrued interest of $0.001.
So as you can see, this is an extremely asynchronous scenario. If I were
to use Cucumber to trigger it, I can probably create a step definition to
insert the $100 balance into the account by 10PM, but it will not be
realistic to use Cucumber to trigger the batch to be run at 11PM as batch
jobs are usually executed by operators using their own scheduling tools
such as Control-M. And having Cucumber then wait and listen a few hours
before verifying the accrued interest, I'm not sure if I'll run into a
timeout or not.
This is just one scenario. Batch runs are very expensive for the bank and
we always tack on as many scenarios as possible to ride on a single batch
run. We also have aging scenarios where we need to run 6 months of batch
just to check whether the final interest at the end of a fixed deposit term
is correct or not (I definitely cannot make Cucumber wait and listen for
that long, can I?)
My question is, is there any example where BDD practices were applied to
large batch scenarios such as these? How would one approach this?
Please note that isolated batch scenarios in a controlled environment are
quite possible to be implemented using Cucumber and I do have it
implemented in some of my proof of concept work. But eventually, we need to
hit a test level that has an entire end-to-end environment, typically in
SIT. In this environment, it is a criteria for multiple test scenarios to
be run in parallel, none of which have complete control over the
environment. Depending on the scope of the project, this environment may
run up to 200 applications. So customer channels such as Internet Banking
will run transactional scenarios, whiles at the core banking system,
scenarios such as interest calculation, automatic transfers etc will be
executed. There will also be accounting scenarios where a general ledger
system consolidates and balances all the accounts in the environment. To do
manual testing in this environment frequently requires at least 30-50
personnel executing transactions and checking on results.
What I am trying to do is to find a way to leverage on a BDD framework to
automate test execution and capture the results so that we do not have to
manually track them all in the environment.
Thanks!
Wong

--
The Cucumber logo is the intellectual property of Cucumber Ltd, a limited
company registered in Scotland, number 456793.

UK Headquarters: Cucumber Ltd, Drumsyniebeg, Lochgoilhead, Cairndow,
Argyll, PA24 8AN UK.

CONFIDENTIALITY NOTICE: The information in this e-mail is confidential and
privileged; it is intended for use solely by the individual or entity named
as the recipient hereof. Disclosure, copying, distribution, or use of the
contents of this e-mail by persons other than the intended recipient is
strictly prohibited and may violate applicable laws. If you have received
this e-mail in error, please delete the original message and notify us by
email immediately. Thank you. Cucumber Ltd.

[CS Wong]

Post by Aslak Hellesøy

Post by CS Wong
[This question was previously asked in StackOverflow but since this is
the official Cucumber community group, I thought I'd stand a better chance
here]
I'm trying to apply BDD practices to my organization using Cucumber. I
work in a bank where the nightly batch job is a huge orchestrated
multi-system flow of batch jobs running and passing data between one
another.

I recently coached a team at a large company in the finance sector. They
have batch jobs written in Cobol running on mainframes.
We figured out an elegant pattern to test them from Cucumber. I intend to
* Given: FTP upload a file to the mainframe. The batch job will read this
file.
* When: trigger the job by uploading a "trigger" file. Another batch job
will poll for the presence of this file and trigger your batch job.
* Then: FTP download the file generated by your batch job. Compare its
contents to expected results.
HTH,
Aslak

Hi Aslak, thanks for your answer. The scenario you described works very
well for isolated testing of specific MF jobs. In my case, instead of
uploading a trigger file, I would probably write a simple program to
trigger the job to run via the Control-M job scheduler on MF. But for E2E
scenarios, it's not as simple as that as part of what we are testing is an
entire flow of different jobs wired to be triggered between each other. For
me, it doesn't really make sense to write a technically-oriented scenario
saying "We will run an E2E batch job" because there are hundreds of real
business scenarios that are embedded into various jobs in the flow. If I
write scenarios according to the business examples, I will end up with very
extreme asynchronous wait times for the results to come back (example given
in earlier post below) which is not very practical.

I was thinking that perhaps a way to do this is to implement way to:


1. Use cucumber to start a scenario.
2. Run through "Given" step definitions to set up the scenario (e.g.
make sure that accounts have correct balances in place etc)
3. Pause the scenario execution at the "When" stage of the scenario and
set the status of the scenario as "Pending".
4. Exit cucumber with a certain execution ID.
5. Implement a callback mechanism where when the batch job that executed
the job completes, I will find some way to invoke cucumber to recall this
particular scenario execution using its execution ID.
6. Cucumber then runs the "Then" steps to check on results.

Would that be possible?

Thanks
Wong

Post by Aslak Hellesøy

Post by CS Wong
During our tests, interactive online tests probably make up only 40-50%
of test scenarios while the rest are embedded inside the batch job. As an
1. Given that my savings account has a balance of $100 as of 10PM
2. When the nightly batch is run at 11PM
3. Then at 3AM after the batch run is finished, I should come back and
see that I have an additional accrued interest of $0.001.
4. And the general ledger of the bank should have an additional entry
for accrued interest of $0.001.
So as you can see, this is an extremely asynchronous scenario. If I were
to use Cucumber to trigger it, I can probably create a step definition to
insert the $100 balance into the account by 10PM, but it will not be
realistic to use Cucumber to trigger the batch to be run at 11PM as batch
jobs are usually executed by operators using their own scheduling tools
such as Control-M. And having Cucumber then wait and listen a few hours
before verifying the accrued interest, I'm not sure if I'll run into a
timeout or not.
This is just one scenario. Batch runs are very expensive for the bank and
we always tack on as many scenarios as possible to ride on a single batch
run. We also have aging scenarios where we need to run 6 months of batch
just to check whether the final interest at the end of a fixed deposit term
is correct or not (I definitely cannot make Cucumber wait and listen for
that long, can I?)
My question is, is there any example where BDD practices were applied to
large batch scenarios such as these? How would one approach this?
Please note that isolated batch scenarios in a controlled environment are
quite possible to be implemented using Cucumber and I do have it
implemented in some of my proof of concept work. But eventually, we need to
hit a test level that has an entire end-to-end environment, typically in
SIT. In this environment, it is a criteria for multiple test scenarios to
be run in parallel, none of which have complete control over the
environment. Depending on the scope of the project, this environment may
run up to 200 applications. So customer channels such as Internet Banking
will run transactional scenarios, whiles at the core banking system,
scenarios such as interest calculation, automatic transfers etc will be
executed. There will also be accounting scenarios where a general ledger
system consolidates and balances all the accounts in the environment. To do
manual testing in this environment frequently requires at least 30-50
personnel executing transactions and checking on results.
What I am trying to do is to find a way to leverage on a BDD framework to
automate test execution and capture the results so that we do not have to
manually track them all in the environment.
Thanks!
Wong

[Andrew Premdas]

Post by CS Wong

Post by Aslak Hellesøy

Post by CS Wong
[This question was previously asked in StackOverflow but since this is
the official Cucumber community group, I thought I'd stand a better chance
here]
I'm trying to apply BDD practices to my organization using Cucumber. I
work in a bank where the nightly batch job is a huge orchestrated
multi-system flow of batch jobs running and passing data between one
another.

I recently coached a team at a large company in the finance sector. They
have batch jobs written in Cobol running on mainframes.
We figured out an elegant pattern to test them from Cucumber. I intend to
* Given: FTP upload a file to the mainframe. The batch job will read this
file.
* When: trigger the job by uploading a "trigger" file. Another batch job
will poll for the presence of this file and trigger your batch job.
* Then: FTP download the file generated by your batch job. Compare its
contents to expected results.
HTH,
Aslak

Hi Aslak, thanks for your answer. The scenario you described works very
well for isolated testing of specific MF jobs. In my case, instead of
uploading a trigger file, I would probably write a simple program to
trigger the job to run via the Control-M job scheduler on MF. But for E2E
scenarios, it's not as simple as that as part of what we are testing is an
entire flow of different jobs wired to be triggered between each other. For
me, it doesn't really make sense to write a technically-oriented scenario
saying "We will run an E2E batch job" because there are hundreds of real
business scenarios that are embedded into various jobs in the flow. If I
write scenarios according to the business examples, I will end up with very
extreme asynchronous wait times for the results to come back (example given
in earlier post below) which is not very practical.
1. Use cucumber to start a scenario.
2. Run through "Given" step definitions to set up the scenario (e.g.
make sure that accounts have correct balances in place etc)
3. Pause the scenario execution at the "When" stage of the scenario
and set the status of the scenario as "Pending".
4. Exit cucumber with a certain execution ID.
5. Implement a callback mechanism where when the batch job that
executed the job completes, I will find some way to invoke cucumber to
recall this particular scenario execution using its execution ID.
6. Cucumber then runs the "Then" steps to check on results.
Would that be possible?
Thanks
Wong

Generally in test environments the test runner has control and the
environment is isolated. So in the case of a batch job foo that runs once a
day, you would write a step like

Given foo has run

or

When I run foo

and implement these in such a way that the batch job runs straight away.

If you don't have that sort of control then you are using Cucumber way out
of its comfort zone, and you have to take responsibility for running things
in such a way to meet your own particular circumstances..

One way to do this might be to split your test into different scenarios and
then write a script to run each scenario e.g.

script
run cucumber upto_foo
wait_for foo
run cucumber after_foo
end

This is unwieldy and clunky, but that's because of your environment, not
cucumber

A better way would be be to set up a better environment, but you seem to be
indicating that your environment is far to complex for that.

If you are going to test a live or near live aynchornous environment with
scenarios that can run over a period of hours/days you have to script to
your particular circumstances i.e. create your own tools, if you want to
automate. As this is likely to outrageously expensive and time consuming,
you might want to consider other alternatives like

1. Manual testing
2. Testing across shorter boundaries


All best

Andrew

Post by CS Wong

Post by Aslak Hellesøy

Post by CS Wong
During our tests, interactive online tests probably make up only 40-50%
of test scenarios while the rest are embedded inside the batch job. As an
1. Given that my savings account has a balance of $100 as of 10PM
2. When the nightly batch is run at 11PM
3. Then at 3AM after the batch run is finished, I should come back and
see that I have an additional accrued interest of $0.001.
4. And the general ledger of the bank should have an additional entry
for accrued interest of $0.001.
So as you can see, this is an extremely asynchronous scenario. If I were
to use Cucumber to trigger it, I can probably create a step definition to
insert the $100 balance into the account by 10PM, but it will not be
realistic to use Cucumber to trigger the batch to be run at 11PM as batch
jobs are usually executed by operators using their own scheduling tools
such as Control-M. And having Cucumber then wait and listen a few hours
before verifying the accrued interest, I'm not sure if I'll run into a
timeout or not.
This is just one scenario. Batch runs are very expensive for the bank
and we always tack on as many scenarios as possible to ride on a single
batch run. We also have aging scenarios where we need to run 6 months of
batch just to check whether the final interest at the end of a fixed
deposit term is correct or not (I definitely cannot make Cucumber wait and
listen for that long, can I?)
My question is, is there any example where BDD practices were applied to
large batch scenarios such as these? How would one approach this?
Please note that isolated batch scenarios in a controlled environment
are quite possible to be implemented using Cucumber and I do have it
implemented in some of my proof of concept work. But eventually, we need to
hit a test level that has an entire end-to-end environment, typically in
SIT. In this environment, it is a criteria for multiple test scenarios to
be run in parallel, none of which have complete control over the
environment. Depending on the scope of the project, this environment may
run up to 200 applications. So customer channels such as Internet Banking
will run transactional scenarios, whiles at the core banking system,
scenarios such as interest calculation, automatic transfers etc will be
executed. There will also be accounting scenarios where a general ledger
system consolidates and balances all the accounts in the environment. To do
manual testing in this environment frequently requires at least 30-50
personnel executing transactions and checking on results.
What I am trying to do is to find a way to leverage on a BDD framework
to automate test execution and capture the results so that we do not have
to manually track them all in the environment.
Thanks!
Wong

--

Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups
"Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

--
------------------------
Andrew Premdas
blog.andrew.premdas.org

[Eric Kessler]

Post by Andrew Premdas

Post by CS Wong

Post by Aslak Hellesøy

Post by CS Wong
[This question was previously asked in StackOverflow but since this is
the official Cucumber community group, I thought I'd stand a better chance
here]
I'm trying to apply BDD practices to my organization using Cucumber. I
work in a bank where the nightly batch job is a huge orchestrated
multi-system flow of batch jobs running and passing data between one
another.

I recently coached a team at a large company in the finance sector. They
have batch jobs written in Cobol running on mainframes.
We figured out an elegant pattern to test them from Cucumber. I intend
* Given: FTP upload a file to the mainframe. The batch job will read
this file.
* When: trigger the job by uploading a "trigger" file. Another batch job
will poll for the presence of this file and trigger your batch job.
* Then: FTP download the file generated by your batch job. Compare its
contents to expected results.
HTH,
Aslak

Hi Aslak, thanks for your answer. The scenario you described works very
well for isolated testing of specific MF jobs. In my case, instead of
uploading a trigger file, I would probably write a simple program to
trigger the job to run via the Control-M job scheduler on MF. But for E2E
scenarios, it's not as simple as that as part of what we are testing is an
entire flow of different jobs wired to be triggered between each other. For
me, it doesn't really make sense to write a technically-oriented scenario
saying "We will run an E2E batch job" because there are hundreds of real
business scenarios that are embedded into various jobs in the flow. If I
write scenarios according to the business examples, I will end up with very
extreme asynchronous wait times for the results to come back (example given
in earlier post below) which is not very practical.
1. Use cucumber to start a scenario.
2. Run through "Given" step definitions to set up the scenario (e.g.
make sure that accounts have correct balances in place etc)
3. Pause the scenario execution at the "When" stage of the scenario
and set the status of the scenario as "Pending".
4. Exit cucumber with a certain execution ID.
5. Implement a callback mechanism where when the batch job that
executed the job completes, I will find some way to invoke cucumber to
recall this particular scenario execution using its execution ID.
6. Cucumber then runs the "Then" steps to check on results.
Would that be possible?
Thanks
Wong

Yes, it is possible. In fact...voila!

https://gist.github.com/enkessler/beb0bc2c7707ab8ae7ee23f30524df7a

Steps 1-3 are fine. Steps 4 and 5 are the ones that may need tweaking based
on your particular set of circumstances, but special exit codes for
individual tests seemed a bit narrow, so I went with a tagging system so
that all of the asynchronous tests could be run at the same time (once
before the batch cycle and once afterward). Step 6 happens normally during
the second run. While the knowledge of whether they are being run for the
first time or the second time is built into each test, the additional
tooling necessary in order to know when to actually kick off execution of
the second run is left as an exercise for the reader. ;)

Post by Andrew Premdas
Generally in test environments the test runner has control and the
environment is isolated. So in the case of a batch job foo that runs once a
day, you would write a step like
Given foo has run
or
When I run foo
and implement these in such a way that the batch job runs straight away.
If you don't have that sort of control then you are using Cucumber way out
of its comfort zone, and you have to take responsibility for running things
in such a way to meet your own particular circumstances..
One way to do this might be to split your test into different scenarios
and then write a script to run each scenario e.g.
script
run cucumber upto_foo
wait_for foo
run cucumber after_foo
end
This is unwieldy and clunky, but that's because of your environment, not
cucumber

Yep. All the usual 'you are using Cucumber wrong' arguments apply here.
However, I think that splitting the test into multiple scenarios would
further hurt its documentation/communication ability. As a single scenario,
it can remain clear that it is one cohesive (if rather temporally spread
out) behavior.

Post by Andrew Premdas
A better way would be be to set up a better environment, but you seem to
be indicating that your environment is far to complex for that.
If you are going to test a live or near live aynchornous environment with
scenarios that can run over a period of hours/days you have to script to
your particular circumstances i.e. create your own tools, if you want to
automate. As this is likely to outrageously expensive and time consuming,
you might want to consider other alternatives like
1. Manual testing
2. Testing across shorter boundaries
All best
Andrew

Post by CS Wong

Post by Aslak Hellesøy

Post by CS Wong
During our tests, interactive online tests probably make up only 40-50%
of test scenarios while the rest are embedded inside the batch job. As an
1. Given that my savings account has a balance of $100 as of 10PM
2. When the nightly batch is run at 11PM
3. Then at 3AM after the batch run is finished, I should come back and
see that I have an additional accrued interest of $0.001.
4. And the general ledger of the bank should have an additional entry
for accrued interest of $0.001.
So as you can see, this is an extremely asynchronous scenario. If I
were to use Cucumber to trigger it, I can probably create a step definition
to insert the $100 balance into the account by 10PM, but it will not be
realistic to use Cucumber to trigger the batch to be run at 11PM as batch
jobs are usually executed by operators using their own scheduling tools
such as Control-M. And having Cucumber then wait and listen a few hours
before verifying the accrued interest, I'm not sure if I'll run into a
timeout or not.
This is just one scenario. Batch runs are very expensive for the bank
and we always tack on as many scenarios as possible to ride on a single
batch run. We also have aging scenarios where we need to run 6 months of
batch just to check whether the final interest at the end of a fixed
deposit term is correct or not (I definitely cannot make Cucumber wait and
listen for that long, can I?)
My question is, is there any example where BDD practices were applied
to large batch scenarios such as these? How would one approach this?
Please note that isolated batch scenarios in a controlled environment
are quite possible to be implemented using Cucumber and I do have it
implemented in some of my proof of concept work. But eventually, we need to
hit a test level that has an entire end-to-end environment, typically in
SIT. In this environment, it is a criteria for multiple test scenarios to
be run in parallel, none of which have complete control over the
environment. Depending on the scope of the project, this environment may
run up to 200 applications. So customer channels such as Internet Banking
will run transactional scenarios, whiles at the core banking system,
scenarios such as interest calculation, automatic transfers etc will be
executed. There will also be accounting scenarios where a general ledger
system consolidates and balances all the accounts in the environment. To do
manual testing in this environment frequently requires at least 30-50
personnel executing transactions and checking on results.
What I am trying to do is to find a way to leverage on a BDD framework
to automate test execution and capture the results so that we do not have
to manually track them all in the environment.
Thanks!
Wong

--

Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

--
------------------------
Andrew Premdas
blog.andrew.premdas.org

[CS Wong]

Post by Eric Kessler

Post by Andrew Premdas

Post by CS Wong
1. Use cucumber to start a scenario.
2. Run through "Given" step definitions to set up the scenario (e.g.
make sure that accounts have correct balances in place etc)
3. Pause the scenario execution at the "When" stage of the scenario
and set the status of the scenario as "Pending".
4. Exit cucumber with a certain execution ID.
5. Implement a callback mechanism where when the batch job that
executed the job completes, I will find some way to invoke cucumber to
recall this particular scenario execution using its execution ID.
6. Cucumber then runs the "Then" steps to check on results.
Would that be possible?
Thanks
Wong

Yes, it is possible. In fact...voila!
https://gist.github.com/enkessler/beb0bc2c7707ab8ae7ee23f30524df7a
Steps 1-3 are fine. Steps 4 and 5 are the ones that may need tweaking
based on your particular set of circumstances, but special exit codes for
individual tests seemed a bit narrow, so I went with a tagging system so
that all of the asynchronous tests could be run at the same time (once
before the batch cycle and once afterward). Step 6 happens normally during
the second run. While the knowledge of whether they are being run for the
first time or the second time is built into each test, the additional
tooling necessary in order to know when to actually kick off execution of
the second run is left as an exercise for the reader. ;)

Thank you very much. I'll be trying this out over the next couple of days.

Post by Eric Kessler
Generally in test environments the test runner has control and the

Post by Andrew Premdas
environment is isolated. So in the case of a batch job foo that runs once a
day, you would write a step like
Given foo has run
or
When I run foo
and implement these in such a way that the batch job runs straight away.
If you don't have that sort of control then you are using Cucumber way
out of its comfort zone, and you have to take responsibility for running
things in such a way to meet your own particular circumstances..
One way to do this might be to split your test into different scenarios
and then write a script to run each scenario e.g.
script
run cucumber upto_foo
wait_for foo
run cucumber after_foo
end
This is unwieldy and clunky, but that's because of your environment, not
cucumber

Yep. All the usual 'you are using Cucumber wrong' arguments apply here.
However, I think that splitting the test into multiple scenarios would
further hurt its documentation/communication ability. As a single scenario,
it can remain clear that it is one cohesive (if rather temporally spread
out) behavior.

Totally agree with you on this. Ultimately, the only thing that I consider
to be sacred is that the business scenario must remain as how the business
analysts / product owners would have written it in their context.

Thanks
Wong

[Justin Radcliffe]

We have something similar to this as well.

We would like to setup 50 accounts in different statuses, then run batch
jobs that could have a few steps that could take 10-40 minutes per step.

Given an <active> account
When I set the status to <closed>
Then the status state should be set to <deactivated>


Given an <closed> account
When I set the status to <active>
Then the status state should be set to <activated>

etc.

etc.


So I can have scenarios/outlines to get all my data setup. I was thinking
we could go store all this data in an 'automation team' database table for
the account number, before status, tobe status etc. so we can validate in
the Then for the batch job here:



Given my account cancellation job is ready to run #checks
the DB to make sure we have the 50 account generated and ready to process
When the account cancellation job is completed # this
is the piece that could have 4 steps that could take 10-40 minutes
Then the accounts are set to the new status #
validates our automation table described above to the table where the batch
job updates.

Does this seem reasonable since we have to get so many different
preconditions setup? That in itself could take and hour or so to process,
so I didn't want to put that all in the precondition of the batch scenario
itself.

Thoughts?

Post by CS Wong
[This question was previously asked in StackOverflow but since this is the
official Cucumber community group, I thought I'd stand a better chance here]
I'm trying to apply BDD practices to my organization using Cucumber. I
work in a bank where the nightly batch job is a huge orchestrated
multi-system flow of batch jobs running and passing data between one
another.
During our tests, interactive online tests probably make up only 40-50% of
test scenarios while the rest are embedded inside the batch job. As an
1. Given that my savings account has a balance of $100 as of 10PM
2. When the nightly batch is run at 11PM
3. Then at 3AM after the batch run is finished, I should come back and
see that I have an additional accrued interest of $0.001.
4. And the general ledger of the bank should have an additional entry for
accrued interest of $0.001.
So as you can see, this is an extremely asynchronous scenario. If I were
to use Cucumber to trigger it, I can probably create a step definition to
insert the $100 balance into the account by 10PM, but it will not be
realistic to use Cucumber to trigger the batch to be run at 11PM as batch
jobs are usually executed by operators using their own scheduling tools
such as Control-M. And having Cucumber then wait and listen a few hours
before verifying the accrued interest, I'm not sure if I'll run into a
timeout or not.
This is just one scenario. Batch runs are very expensive for the bank and
we always tack on as many scenarios as possible to ride on a single batch
run. We also have aging scenarios where we need to run 6 months of batch
just to check whether the final interest at the end of a fixed deposit term
is correct or not (I definitely cannot make Cucumber wait and listen for
that long, can I?)
My question is, is there any example where BDD practices were applied to
large batch scenarios such as these? How would one approach this?
Please note that isolated batch scenarios in a controlled environment are
quite possible to be implemented using Cucumber and I do have it
implemented in some of my proof of concept work. But eventually, we need to
hit a test level that has an entire end-to-end environment, typically in
SIT. In this environment, it is a criteria for multiple test scenarios to
be run in parallel, none of which have complete control over the
environment. Depending on the scope of the project, this environment may
run up to 200 applications. So customer channels such as Internet Banking
will run transactional scenarios, whiles at the core banking system,
scenarios such as interest calculation, automatic transfers etc will be
executed. There will also be accounting scenarios where a general ledger
system consolidates and balances all the accounts in the environment. To do
manual testing in this environment frequently requires at least 30-50
personnel executing transactions and checking on results.
What I am trying to do is to find a way to leverage on a BDD framework to
automate test execution and capture the results so that we do not have to
manually track them all in the environment.
Thanks!
Wong