Il 05/02/2013 17:53, Matt Wynne ha scritto:
>
> On 4 Feb 2013, at 08:21, aslak hellesoy <aslak.hellesoy-***@public.gmane.org
> <mailto:aslak.hellesoy-***@public.gmane.org>> wrote:
>
>>
>>
>>
>> On Sat, Feb 2, 2013 at 2:36 PM, Massimo Manca
>> <massimo.manca.pn-***@public.gmane.org <mailto:massimo.manca.pn-***@public.gmane.org>> wrote:
>>
>> Excuse me if I entered this discussion so late, I would like to
>> show you what in practice I am doing to test embedded projects
>> mainly in C and C++ and what I had introduced to write clear
>> requirements using Gherkin-like syntax. I know that there are
>> some constructs not present in Gherkin so considering that Aslak
>> is thinking to change a little bit under the hood Gherkin I would
>> like to know if other engineers should confirm the usefulness of
>> my approach (and also every criticism about it) and eventually
>> ask to implement in the Gherkin 3 release.
>>
>> This is an example written in Gherkin-style:
>>
>> # Rel. 0.2 MP & MM 27/11/2012
>> # Project Test
>> # file First.feature
>> # Brief descriptive text
>> #
>> Feature: Device startup
>> To be able to use the system... bla bla bla
>> and then... bla bla
>> I must... bla bla bla
>>
>> @beforeEveryScenario
>> Background:
>> Given the device is installed in the right place
>> And is connected to the power source
>> And is connected to the communication media
>>
>> @subscenario
>> @success
>> Scenario: Device startup procedure completed
>> Given User powered up the device
>> When Device started and completed self test procedure without errors
>> Then Ready.Led = YELLOW.ON
>> And Device.Status = StartUp
>>
>> @subscenario
>> @fail
>> Scenario Outline: Device startup procedure failed
>> Given Device startup procedure completed
>> When Device finished the procedure returning <error>
>> Then Device.Led.color = <color>
>> And Device.Led.status = <light>
>> And Device.Status = <status>
>> And User switches OFF the power source switch
>> And User repeats the device startup procedure or calls the technical service
>>
>> Examples: Errors
>> | error | color | light | status | comment or reference |
>> | NO POWER | - | - | - | |
>> | ROM CRC | RED | FIXED | ROMFLT | |
>> | RAM | RED | BLINK | RAMFLT | |
>> | RF | YELL. | FIXED | RFFLT | |
>> | NONE RFID | ORAN. | FIXED | RFIDFLT | |
>> #
>> # Here I place suggestions, observations and criticisms emerged by discussion with the team
>> # and with the customer and every other note should be useful to modify the scenario
>> # it is a temporary block that will be deleted after the modified implementation will be ready
>> # tested, confirmed and approved (the history is preserved by the source code version control system
>> #
>>
>>
>> @subscenario
>> @success
>> Scenario: User is known
>> Given User inserted the user's card in the slot
>> And the card was previously written with data for a normal user
>> When Device knows the user
>> Then Ready.Led = GREEN.ON
>> And Device.Status = Ready
>>
>> @subscenario
>> @fail
>> Scenario: User isn't known
>> Given User inserted the user's card in the slot
>> And the card was previously written with data for a wrong user
>> When Device doesn't know the user
>> Then Ready.Led = RED.ON
>> #
>> # Here I place suggestions, observations and criticisms emerged by discussion with the team
>> # and with the customer and every other note should be useful to modify the scenario
>> # it is a temporary block that will be deleted after the modified implementation will be ready
>> # tested, confirmed and approved (the history is preserved by the source code version control system
>> #
>>
>>
>> @mainscenario
>> @success
>> Scenario: User logon and device ready to work
>> Given Device startup procedure completed
>> And User is known
>> When Device receives the 1st data string from GPS
>> Then Device updates geographic position
>> And searches for the nearest server
>> And connects to the sensor network
>>
>> @mainscenario
>> @fail
>> Scenario: Device not ready to work
>> Given Background conditions satisfied
>> When Device startup procedure failed
>> Then Device isn't operational
>>
>>
>> @mainscenario
>> @fail
>> Scenario: Device doesn't recognize the user
>> Given Device startup procedure completed
>> When Device doesn't recognize the user
>> Then Device isn't operational
>>
>>
>> #
>> # Here I place suggestions, observations and criticisms emerged by discussion with the team
>> # and with the customer and every other note should be useful to modify the scenario
>> # it is a temporary block that will be deleted after the modified implementation will be ready
>> # tested, confirmed and approved (the history is preserved by the source code version control system
>> #
>>
>> #END OF THE EXAMPLE
>>
>>
>> This example is the result of an initial discussion where was realized only the successful @mainscenario and then it was detailed
>> better with @subscenarios to translate the initial idea to executable test steps that are sufficient to
>> test the real application.
>>
>>
>> I know that there is some skepticism about this way to write BDD style testing requirements but in my personal experience it
>> is very well suited in the embedded field to simply automate acceptance tests (also if it needs some improvements).
>>
>> As you can see it is not so simple to define a main scenario to collect every sub scenario inside.
>> The best strategy I implemented instead to implement the 3 @mainscenario is to use a Scenario Outline as this:
>>
>> @mainscenario
>> Scenario Outline: Device startup
>> Given Device startup procedure completed
>> When Device startup procedure completed returns < startError >
>> And User recognition completed returns < userError >
>> Then Device.Led.color = <color>
>> And Device.Led.status = <light>
>> And Device.Status = <status>
>>
>> Examples: Errors
>> | startError | userError | color | light | status | comment or reference |
>> | NO POWER | - | - | - | - | |
>> | ROM CRC | - | RED | FIXED | ROMFLT | |
>> | RAM | - | RED | BLINK | RAMFLT | |
>> | RF | - | YELL. | FIXED | RFFLT | |
>> | NONE RFID | - | ORAN. | FIXED | RFIDFLT | |
>> ...
>> | NONE ERROR | UNKNOWN | RED | FIXED | RFIDFLT | |
>> | NONE ERROR | KNOWN | GREEN | FIXED | READY | |
>>
>> as you can understand in this case is not possible that the system could begin to test the user if the startup failed
>> but in many other situations I found it is possible so the table grows and is not so simple to maintain it.
>>
>> So I think Gherkin could support a model where a main scenario should be the result of the composition of
>> some sub scenarios and sub scenarios should be seen as steps of it so that I could re assume my idea in this way:
>>
>> MainScenario: Device startup #the same title of the feature
>> Background
>> Step(Device startup procedure, @success, @fail) #title and a scenario list with every element comma separated
>> Step(User logon, @success, @fail)
>> After
>>
>> we could identify the scenarios with a tag next to @scenario and @subscenario or we can identify them by the title
>> assuming the same title and @success or @fail tags to identify the purpose of the scenario.
>>
>> I'm sorry, you lost me. I think I understand that you want to compose
>> scenarios, but I don't understand why and how. Your proposal sounds
>> very non-intuitive.
>>
>> Did anyone else understand this?
>
> It sounds to me like the OP wants GivenScenario to come back, or
> something like that.
It is a try to put in one MainScenario every other Scenario needed to
realize the Feature. So to test the feature is necessary to execute the
main scenario. The 1st Step means that I want to execute the "Device
startup procedure" choosing by the successful scenario (indicated with
@success) or one of the unsuccessful scenarios (in this case only one
tagged with @fail) of course every unsuccessful scenario should have a
different tag. Then the tool that reads this file should execute every
possible combination of scenarios (only 1 combination will produce the
successful scenario for the feature).
My way to write requirements using Gherkin syntax: initially I write the
successful scenario at an high level of abstraction that I ask some
questions to understand better its business value and its behavior so
normally I detail better the scenario or I decompose the scenario in
more scenarios. Then when I finished to understand the successful
behavior I start to understand what can go wrong and I add more
unsuccessful scenarios.
Hypothetically every successful scenario may have 1 or more unsuccessful
scenarios so if the feature is not very simple the application should
have to realize more then 1 successful scenario and several unsuccessful
scenarios to be tested. For these reasons I would like to have 1 main
scenario for every Feature that can "call" sequentially the others to
test every "execution path" in terms of Scenarios.
>
>> In my opinion this should simplify the translation of the scenarios to executable code and could help to decompose
>> scenarios in a more detailed scenarios that should be useful in some situation to produce an executable scenario
>> more near to the implementation to improve the testability.
>>
>>
>> Another thing that should be very useful is to change the meaning of Background to Before and After, so that should be possible
>> to have a Before that is common to every scenario in a feature (I mean a block that should be true before every scenario) and
>> a Before that should be specific for a particular scenario.
>> The same concept should be used for After.
>>
>>
>>
>>
>>
>> Il 01/02/2013 22:31, Rex Hoffman ha scritto:
>>> Aslak, Are the kinds of bugs that arise gherkin lexing bugs or
>>> cucumber-jvm execution or reporting bugs? I have no knowledge
>>> of the difficulties with lexing errors, but think I can speak a
>>> bit to cucumber reporting and execution errors....
>>>
>>> I am interested in the evolution of gherkin, but I'm not sure
>>> I'm convinced that improved stability and loose coupling
>>> couldn't be achieved with the existing gherkin implementation
>>> (At least in cucumber-jvm, ruby, js and others I, again, lack
>>> knowledge).
>>>
>>> While a cleaner implementation of the gherkin parser (and
>>> decoupling it from the reporting API's in gherkin itself) would
>>> be helpful by requiring minimal effort conversion to work with
>>> junit I don't think its necessary to clean cucumber-jvm itself.
>>> In fact, working to loosely couple cucumber-jvm to gherkin
>>> (with one quick translator layer) could make gherkin-3 trivially
>>> easy to integrate with cucumber-jvm.
>>>
>>> For instance, I'm sure we can translate all the output of the
>>> current gherkin parser to a Scenarios that that are treated as
>>> tests in junit.... with their reporter callbacks to generate
>>> whatever kind of report we wish, and in a thread safe way. I
>>> did this while testing out testng integration, before being
>>> stopped by reporting issues.
>>>
>>> I'd be happy to work on decoupling cucumber-jvm from gherkin, if
>>> you have interest in pulling this kind of work in.... but I'm
>>> not sure I see the value of re-implementing the lexer, not that
>>> I'm a fan of ragel...
>>>
>>>
>>> My ideal view of the world would be
>>>
>>> parser (maps of features/scenarios/etc...) -> interpreter to
>>> scenarios (builds scenarios) -> optional manipulators (multiplex
>>> tests, contextualize them, spring, pico, etc) -> inject
>>> listeners for individual reporting (to be aggregated/interpreted
>>> down stream) ->converter (scenarios - unit tests) --> executor
>>> (works in parallel?) --> report aggregator
>>>
>>> My 2 cents.
>>>
>>> On Friday, February 1, 2013 5:42:55 AM UTC-8, Aslak Hellesøy wrote:
>>>
>>>
>>>
>>>
>>> On Fri, Feb 1, 2013 at 1:17 PM, Andrew Premdas
>>> <apre...-***@public.gmane.org> wrote:
>>>
>>> Gherkin 3 sounds great, but the line "Reports would
>>> follow the Unit/Fragment structure instead of the
>>> Gherkin source structure" worries me a little. Could
>>> this be detrimental to Cucumber's current ability to let
>>> you know what language in your feature file is causing
>>> you problems. Cucumber's current ability to let its
>>> output follow the original Gherkin source is rather
>>> important for situations where you want to use Cucumber
>>> to document what your application is currently doing. If
>>> you could only have a more granular output, the
>>> difference between the text of the source and output
>>> would be significant, particularly for people who make
>>> extensive use of tables and example groups.
>>>
>>>
>>> Good points. Since the results will contain back-references
>>> to the original source (aka source map), a report that looks
>>> like the original source with colours and extra info can be
>>> generated simply by reading the original source and
>>> outputting it again with the results.
>>>
>>> This is something we can add later.
>>>
>>> Aslak
>>>
>>> With this in mind I think it would be good to have a
>>> requirement that says that it should be easy/feasible to
>>> construct existing reports from the new structure as
>>> well as more granular ones.
>>>
>>> All best
>>>
>>> Andrew
>>>
>>>
>>> On 31 January 2013 21:35, aslak hellesoy
>>> <aslak.h...-***@public.gmane.org> wrote:
>>>
>>> Hi all!
>>>
>>> I want to kick off Gherkin 3 - a new cross-platform
>>> Gherkin parser that will address most of the
>>> complexity and bugs in the various Cucumber
>>> implementations. I am looking for volunteers to help
>>> us implement this in C/Ruby, Java, JavaScript, C#
>>> and C/Python (for the various Cucumber
>>> implementations out there). It's a fun challenge
>>> involving parser generators (Lex/Yacc - Flex/Bison)
>>> and various programming languages.
>>>
>>> The reason I want to do this is that I have come to
>>> the realisation that if the Gherkin library was
>>> different, then a lot of the complexity and bugs in
>>> Cucumber would go away. (Cucumber's implementation
>>> of Background, Scenario Outline, Hooks, JSON reports
>>> etc is complicated, and has some nasty bugs that are
>>> very hard to fix with the current Gherkin design).
>>>
>>> I think Gherkin 3 can solve that. Over the past
>>> couple of months I have been experimenting with a
>>> new cross-platform parser design that builds an AST
>>> [1]. This AST can then be traversed by a visitor to
>>> interpret it or transform it. You can find a proof
>>> of concept of this design in the bool project [2].
>>> There is nothing novel here, except perhaps for
>>> demonstrating how this can be done in a
>>> cross-platform fashion.
>>>
>>> Gherkin 3 would generate an AST for each Feature
>>> file. This AST would then be transformed to a
>>> simpler structure that is easier for Cucumber to
>>> deal with.
>>> Background steps would be inlined, Scenario Outlines
>>> and their Examples tables would be expanded and
>>> replaced with regular Scenarios. You could think of
>>> this as a sort of compilation phase that happens in
>>> Gherkin before handing data over to Cucumber.
>>>
>>> The format passed onto Cucumber would just be a list
>>> of Units, and each Unit would consist of zero or
>>> more Fragments. Typically, a Scenario or Scenario
>>> Outline Examples row would become a Unit, and Steps
>>> would become a Fragment.
>>>
>>> Cucumber would add Fragments to each Unit for
>>> Before/After hooks and then execute the whole thing.
>>> Here is an example Unit:
>>>
>>> Unit
>>> +--Fragment (may come from a Before)
>>> +--Fragment (may come from a Given)
>>> +--Fragment (may come from a When)
>>> +--Fragment (may come from a Then)
>>> +--Fragment (may come from an After)
>>>
>>> This structure is much easier to deal with for
>>> Cucumber, as it doesn't need to know about the
>>> structure of the source. This would allow us to
>>> refactor and simplify Cucumber drastically. For
>>> Cucumber-JVM's JUnit integration, a Unit would
>>> become a JUnit test.
>>>
>>> The Gherkin grammar would be the same as now. Users
>>> wouldn't have to modify their feature files at all,
>>> but they would notice a difference in the reporting.
>>> Reports would follow the Unit/Fragment structure
>>> instead of the Gherkin source structure.
>>>
>>> Line numbers in the reports would still refer to the
>>> original line from the gherkin source, so that it's
>>> easy to trace the output back to the source.
>>>
>>> Interested in helping out? The first step would be
>>> to write a working Lex/Yacc [3] grammar for Gherkin
>>> (English only) and use a similar project structure
>>> to the bool project. I'll set up a new gherkin3
>>> github project shortly.
>>>
>>> Cheers,
>>> Aslak
>>>
>>> [1] http://en.wikipedia.org/wiki/Abstract_syntax_tree
>>> [2] https://github.com/cucumber/bool
>>> [3] Nobody uses Lex/Yacc anymore. We'll use Jison
>>> for JavaScript, Flex/Bison for C/MRI and C/Python,
>>> JFlex/Bison for JRuby/Java and GPPG for C#.
>>>
>>> --
>>> -- Rules --
>>>
>>> 1) Please prefix the subject with [Ruby], [JVM] or [JS].
>>> 2) Please use interleaved answers
>>> http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
>>> 3) If you have a question, don't reply to an
>>> existing message. Start a new topic instead.
>>>
>>> You received this message because you are subscribed
>>> to the Google Groups Cukes group. To post to this
>>> group, send email to cu...-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org To
>>> unsubscribe from this group, send email to
>>> cukes+un...-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org For more options,
>>> visit this group at
>>> https://groups.google.com/d/forum/cukes?hl=en
>>> ---
>>> You received this message because you are subscribed
>>> to the Google Groups "Cukes" group.
>>> To unsubscribe from this group and stop receiving
>>> emails from it, send an email to
>>> cukes+un...-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
>>> For more options, visit
>>> https://groups.google.com/groups/opt_out.
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> ------------------------
>>> Andrew Premdas
>>> blog.andrew.premdas.org <http://blog.andrew.premdas.org/>
>>> --
>>> -- Rules --
>>>
>>> 1) Please prefix the subject with [Ruby], [JVM] or [JS].
>>> 2) Please use interleaved answers
>>> http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
>>> 3) If you have a question, don't reply to an existing
>>> message. Start a new topic instead.
>>>
>>> You received this message because you are subscribed to
>>> the Google Groups Cukes group. To post to this group,
>>> send email to cu...-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org To unsubscribe
>>> from this group, send email to
>>> cukes+un...-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org For more options, visit
>>> this group at https://groups.google.com/d/forum/cukes?hl=en
>>> ---
>>> You received this message because you are subscribed to
>>> the Google Groups "Cukes" group.
>>> To unsubscribe from this group and stop receiving emails
>>> from it, send an email to cukes+un...-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
>>> For more options, visit
>>> https://groups.google.com/groups/opt_out.
>>>
>>>
>>>
>>>
>>> --
>>> -- Rules --
>>>
>>> 1) Please prefix the subject with [Ruby], [JVM] or [JS].
>>> 2) Please use interleaved answers
>>> http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
>>> 3) If you have a question, don't reply to an existing message.
>>> Start a new topic instead.
>>>
>>> You received this message because you are subscribed to the
>>> Google Groups Cukes group. To post to this group, send email to
>>> cukes-/***@public.gmane.org <mailto:cukes-/***@public.gmane.org>. To
>>> unsubscribe from this group, send email to
>>> cukes+unsubscribe-/***@public.gmane.org
>>> <mailto:cukes+unsubscribe-/***@public.gmane.org>. For more options,
>>> visit this group at https://groups.google.com/d/forum/cukes?hl=en
>>> ---
>>> You received this message because you are subscribed to the
>>> Google Groups "Cukes" group.
>>> To unsubscribe from this group and stop receiving emails from
>>> it, send an email to cukes+unsubscribe-/***@public.gmane.org
>>> <mailto:cukes+unsubscribe-/***@public.gmane.org>.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>>
>>
>>
>> --
>> -- Rules --
>>
>> 1) Please prefix the subject with [Ruby], [JVM] or [JS].
>> 2) Please use interleaved answers
>> http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
>> 3) If you have a question, don't reply to an existing message.
>> Start a new topic instead.
>>
>> You received this message because you are subscribed to the
>> Google Groups Cukes group. To post to this group, send email to
>> cukes-/***@public.gmane.org <mailto:cukes-/***@public.gmane.org>. To
>> unsubscribe from this group, send email to
>> cukes+unsubscribe-/***@public.gmane.org
>> <mailto:cukes%2Bunsubscribe-/***@public.gmane.org>. For more options,
>> visit this group at https://groups.google.com/d/forum/cukes?hl=en
>> ---
>> You received this message because you are subscribed to the
>> Google Groups "Cukes" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to cukes+unsubscribe-/***@public.gmane.org
>> <mailto:cukes%2Bunsubscribe-/***@public.gmane.org>.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>>
>>
>> --
>> -- Rules --
>>
>> 1) Please prefix the subject with [Ruby], [JVM] or [JS].
>> 2) Please use interleaved answers
>> http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
>> 3) If you have a question, don't reply to an existing message. Start
>> a new topic instead.
>>
>> You received this message because you are subscribed to the Google
>> Groups Cukes group. To post to this group, send email to
>> cukes-/***@public.gmane.org <mailto:cukes-/***@public.gmane.org>. To
>> unsubscribe from this group, send email to
>> cukes+unsubscribe-/***@public.gmane.org
>> <mailto:cukes+unsubscribe-/***@public.gmane.org>. For more options, visit
>> this group at https://groups.google.com/d/forum/cukes?hl=en
>> ---
>> You received this message because you are subscribed to the Google
>> Groups "Cukes" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to cukes+unsubscribe-/***@public.gmane.org
>> <mailto:cukes+unsubscribe-/***@public.gmane.org>.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
> cheers,
> Matt
>
> --
> http://mattwynne.net || https://twitter.com/mattwynne || http://the-cucumber-book.com
> || http://bddkickstart.com || http://www.relishapp.com
>
>
>
>
> --
> -- Rules --
>
> 1) Please prefix the subject with [Ruby], [JVM] or [JS].
> 2) Please use interleaved answers
> http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
> 3) If you have a question, don't reply to an existing message. Start a
> new topic instead.
>
> You received this message because you are subscribed to the Google
> Groups Cukes group. To post to this group, send email to
> cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org To unsubscribe from this group, send email to
> cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org For more options, visit this group
> at https://groups.google.com/d/forum/cukes?hl=en
> ---
> You received this message because you are subscribed to the Google
> Groups "Cukes" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
--
-- Rules --
1) Please prefix the subject with [Ruby], [JVM] or [JS].
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
3) If you have a question, don't reply to an existing message. Start a new topic instead.
You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cukes-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org To unsubscribe from this group, send email to cukes+***@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.