[Bug 13849] New: Introduce acceptance tests with cucumber
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 Bug ID: 13849 Summary: Introduce acceptance tests with cucumber Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: ASSIGNED Severity: enhancement Priority: P5 - low Component: Test Suite Assignee: jonathan.druart@biblibre.com Reporter: jonathan.druart@biblibre.com QA Contact: gmcharlt@gmail.com During the hackfest 15 in Marseille, akafred present us how he integrated acceptance tests in Koha using Cucumber. I would like to get developer's opinions on the need of this kind of tests into Koha. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 --- Comment #1 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 36940 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=36940&action=edit Bug 13849: POC - my first Cucumber contact During the hackfest 15 in Marseille, akafred present us how he integrated acceptance tests in Koha using Cucumber. With this patch, I would like to get other developer opinions on the usefulness of this kind of tests for the Koha software. To test: 0/ * Install the selenium server (have a look at bug 13691) * Install libtest-bdd-cucumber-perl (pkg) and App::pherkin (cpan) * Edit t/db_dependent/cucumber/features/step_definitions/basic_steps.pl and fill the login and password of a superlibrarian user. Maybe you should also change the address of the selenium server (search for 'remote_server_addr'). 1/ Start the selenium server and launch the tests: $ cd t/db_dependent/cucumber $ pherkin or $ prove tests.t Even if the second method is recommended in Test::BDD::Cucumber::Manual::Integration, I prefer the first one Indeed, the tests will pass if a step is not defined (maybe something is wrong somewhere...). -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |In Discussion See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=13691 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |olli-antti.kivilahti@jns.fi --- Comment #2 from Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi> --- Hi there! Awesome that somebody else is in flames about Cucumber! I have been writing the Cucumber tests into another directory structure, and I have encountered some issues with it. My file hierarchy: kohaclone/t/Cucumber/runCucumberTests.pl #Binds *::Cucumber::Manual::Integration #And runs all Cucumber tests. kohaclone/t/Cucumber/Overdues/features/*.feature #Feature definitions kohaclone/t/Cucumber/Overdues/features/step_definitions/*_steps.pl #Step handlers kohaclone/t/Cucumber/TestObjects/Koha/Overdues/*.pm #Packages to actually #implement the test code. I cannot set debugger breakpoints #into the step-files, because of how the Test::BDD-modules #load them as code closures, so I implement the business logic #in the TestObjects. Return the comparable values and run the #Test::More function on the returned values, eg two objects or #an error code. Note regarding passing test data arrays in the Scenario description: Examples: | branchCode | borrowerCategory | letterNumber | | | S | 1 | | | S | 2 | | | S | 3 | | | PT | 1 | | | PT | 2 | | | PT | 3 | | CPL | PT | 1 | | CPL | PT | 2 | | FTL | YA | 1 | | FTL | YA | 2 | | FTL | YA | 3 | Instead of getting the array values from the step-definition regexp, you can get them from the StepContext-object: See my attachment scenarioNumberForTableDriveScenarios.txt -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 --- Comment #3 from Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi> --- Created attachment 36942 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=36942&action=edit scenarioNumberForTableDrivenScenarios. Keep track of the Scenario data array location to get data as HASH instead of ARRAY. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #36942|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 --- Comment #4 from Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi> --- Created attachment 37622 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=37622&action=edit Example Cucumber tests for the overdues rewrite Having written a new overdue gathering and sending module and using Cucumber tests, here are the test features I used. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 --- Comment #5 from Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi> --- Created attachment 37678 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=37678&action=edit Trigger script for Cucumber tests suite -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 --- Comment #6 from Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi> --- Some tutorial on how to do Cukes in Koha! http://pastebin.com/3Tya8kwR -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 Julian FIOL <julian.fiol@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |julian.fiol@biblibre.com --- Comment #7 from Julian FIOL <julian.fiol@biblibre.com> --- To test: 0/ * Install the selenium server (have a look at bug 13691) * Install libtest-bdd-cucumber-perl (pkg) and App::pherkin (cpan) * Edit t/db_dependent/cucumber/features/step_definitions/basic_steps.pl and fill the login and password of a superlibrarian user. Maybe you should also change the address of the selenium server (search for 'remote_server_addr'). 1/ Start the selenium server and launch the tests: $ cd t/db_dependent/cucumber $ pherkin or $ prove tests.t Hi Jonathan, I tried your patch using pherkin and prove, and I have the same issue at both times : Create a Category is OK [...] But create a patron crash after submit : Create a patron As a librarian I want to create a new patron Scenario: Create a patron Given I am logged in as a library admin Given I have gone to the new patron page When I have filled the field surname with "test_patron_surname" And I have filled the field cardnumber with "4242424242" And I have filled the field userid with "test_username" And I have filled the field password with "password" And I have filled the field password2 with "password" And I have clicked on the submit button Then The patron "test_username" should have been created step defined at features/1-patron.feature line 22. ok 1 - Starting to execute step: The patron "test_username" should have been created not ok 2 - The patron test_username should exist # Failed test 'The patron test_username should exist' # at features/step_definitions/1-patron_steps.pl line 12. # got: '0' # expected: '1' 1..2 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 --- Comment #8 from Jonathan Druart <jonathan.druart@biblibre.com> --- All works perfectly here. Make sure you have a superlibrarian user "koha" with pwd "koha" or edit features/step_definitions/basic_steps.pl -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 --- Comment #9 from Julian FIOL <julian.fiol@biblibre.com> --- Works fine now My problem was the 'BorrowerMandatoryField' syspref which was waiting more fields filled than tests did. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 --- Comment #10 from Julian FIOL <julian.fiol@biblibre.com> --- Created attachment 39983 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=39983&action=edit Bug 13849 : Adding Cucumber tests to create a new biblio To test: 0/ * Install the selenium server (have a look at bug 13691) * Install libtest-bdd-cucumber-perl (pkg) and App::pherkin (cpan) * Edit t/db_dependent/cucumber/features/step_definitions/basic_steps.pl and fill the login and password of a superlibrarian user. Maybe you should also change the address of the selenium server (search for 'remote_server_addr'). 1/ Start the selenium server and launch the tests: $ cd t/db_dependent/cucumber $ pherkin or $ prove tests.t This patch add a new scenario creating a new biblio -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13849 --- Comment #11 from Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi> --- Created attachment 40070 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=40070&action=edit Cucumber test suite Vaara-style I discussed with Oslo about how to best organize these Cucumber tests. They hinted that we should use Page object pattern to do these Cucumber/Selenium tests. Which is a jolly good idea! I haven't had time to do any of those Page Object Pattern-driven objects for hiding ugly implementation specifics for web-crawling. I also have some new features in production utilizing Cucumber tests and I started to encounter some test structuring issues. So I cooked up a better way to organize the test components. See testsStructuring.tutorial in the .tar.gz This is 100% compatible with the runCucumberTests.t-script and IMHO makes for a much more beautiful code structuring than the way of doing things in the Perl documentation for Cucumber. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org