[Koha-devel] Testing Idea

[Jonathan Druart]

Hello Travis and all!

Thank you for launching this very interesting subject! Indeed we have to
find a way to test scenarios.

Let me expose you my opinion about this, after having played a little bit
with both solutions.

At the first approach, Test::WWW::Mechanize looks very interesting. Easy to
use, tried and true method and approved in the perl's world.

But if you are not enough bearded, it is not for you :-)
The wiki page (
http://wiki.koha-community.org/wiki/Interface_testing_with_WWW::Mechanize)
speaks about a record script (httprecorder).
I got some troubles with it.
- I did not use it with success on my local Koha installation. The script
logs all my actions on each website except for my local instance. I don't
know why :-/
- On a remote installation, I got a very ugly interface caused by a
javascript error (opwindow is null while (opwindow.opener) blah blah). This
code seems added by the recorder. Quite strange!

Did someone else already get these issues ?
So T::W::M is easy to use, modulable, etc. but to write all the tests
ourselves (manually) will be a pain !
A big problem  is that as T::W::M does not support javascript (
http://search.cpan.org/~jesse/WWW-Mechanize-1.72/lib/WWW/Mechanize/FAQ.pod#JavaScript),
we will not be able to test all the interface features.


Selenium provides a simple browser extension (
http://release.seleniumhq.org/selenium-ide/). Recording and replaying your
scenario is very simple.
But the last Perl formatters is out to date.
So I tested the version 1.1.0 (
http://release.seleniumhq.org/selenium-ide/1.1.0/selenium-ide-1.1.0.xpi).
After doing my tests, I export them into a perl test file (using
Test::WWW::Selenium and Test::More).
The generated code could be quite verbose. It is an utopian dream to think
we will use these tests as it. To have maintainable tests we will have to
do some refactoring for a lot of actions.

To launch these tests, I have to install the selenium server (30Mo
http://selenium.googlecode.com/files/selenium-server-standalone-2.26.0.jar).
"java -jar server.jar" and I can launch correctly my tests, but
through
a firefox instance. How can I launch them without X server? It is possible
: http://www.alittlemadness.com/2008/03/05/running-selenium-headless


In addition of a test suite, as Travis already said, it would be great to
have some bug reports with a corresponding scenario in order to reproduce
easily the bug. Otherwise a test suite could be coupled with a new
enhancement explaining how to test it.

We have to decide what we do want exactly and to which purpose.

If we think that some librarians are potentially able to generate  easily
test suites (when opening a bug report) or to replay a scenario (when
testing a new enhancement), I think we have to choose one of these
solutions.

But which one ? :)

Has anyone else already used one of these projects, and has feedback to
share ?

Regards,

Jonathan


2012/11/17 Travis Elliott Davis <elliott at bywatersolutions.com>

> I have been exploring the idea of automated UI testing for awhile now and
> have come up with a few different solutions.   My top two picks currently
> are: WWW::Mechanize in conjunction with HTTP::Recorder, or the FOSS project
> Selenium.  Both ideas have merit in my opinion but my favorite is Selenium.
>
>
> Reasons Why:
>
>    1. Selenium has a FF plugin which makes recording tests cases and
>    suites easy as pie
>    2. Selenium has a new "web driver" which makes testing more dynamic
>    pages possible.  For a more detailed explanation see
>    http://seleniumhq.org/docs/03_webdriver.html
>    3. You can still perform headless testing.  However, it is slightly
>    more difficult to do with Selenium than with WWW::Mechanize.
>    4. Selenium will write Perl test scripts for you, right from the FF
>    plugin.
>
>
> I'm sure there are more reasons than the ones listed above but I would
> just like to get the conversation going.
>
> The other thing that Selenium would allow us to do is automate our bug and
> enhancement testing.  For example, when you submit a patch you may also
> submit a selenium test so that whoever is testing your patch may replay
> your test on their development environment.  Obviously there are some cases
> where this would be more helpful than others (I'm thinking of varying
> search results) but it is a start.
>
> Please let me know your thoughts.  I encourage you to visit the Selenium
> website and test drive Selenium for yourself.  Also, please note that the
> FF plugin on the FF add ons page seems to be out of date so try getting the
> XPI from the Selenium site.
>
> Best Regards,
>
>
> Elliott Davis
>
>
>
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20121123/0a53fd71/attachment.html>