Yarn: a scenario testing tool

Follow me also on Twitter  

Those who follow me on Twitter may have read my recent tweet about Yarn, a really nice software by Lars Wirzenius I discovered the last week-end while attending the Mini-Debconf UK in Cambridge.

To make a long story short, Yarn allows the user to write scenarios of the executions of softwares. This scenario will be automatically played by Yarn which also returns the overview of the execution, with the goal to test the behaviour of these softwares.

1. Small example of how to use Yarn

Let’s have a look of how to use Yarn with a real example. The scenario will test if the software Brebis, a fully automatized backup checker, executes in an expected way. We can proceed as follow :

  1. set up the test environment
  2. launch a command with the software under test in the test environment
  3. verify the execution of the test

This is an example of a Yarn scenario file, the file is formatted using the Markdown syntax:

    SCENARIO basic brebis execution
    GIVEN setting up brebis
    AND generating backup configuration with brebis
    WHEN brebis verifies a backup
    THEN verify brebis output

    IMPLEMENTS GIVEN setting up brebis
    hg clone http://hg.brebisproject.org $DATADIR/brebis
    mkdir -p $DATADIR/brebis/yarn-test
    cp $DATADIR/brebis/functional-tests/expected-generated-list-for-tar-archive/expected-generated-list-for-tar-archive.tar.gz $DATADIR/brebis/yarn-test

    IMPLEMENTS GIVEN generating backup configuration with brebis
    $DATADIR/brebis/brebis.py -G $DATADIR/brebis/yarn-test/expected-generated-list-for-tar-archive.tar.gz

    IMPLEMENTS WHEN brebis verifies a backup
    $DATADIR/brebis/brebis.py -c $DATADIR/brebis/yarn-test -l $DATADIR/brebis/yarn-test/brebis.log

    IMPLEMENTS THEN verify brebis output
    if [ -s $DATADIR/brebis/yarn-test/brebis.log ]; then return 1; else return 0; fi

2. Result of the execution of the Yarn scenario

Yarn provides at the end of the scenario an overview of the execution, as show below:

$ yarn brebis-scenario
Scenario test suite PASS, with 1 scenarios (4 total steps), in 16.4 seconds

3. Details of the Yarn scenarios

The five first lines are basically your scenario. Each step takes a keyword and a sentence to identify a step. You define a name with the SCENARIO keyword. The next step, GIVEN, allows generally the setup of the test environment. It is possible of course to define different steps with the keywork AND.

Once the environment setup has been completed, we launch the test with the keyword WHEN. At last we check the result of the execution with THEN. It is worth noting that if you need further steps after checking the result of the execution, you can use FINALLY.

Very helpful while setting up the test environment is the variable $DATADIR, automatically initialized by Yarn. It provides the path to a temporary directory where you can store files during the execution of your scenario. This directory is by default removed at the end of the scenario.

Now the real trick: at each step I described above is associated shell commands to execute thanks to the IMPLEMENTS keyword.It’s in my opinion a great way to increase the understanding of what the scenario really does. In a short glance of the first lines, if the names of the steps are carefully chosen, you will understand what the scenario does without needing to read the actual shell code. Simple and efficient.

4. More about Yarn

Yarn is written in Python and now available in Debian in the package cmdtest. The sources are available here. In my opinion, if you were looking for this kind of tool, Yarn is in a really good direction. This project is still young but the core features do the job and the upstream is eager to receive feedbacks (and patches) :)

Some links:

Interested in Yarn? What do you think about it? Let us know in the comments of this post. 

About these ads

0 Responses to “Yarn: a scenario testing tool”



  1. Poster un commentaire

Laisser un commentaire

Entrez vos coordonnées ci-dessous ou cliquez sur une icône pour vous connecter:

Logo WordPress.com

Vous commentez à l'aide de votre compte WordPress.com. Déconnexion / Changer )

Image Twitter

Vous commentez à l'aide de votre compte Twitter. Déconnexion / Changer )

Photo Facebook

Vous commentez à l'aide de votre compte Facebook. Déconnexion / Changer )

Photo Google+

Vous commentez à l'aide de votre compte Google+. Déconnexion / Changer )

Connexion à %s




À propos de l’auteur

Carl Chenet, architecte système et expert GNU/Linux indépendant. N'hésitez pas à faire appel à mes services.
>>> Mon offre de services

Soutenir l’auteur

Soutenir les activités Debian de l'auteur Faire un don Pourquoi faire un don ?

Faire un don en bitcoins

Adresse pour les dons en bitcoins :

1BqyaQLfAXiJwvVqR3cbYz8SuzmrkFifho

Suivez-moi aussi sur Identi.ca !

Catégories


Suivre

Recevez les nouvelles publications par mail.

Joignez-vous à 29 followers

%d bloggers like this: