Presentation

This blog is developed by the testing team of the Alarcos Research Group (Institute of Technologies and Information Systems, Instituto de Tecnologías y Sistemas de Información, Universidad de Castilla-La Mancha, Spain). In the testing team, we mainly work on the automation of the software testing process. We deal with:

-Mutation testing.

-Model-based testing.

-Testing in Software Product Lines.

-Testing of Information Systems.

-Combinatorial testing.

Wednesday, January 8, 2014

Friday, October 5, 2012

Some recent publications

In the last weeks, the Testing team has got some successes disseminating its work:

  1. Pérez Lamancha B., Polo M., Caivano D., Piattini M. and Visaggio G. (2012). Automated generation of Test Oracles using a model-driven approach. Information and Software Technology. Accepted and in press. Available at: http://dx.doi.org/10.1016/j.infsof.2012.08.009
  2. Reales P., Polo M. and Offutt J. (2012). Mutation at the multi-class and system levels. Science of Computer Programming. Accepted and in Press. Available at: http://dx.doi.org/10.1016/j.scico.2012.02.005
  3. Reales P., Polo M. and Fernández JL. (2012). Validating 2nd-order mutation at system level. IEEE Transactions on Software Engineering. Accepted and in Press. Available at: http://doi.ieeecomputersociety.org/10.1109/TSE.2012.39
  4. Reales P. and Polo M. (2012). Parallel mutation testing. Software Testing, Verification and Reliability. Accepted and in press. Available at: http://onlinelibrary.wiley.com/doi/10.1002/stvr.1471/abstract
Moreover, we have published a text book (in Spanish) where we present the main concepts of combinatorial and mutation testing: Técnicas combinatorias y de mutación para testing de sistemas software, ed. Ra-Ma, Madrid, Spain, 2012: http://www.ra-ma.es/libros/TECNICAS-COMBINATORIAS-Y-DE-MUTACION-PARA-TESTING-DE-SISTEMAS-SOFTWARE/75098/978-84-9964-146-1

Saturday, June 16, 2012

expoQA 2012

Last 6 of June, we were at the International Conference on Software Testing & Quality (expoQA 2012), which was organized by nexoQA and was held in the Hotel Auditorioum, in Madrid.
Federico had a tutorial about performance testing, whilst Macario gave a talk entitled "Mutation: a testing technique ready to its transference to industry", which got the Technical Committee Award to the best technical paper.
Moreover, we did some demos of the Bacterio tool for mutation testing at the stand of Panel Sistemas Informáticos.

Monday, May 21, 2012

Semaphores: a new Java system for testing



Some time ago, we developed a small Java system for doing a first approach to analyze test case traceability, that we published in the SEKE 2009 conference (Int. Conf. on Software Engineering and Knowledge Engineering: download here the paper Some experiments on test case traceability). Our goal was to study how the coverage reached by a test suite on a high-level representation of a system is related to the coverage reached by those very same test cases in the system implementation.
The system simules a street with two semaphores. Pedestrians may press a button on any of the two semaphores for requesting red. There is a Manager that controls the light flow of two semaphores: when there are no pedestrians, the manager changes the light of both semaphores (a and b) sending them the change event every a fixed number of seconds (60, 63, 66, 83, and 86). However, a pedestrian may request the red light in any of the semaphores: if the semaphore where red is requested is in yellow or red, nothing happens; if it is in green and the semaphore is a, then the managers changes a to yellow either 20 seconds after the request or, if less than 20 seconds remain, in this time. If the red light is requested on b, then the request is passed to a:
The structure of the system is shown in the next figure: the window is represented by the JSemaphores class, which knows an instance of the Manager. User events are passed from JSemaphores to the Manager, who deals with the two instances of Sempahore it knows. There is also a Watch for controlling the color cycles where no pedestrians request red.

The behavior of the Manager class is given by the following state machine:

A text file representation of this state machine, valid to be processed by CTWeb for generating test cases, is the following:
% This is a small example of a state machine description file

Initial node
JC

% Transitions have: source state TAB symbol of the alphabet TAB target state
Transitions
JC setA A established
A established setB GG
GG requestRedOnA GG
GG requestRedOnB GG
GG setTime60 YG
YG setTime63 RY
RY setTime66 RR
RR setTime83 GR
GR requestRedOnA TGR
GR requestRedOnB TGR
TGR setTime86 GG
GR setTime86 GG

% Symbols can be mapped to method calls of the system using: symbol TAB method.
Symbol aliases
setA Manager manager=new Manager(this); Semaphore a=new Semaphore(manager); manager.setA(a);
setB Semaphore b=new Semaphore(manager); manager.setB(b);
requestRedOnA manager.requestRed(a);
requestRedOnB manager.requestRed(b);
setTime60 manager.setTime(60);
setTime63 manager.setTime(63);
setTime66 manager.setTime(66);
setTime83 manager.setTime(83);
setTime86 manager.setTime(86);

% States can also be used to the further creation of action oracles.
% The syntax is State TAB expression and the label is State aliases. For example:
State aliases
A established assertTrue(a.toString().equals("GREEN"));
GG assertTrue(manager.toString().equals("GREEN,GREEN,false"));
YG assertTrue(manager.toString().equals("YELLOW,GREEN,false"));
RY assertTrue(manager.toString().equals("RED,YELLOW,false"));
RR assertTrue(manager.toString().equals("RED,RED,false"));
GR assertTrue(manager.toString().equals("GREEN,RED,false"));
TGR assertTrue(manager.toString().equals("GREEN,RED,true"));

If you process the state machine with CTWeb with All pairs coverage, you get the following tests to path:

Path: [1] [setA, setB, requestRedOnA, requestRedOnA, requestRedOnB, requestRedOnA, setTime60, setTime63, setTime66, setTime83, requestRedOnA, setTime86, requestRedOnA]

Path: [2] [setA, setB, requestRedOnB, requestRedOnB, setTime60]
Path: [3] [setA, setB, setTime60]
Path: [4] [setA, setB, setTime60, setTime63, setTime66, setTime83, requestRedOnA, setTime86, requestRedOnB]
Path: [5] [setA, setB, setTime60, setTime63, setTime66, setTime83, requestRedOnA, setTime86, setTime60]
Path: [6] [setA, setB, setTime60, setTime63, setTime66, setTime83, setTime86, requestRedOnA]
Path: [7] [setA, setB, setTime60, setTime63, setTime66, setTime83, setTime86, requestRedOnB]
Path: [8] [setA, setB, setTime60, setTime63, setTime66, setTime83, setTime86, setTime60]
Path: [9] [setA, setB, setTime60, setTime63, setTime66, setTime83, requestRedOnB, setTime86]
Path: [10] [setA, setB, setTime60, setTime63, setTime66, setTime83, setTime86]



The code for the JUnit test case for the first path, generated by CTWeb is:
Manager manager=new Manager(this); Semaphore a=new Semaphore(manager); manager.setA(a);
assertTrue(a.toString().equals("GREEN"));
assertTrue(a.toString().equals("GREEN"));
Semaphore b=new Semaphore(manager); manager.setB(b);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.requestRed(a);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.requestRed(a);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.requestRed(b);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.requestRed(a);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.setTime(60);
assertTrue(manager.toString().equals("YELLOW,GREEN,false"));
manager.setTime(63);
assertTrue(manager.toString().equals("RED,YELLOW,false"));
manager.setTime(66);
assertTrue(manager.toString().equals("RED,RED,false"));
manager.setTime(83);
assertTrue(manager.toString().equals("GREEN,RED,false"));
manager.requestRed(a);
assertTrue(manager.toString().equals("GREEN,RED,true"));
manager.setTime(86);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.requestRed(a);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));

Here, you can download the complete code of this system and a test suite covering All pairs generated by CTWeb from the test description of the state machine. Feel free to use this stuff.


The test suite may be executed by Bacterio, our mutation testing tool, which is free for students, professors and researchers. The next figure shows the results of executing the test suite (which proceeds from applying All pairs to the state machine) against the semaphores implementation: note that only 36.95% of mutants are killed.


Friday, March 9, 2012

MATE: Methodology for Automated model-driven TEsting

Some days ago, we have a published an article in the Testing Experience magazine explaining MATE.
Besides a typical uruguayan and argentine beverage, MATE is also complete methodology for automated model-driven testing that works both in traditional software development as in software product lines contexts.
MATE is completely based on OMG standards: UML, QVT, MOFM2T, UML Testing Profile... and has also capabilities for generating code.
The article can be downloaded for free from this link of the this link of the magazine web page.

Friday, February 17, 2012

The Java precision...

Hi all,

The other day i was testing a system that calcuates weights and i discobered a precision bug in Java.

Try to run this:

public void testPor3(){
double a = 3;
double b = a * 0.2;
System.out.println(b);
double af = 3;
double bf = a / 5;
System.out.println(bf);
}

The output that i got was:

0.6000000000000001
0.6


Then, i and Fede googled about this error and we found this (http://www.velocityreviews.com/forums/t139008-java-double-precision.html):

double val = 0;
for(int i=0;i<10;i++) {
val+=0.1;
System.out.println(val);
}

has the following (terrible) output:

0.1
0.2
0.30000000000000004
0.4
0.5
0.6
0.7
0.7999999999999999
0.8999999999999999
0.9999999999999999


What a mess!