Semaphores: a new Java system for testing

Some time ago, we developed a small Java system for doing a first approach to analyze test case traceability, that we published in the SEKE 2009 conference (Int. Conf. on Software Engineering and Knowledge Engineering: download here the paper Some experiments on test case traceability). Our goal was to study how the coverage reached by a test suite on a high-level representation of a system is related to the coverage reached by those very same test cases in the system implementation.

The system simules a street with two semaphores. Pedestrians may press a button on any of the two semaphores for requesting red. There is a Manager that controls the light flow of two semaphores: when there are no pedestrians, the manager changes the light of both semaphores (a and b) sending them the change event every a fixed number of seconds (60, 63, 66, 83, and 86). However, a pedestrian may request the red light in any of the semaphores: if the semaphore where red is requested is in yellow or red, nothing happens; if it is in green and the semaphore is a, then the managers changes a to yellow either 20 seconds after the request or, if less than 20 seconds remain, in this time. If the red light is requested on b, then the request is passed to a:

The structure of the system is shown in the next figure: the window is represented by the JSemaphores class, which knows an instance of the Manager. User events are passed from JSemaphores to the Manager, who deals with the two instances of Sempahore it knows. There is also a Watch for controlling the color cycles where no pedestrians request red.

The behavior of the Manager class is given by the following state machine:

A text file representation of this state machine, valid to be processed by CTWeb for generating test cases, is the following:

% This is a small example of a state machine description file

Initial node

JC

% Transitions have: source state TAB symbol of the alphabet TAB target state

Transitions

JC setA A established

A established setB GG

GG requestRedOnA GG

GG requestRedOnB GG

GG setTime60 YG

YG setTime63 RY

RY setTime66 RR

RR setTime83 GR

GR requestRedOnA TGR

GR requestRedOnB TGR

TGR setTime86 GG

GR setTime86 GG

% Symbols can be mapped to method calls of the system using: symbol TAB method.

Symbol aliases

setA Manager manager=new Manager(this); Semaphore a=new Semaphore(manager); manager.setA(a);

setB Semaphore b=new Semaphore(manager); manager.setB(b);

requestRedOnA manager.requestRed(a);

requestRedOnB manager.requestRed(b);

setTime60 manager.setTime(60);

setTime63 manager.setTime(63);

setTime66 manager.setTime(66);

setTime83 manager.setTime(83);

setTime86 manager.setTime(86);

% States can also be used to the further creation of action oracles.

% The syntax is State TAB expression and the label is State aliases. For example:

State aliases

A established assertTrue(a.toString().equals("GREEN"));

GG assertTrue(manager.toString().equals("GREEN,GREEN,false"));

YG assertTrue(manager.toString().equals("YELLOW,GREEN,false"));

RY assertTrue(manager.toString().equals("RED,YELLOW,false"));

RR assertTrue(manager.toString().equals("RED,RED,false"));

GR assertTrue(manager.toString().equals("GREEN,RED,false"));

TGR assertTrue(manager.toString().equals("GREEN,RED,true"));

If you process the state machine with CTWeb with All pairs coverage, you get the following tests to path:

Path: [1] [setA, setB, requestRedOnA, requestRedOnA, requestRedOnB, requestRedOnA, setTime60, setTime63, setTime66, setTime83, requestRedOnA, setTime86, requestRedOnA]

Path: [2] [setA, setB, requestRedOnB, requestRedOnB, setTime60]
Path: [3] [setA, setB, setTime60]
Path: [4] [setA, setB, setTime60, setTime63, setTime66, setTime83, requestRedOnA, setTime86, requestRedOnB]
Path: [5] [setA, setB, setTime60, setTime63, setTime66, setTime83, requestRedOnA, setTime86, setTime60]
Path: [6] [setA, setB, setTime60, setTime63, setTime66, setTime83, setTime86, requestRedOnA]
Path: [7] [setA, setB, setTime60, setTime63, setTime66, setTime83, setTime86, requestRedOnB]
Path: [8] [setA, setB, setTime60, setTime63, setTime66, setTime83, setTime86, setTime60]
Path: [9] [setA, setB, setTime60, setTime63, setTime66, setTime83, requestRedOnB, setTime86]
Path: [10] [setA, setB, setTime60, setTime63, setTime66, setTime83, setTime86]

The code for the JUnit test case for the first path, generated by CTWeb is:

Manager manager=new Manager(this); Semaphore a=new Semaphore(manager); manager.setA(a);
assertTrue(a.toString().equals("GREEN"));
assertTrue(a.toString().equals("GREEN"));
Semaphore b=new Semaphore(manager); manager.setB(b);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.requestRed(a);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.requestRed(a);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.requestRed(b);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.requestRed(a);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.setTime(60);
assertTrue(manager.toString().equals("YELLOW,GREEN,false"));
manager.setTime(63);
assertTrue(manager.toString().equals("RED,YELLOW,false"));
manager.setTime(66);
assertTrue(manager.toString().equals("RED,RED,false"));
manager.setTime(83);
assertTrue(manager.toString().equals("GREEN,RED,false"));
manager.requestRed(a);
assertTrue(manager.toString().equals("GREEN,RED,true"));
manager.setTime(86);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));
manager.requestRed(a);
assertTrue(manager.toString().equals("GREEN,GREEN,false"));

Here, you can download the complete code of this system and a test suite covering All pairs generated by CTWeb from the test description of the state machine. Feel free to use this stuff.

The test suite may be executed by Bacterio, our mutation testing tool, which is free for students, professors and researchers. The next figure shows the results of executing the test suite (which proceeds from applying All pairs to the state machine) against the semaphores implementation: note that only 36.95% of mutants are killed.