Departamento de Informática da Universidade da Beira Interior
SOCIA Lab. – Soft Computing and Image Analysis Group
Department of Computer Science, University of Beira Interior, 6201-001 Covilhã, Portugal
NICE:II Frequently Asked Questions
1. What is the NICE:II contest?
The NICE:II contest is an iris recognition contest that will evaluate the following task: to compare and compute the dissimilarity between segmented noisy iris images for biometric purposes.
2. What is the main goal of the NICE:II contest?
The NICE:II main goal is to evaluate the robustness to noise of iris encoding and matching methods, toward iris recognition systems that operate in less constrained imaging conditions, eventually covertly.
3. What distinguishes the NICE:II from other similar contests?
There are two main factors that distinguish the NICE:II contest from others:
· Other similar biometric contests (e.g., Iris Challenge Evaluation and Fingerprint Verification Competition) evaluate the complete recognition process, from the raw data preprocessing to the final identity decision. Oppositely, the NICE:II contest is focused on the encoding and matching stages. Thus, segmentation errors will not corrupt the final results, which enables a fair comparison between encoding and matching proposals.
· The NICE:II contest operates on the highly noisy data of the UBIRIS.v2 database. This database has one fundamental characteristic that distinguishes it from the remaining ones: here the noise factors, rather than avoided, are induced. This enables the effective evaluation of the algorithms robustness.
4. Who can participate in the NICE:II contest?
The NICE:II contest is open to individuals and institutions, either with academic, industrial, research or commercial purposes. Participations are allowed from any country of the world. Also, it is completely free of any monetary charge.
5. When and where will the NICE:II contest take place?
The NICE:II is organized by the SOCIA Lab (Soft Computing and Image Analysis Group) and it will take place between October 2009 (start of the application forms reception period) and October 2010 (publication of the results). The evaluation of the contest participations will take place at the SOCIA Lab, Department of Computer Science, University of Beira Interior, Covilhã, Portugal. Obviously, as the participations are submitted through the web, it is not required the physical presence of any contest participant.
6. What are the main advantages for the NICE:II participants?
Apart of evaluate the robustness of the iris encoding and matching methods on highly noisy data, such as the contained by the second version of the UBIRIS database (UBIRIS.v2), the NICE:II best participants will have the opportunity to publish their algorithms in a high reputable ISI-indexed International Journal, such as the Pattern Recognition Letters Journal. Also, the remaining participants will have the opportunity to publish their algorithms in the NICE:II contest proceedings (electronically, with ISBN). Finally, participants will receive a Java-based evaluation framework (binaries and source code), which will be used both in the development and in the evaluation stage to conclude about the algorithms’ accuracy.
7. Is the publication of the results obligatory?
Although the NICE:II participations can’t be anonymous, it’s up to the participants to decide whether they want to publish their algorithms, either in the Pattern Recognition Letters Journal, in the NICE:II proceedings or nowhere.
8. What kinds of executables are accepted?
The NICE:II organizing committee has decided to accept standalone executables that must execute in one of the following operating systems (Microsoft Windows XP, with Service Pack 2 or Fedora Core 6). Moreover, the main executable must be able to be called from the command line and receive two arguments: the input and output data paths.
Also, MATLAB functions are accepted too: the main function should be called from the MATLAB command-line and have similar requirements to the standalone application.
9. What libraries will be provided by the NICE:II organizing committee?
No libraries will be provided and previously installed on the NICE:II evaluation systems. All libraries that an executable requires will have to be provided and placed within the respective “lib” folder, or incorporated into the executable.
10. What is the expected installation procedure?
All the participations are submitted through the NICE:II email, in a compacted file. Further, there are two options:
· The submittal is installed via direct copy to the correspondent participation folder. In this case, it should not be required any configuration steps.
· If some configuration step is required, the submittal must include an installation program that performs all the file extractions and configurations.
11. What types of noise factors the second version of the UBIRIS database contains?
The aim of the second version of the UBIRIS database (UBIRIS.v2) is to realistically simulate less constrained image capturing conditions, either at-a-distance, on-the-move or with minimal users cooperation. When compared to its predecessor, this database contains more images and with more realistic noise factors, namely:
1. Poor focused iris images.
2. Off-angle iris images.
3. Rotated iris images.
4. Motion blurred iris images.
5. Iris obstructions due to eyelashes.
6. Iris obstructions due to eyelids.
7. Iris obstructions due to glasses.
8. Iris obstructions due to contact lenses.
9. Iris with specular reflections.
10. Iris with lighting reflections.
11. Partial captured iris images.
12. Out-of-iris images.
12. From where application executables read the input data? And where write the output data to?
The submitted application executables should receive as input 5 parameters: the path of two iris images and of the corresponding binary maps that give the iris segmentation. The last parameter is the path where to write the text file containing the dissimilarity measure. For instance:
> User1 i1.tiff i1.bmp i2.tiff i2.bmp i1_i2.txt
The above command will execute the application “User1”. The program should read the images “i1.tiff”, “i2.tiff”, “i1.bmp”, “i2.bmp” and write the output text file name “i1_i2.txt”. This file should contain exclusively a numeric dissimilarity value between the iris images.
Also, please see the answer to question 14, which details the entire folder structure of the NICE:II contest framework.
13. How should, exactly, the dissimilarity value be produced?
Essentially, the submitted NICE:II participations should distinguish between the intra-class and the inter-class comparisons. The dissimilarity value should be a metric that clearly distinguishes between intra-class comparisons (different images of the same iris) and inter-class comparisons (images of irises captured from different eyes).
In the following, two examples of the desired dissimilarity value will be given. In the first example, two iris images from the same subject give a low dissimilarity value. The second example compares two irises from different subjects, which should give a high dissimilarity value.
Figure 1: Intra-Class comparison, low dissimilarity value
Figure 2: Inter-Class comparison, high dissimilarity value
14. What is the folder structure of the NICE:II Java evaluation framework?
Figure 3: Folder structure of the NICE:II contest evaluation framework.
The above figure illustrates the hierarchical directory structure of the NICE:II contest evaluation framework. The description of each folder’s purpose is as follows:
· NICE:II: main folder. This folder can be installed in any hard drive location. Thus, the participants’ executables should always use relative paths.
· NICE:II/Bin: binary files required to run the evaluation framework.
· NICE:II/Config: configuration files required by the above mentioned binaries.
· NICE:II/Data: main data directory.
· NICE:II/Data/Training: data files received at the start of the NICE:II contest. These will be used by the participants to develop their methods. Iris images of the training set and corresponding binary images in “.bmp” format that give the segmentation of the iris. All the iris images have the same dimensions, resolution, bit depth and format (“.tiff”). The binary segmentation maps have the same name and “bmp” extension.
· NICE:II/Data/Test: data files delivered to NICE:II participants after the deadline for the applications executables.
· NICE:II/Doc: Help and manual files.
· NICE:II/Participants: main folder of the contest participants. Each valid NICE:II participation will have a sub-folder correspondent to the given username.
· NICE:II/Participants/User1: folder of the NICE:II participation with username equal to “User1”.
· NICE:II/Participants/User1/Bin: application executables of the “User1” NICE:II participation. Also, libraries needed by the executables should be installed into this folder.
· NICE:II/Participants/User1/Output: this folder contains the output images correspondent to all the executions of the “User1” application. This can be useful in the algorithms development stage.
· NICE:II/Participants/User1/Output/2007_12_11_18_15_12: folder that contains the output images of the “User1” application executable, which was run in December 11th, 2007 at 18:15:12.
· NICE:II/Participants/UserN: folder of the NICE:II participation with username equal to “UserN”.
· NICE:II/Results: main folder of the contest results. It contains the results of all the NICE:II participations and executions in two different formats: XML and CSV files.
· NICE:II/Results/Csv: folder that contains the contest results in comma separated file format. The description of all the fields is given in the “NICE:II/Doc” folder.
· NICE:II/Results/Xml: folder that contains the contest results in eXtensible Markup Language file format. The used DTD can be found [here].
· NICE:II/Src: source files of the NICE:II contest framework. These are given essentially to enable further optimization / customization by the contest participants.
15. What are the folder permissions for a submitted executable?
Associated with any NICE:II participation it will be created a correspondent user by the evaluation framework. This user will have read / write permissions in the correspondent participation folder (for instance, NICE:II/Participants/User1 in the above example) and read permissions in the data input folders (NICE:II/Data/Training/Images and NICE:II/Data/Test/Images).
16. I still have some doubt that is not answered in his FAQ. What should I do?
You should contact the NICE:II organizing committee, through the following email address: firstname.lastname@example.org.
DI-UBI Bloco VI Rua Marquês de Ávila e Bolama P- 6201-001 Covilhã PORTUGAL