2 Timing Sample

A fixed sample of images was randomly and proportionally selected from the ELFT datasets. The sample is used to assess whether an implementation adheres to the computational speed requirements from the ELFT Test Plan. These values are chosen in such a way that allows the implementation flexibility while allowing NIST to complete the evaluation in a reasonable amount of time. If an implementation exceeds the maximum allowable duration, the participant will be asked to reduce the processing time of their software prior to NIST completing the evaluation. As such, all published ELFT submissions conform to the published speed requirements.

2.1 Processor Details

All measurements in this section were performed on a machine equipped with Intel Xeon Gold 6254 Central Processing Units (CPUs). Each CPU features a 3.10 GHz base frequency and 24.75 MB of cache. Timing tests are all single threaded—implementations are not permitted to use more than one thread during any function measured here. As such, these values can be used to understand expected scaled performance. NIST testing code embraces the single-threaded nature of implementations to fork processes during other non-timed portions of this evaluation, allowing participants to write thread-unsafe code while still using NIST resources to their maximum efficiency. This CPU supports executing several families of processor intrinsic functions, including AVX-512¹.

2.2 Composition

Table 2.1 shows the quantity of each type of fingerprint image comprising the timing sample dataset.

Table 2.1: Number of images of each generalized finger position comprising the timing sample dataset.
Image Type	Quantity
Latent	250
Four Finger	476
Full Palm	40
Partial Palm	47
Single Finger	2 784

2.3 Feature Extraction

Features were extracted from all images depicted in Table 2.1 and stored in templates. If a sample contained EFS data, it was included during this test.

2.3.1 Template Size

Table 2.2 and Figure 2.1 show the distribution of file sizes of templates. Failures of any kind reported during template generation result in NIST code writing 0 byte files. These files are excluded from the template size analysis in this section.

Table 2.2: Template file size summary statistics as seen on the Timing Sample dataset, in kB.
Image Type	Minimum	25%	Median	Mean	75%	Maximum	Failures	Attempts
Latent	2.7	7.2	9.9	14.2	14.5	196.3	7	250
Single Finger	0.3	10.9	13.8	14.1	17.3	27.9	0	2 784
Four Finger	5.2	17.9	21.1	21.2	24.3	40.4	0	476
Partial Palm	0.1	67.4	91.5	95.1	112.9	302.4	0	47
Full Palm	240.5	274.5	321.4	322.2	353.3	450.7	0	40

$Violin plot of template file sizes as seen on the Timing Sample dataset. Vertical lines from left to right indicate the 25\%, 50\%, and 75\% quantiles respectively.$

Figure 2.1: Violin plot of template file sizes as seen on the Timing Sample dataset. Vertical lines from left to right indicate the 25%, 50%, and 75% quantiles respectively.

2.3.2 Template Creation Duration

Table 2.3 and Figure 2.2 show the distribution of template creation durations in seconds. Failures of any kind reported during template generation result in NIST code writing 0 byte files, but only after the template creation method returns. These times are included in the template creation duration analysis in this section.

Table 2.3: Duration of template creation in seconds for images from the Timing Sample dataset.
Image Type	Minimum	25%	Median	Mean	75%	Maximum	Failures	Attempts
Latent	0.3	1.6	2.2	3.1	3.2	31.6	7	250
Single Finger	0.1	0.7	0.8	0.8	0.9	1.2	0	2 784
Four Finger	0.2	0.6	0.7	0.7	0.8	1.8	0	476
Partial Palm	2.0	3.7	4.7	4.8	5.3	19.4	0	47
Full Palm	10.6	11.6	12.3	12.7	13.5	17.1	0	40

$Violin plot of the duration of template creation in seconds for images from the Timing Sample dataset. Vertical lines from left to right indicate the 25\%, 50\%, and 75\% quantiles respectively.$

Figure 2.2: Violin plot of the duration of template creation in seconds for images from the Timing Sample dataset. Vertical lines from left to right indicate the 25%, 50%, and 75% quantiles respectively.

2.3.3 Template Creation Memory Consumption

Figure 2.3 shows the amount of RAM consumed by the single testing process as a function of time during the template creation procedure, including RAM consumed by the NIST testing apparatus.

Figure 2.3: Amount of RAM used while creating templates in the Timing Sample dataset.

2.4 Enrollment Database

Reference templates are combined into a participant-defined database structure for optimal searching. Each database consisted of ≈ 1 600 000 distractor subjects. Each subject had at least one, but typically twenty, distal phalanges distributed over rolled and flat impression captures to enroll. ≈ 150 000 had one or more palm captures.

While the participant-defined enrollment database should contain information about all references, the file size may be significantly different than the space consumed by concatenation of all individual reference templates. Additionally, the participant-defined database structure may be a structure unique or especially optimized for this evaluation and not necessarily similar to a structure deployed operationally. The sum of file sizes for both types of reference storage are shown in Table 2.4.

Table 2.4: Sum of file storage needed to hold all distractor reference templates in the Timing Sample.
Storage Type	Size
Participant-Defined Enrollment Database	412.4 GB
Raw Templates on Disk	408.7 GB

2.5 Search

Out of the latent templates generated in Table 2.1, a fixed random sample of 25 of the resulting latent templates were searched against the enrollment database described in Subsection 2.4. The results presented in Subsection 2.5 are based on the measurements made on or during those 25 searches.

2.5.1 Search Duration

Table 2.5 and Figure 2.4 show the amount of time elapsed during searches of the fixed search probe set when searching against the enrollment database described in Subsection 2.4. While unsuccessful searches expend operator time, they are not included in this metric, because search failures typically occur instantaneously (e.g., a template indicates that a probe was of too poor quality to search), which can artificially lower the average search time.

ELFT defines maximum average search durations for participants based on the number of subjects in the enrollment database. Due to the potential for extended runtimes, NIST may choose to allow some discretion in the enforcement of maximum search durations during times of high demand for compute resources. For example, if a maximum average search duration was 4 hours, but after completing all searches, the average search duration was 4.5 hours, it may be prudent to continue the evaluation, since a resubmission may require regeneration of millions of templates and several thousand repeated searches.

Note: In March 2023, NIST lowered the number of searches from 100 to 25, with all 25 probes depicting a distal phalanx. It also doubled the average quantity of impressions per subject by combining previously separate plain and rolled impressions for each subject. ELFT does not mandate the strategy in which multiple impressions of the same reference finger are stored or searched in the enrollment database, but it does impose search time maximums on a per-subject basis, not per-impression. This means that in Table 2.5 and Figure 2.4, there may be average search durations significantly higher than the evaluation permitted maximum for implementations submitted prior to March 2023.

Table 2.5: Search time durations of the search probe set from the Timing Sample dataset, in seconds. This data is visualized in Figure 2.4.
Mated?	Min	25%	Median	Mean	75%	Maximum	Failures	Searches
False	2 955	4 986	6 432	7 351	8 490	17 204	4	25
True	3 524	5 287	6 522	7 729	8 525	17 759	4	25

$Violin plot of search time durations of the search probe set from the Timing Sample dataset. Vertical lines from left to right indicate the 25\%, 50\%, and 75\% quantiles respectively.$

Figure 2.4: Violin plot of search time durations of the search probe set from the Timing Sample dataset. Vertical lines from left to right indicate the 25%, 50%, and 75% quantiles respectively.

2.5.2 Search Memory Consumption

Figure 2.5 shows the amount of RAM consumed by the single testing process as a function of time during the search procedure, including RAM consumed by the NIST testing apparatus. Implementations were permitted to use up to 300 GB of RAM (of a total available 384 GB) to load their enrollment database, the rest of which was stored on a local solid-state storage device. Note the different scales on each panel—implementations that do not change the contents of RAM may not show variation.

Figure 2.5: Amount of RAM used while searching templates in the Timing Sample dataset.

The complete set of advertised CPU flags is fpu, vme, de, pse, tsc, msr, pae, mce, cx8, apic, sep, mtrr, pge, mca, cmov, pat, pse36, clflush, dts, acpi, mmx, fxsr, sse, sse2, ss, ht, tm, pbe, syscall, nx, pdpe1gb, rdtscp, lm, constant_tsc, art, arch_perfmon, pebs, bts, rep_good, nopl, xtopology, nonstop_tsc, cpuid, aperfmperf, pni, pclmulqdq, dtes64, monitor, ds_cpl, vmx, smx, est, tm2, ssse3, sdbg, fma, cx16, xtpr, pdcm, pcid, dca, sse4_1, sse4_2, x2apic, movbe, popcnt, tsc_deadline_timer, aes, xsave, avx, f16c, rdrand, lahf_lm, abm, 3dnowprefetch, cpuid_fault, epb, cat_l3, cdp_l3, invpcid_single, intel_ppin, ssbd, mba, ibrs, ibpb, stibp, ibrs_enhanced, tpr_shadow, vnmi, flexpriority, ept, vpid, ept_ad, fsgsbase, tsc_adjust, bmi1, avx2, smep, bmi2, erms, invpcid, cqm, mpx, rdt_a, avx512f, avx512dq, rdseed, adx, smap, clflushopt, clwb, intel_pt, avx512cd, avx512bw, avx512vl, xsaveopt, xsavec, xgetbv1, xsaves, cqm_llc, cqm_occup_llc, cqm_mbm_total, cqm_mbm_local, dtherm, ida, arat, pln, pts, pku, ospke, avx512_vnni, md_clear, flush_l1d, arch_capabilities↩