2 Timing Sample
A fixed sample of images was randomly and proportionally selected from the ELFT datasets. The sample is used to assess whether an implementation adheres to the computational speed requirements from the ELFT Test Plan. These values are chosen in such a way that allows the implementation flexibility while allowing NIST to complete the evaluation in a reasonable amount of time. If an implementation exceeds the maximum allowable duration, the participant will be asked to reduce the processing time of their software prior to NIST completing the evaluation. As such, all published ELFT submissions conform to the published speed requirements.
2.1 Processor Details
All measurements in this section were performed on a machine equipped with Intel Xeon Gold 6254 Central Processing Units (CPUs). Each CPU features a 3.10 GHz base frequency and 24.75 MB of cache. Timing tests are all single threaded—implementations are not permitted to use more than one CPU core during any function measured here. As such, these values can be used to understand expected scaled performance. NIST testing code embraces the single-threaded nature of implementations to fork
processes during other non-timed portions of this evaluation, allowing participants to write thread-unsafe code while still using NIST resources to their maximum efficiency. This CPU supports executing several families of processor intrinsic functions, including AVX-5121.
2.2 Composition
Table 2.1 shows the quantity of each type of fingerprint image comprising the timing sample dataset.
Image Type | Quantity |
---|---|
Latent | 243 |
Four Finger | 476 |
Full Palm | 40 |
Partial Palm | 47 |
Single Finger | 2 784 |
2.3 Feature Extraction
Features were extracted from all images depicted in Table 2.1 and stored in templates. If a sample contained EFS data, it was not included during this test.
2.3.1 Template Size
Table 2.2 and Figure 2.1 show the distribution of file sizes of templates. Failures of any kind reported during template generation result in NIST code writing 0 byte files. These files are excluded from the template size analysis in this section.
Image Type | Minimum | 25% | Median | Mean | 75% | Maximum | Failures | Attempts |
---|---|---|---|---|---|---|---|---|
Latent | 1.3 | 19.4 | 26.3 | 27.5 | 33.3 | 95.5 | 0 | 243 |
Single Finger | 2.5 | 8.3 | 10.1 | 10.1 | 11.9 | 19.2 | 0 | 2 784 |
Four Finger | 10.3 | 14.7 | 16.2 | 16.2 | 17.6 | 24.7 | 0 | 476 |
Partial Palm | 0.3 | 10.8 | 16.6 | 14.1 | 18.1 | 26.4 | 0 | 47 |
Full Palm | 21.4 | 24.8 | 26.0 | 26.0 | 27.4 | 29.9 | 0 | 40 |
2.3.2 Template Creation Duration
Table 2.3 and Figure 2.2 show the distribution of template creation durations in seconds. Failures of any kind reported during template generation result in NIST code writing 0 byte files, but only after the template creation method returns. These times are included in the template creation duration analysis in this section.
Image Type | Minimum | 25% | Median | Mean | 75% | Maximum | Failures | Attempts |
---|---|---|---|---|---|---|---|---|
Latent | 2.4 | 6.5 | 8.4 | 11.0 | 11.9 | 95.5 | 0 | 243 |
Single Finger | 0.0 | 0.2 | 0.2 | 0.3 | 0.3 | 0.5 | 0 | 2 784 |
Four Finger | 0.3 | 0.4 | 0.5 | 0.5 | 0.5 | 0.9 | 0 | 476 |
Partial Palm | 0.7 | 1.8 | 2.1 | 2.2 | 2.7 | 4.1 | 0 | 47 |
Full Palm | 3.0 | 3.9 | 4.2 | 4.2 | 4.5 | 5.3 | 0 | 40 |
2.4 Enrollment Database
Reference templates are combined into a participant-defined database structure for optimal searching. Each database consisted of ≈ 1 600 000 distractor subjects. Each subject had at least one, but typically twenty, distal phalanges distributed over rolled and flat impression captures to enroll. ≈ 150 000 had one or more palm captures.
While the participant-defined enrollment database should contain information about all references, the file size may be significantly different than the space consumed by concatenation of all individual reference templates. Additionally, the participant-defined database structure may be a structure unique or especially optimized for this evaluation and not necessarily similar to a structure deployed operationally. The sum of file sizes for both types of reference storage are shown in Table 2.4.
Storage Type | Size |
---|---|
Participant-Defined Enrollment Database | 439.8 GB |
Raw Templates on Disk | 252.4 GB |
2.5 Search
Out of the latent templates generated in Table 2.1, a fixed random sample of 24 of the resulting latent templates were searched against the enrollment database described in Subsection 2.4. The results presented in Subsection 2.5 are based on the measurements made on or during those 24 searches.
2.5.1 Search Duration
Table 2.5 and Figure 2.4 show the amount of time elapsed during searches of the fixed search probe set when searching against the enrollment database described in Subsection 2.4. While unsuccessful searches expend operator time, they are not included in this metric, because search failures typically occur instantaneously (e.g., a template indicates that a probe was of too poor quality to search), which can artificially lower the average search time.
ELFT defines maximum average search durations for participants based on the number of subjects in the enrollment database. Due to the potential for extended runtimes, NIST may choose to allow some discretion in the enforcement of maximum search durations during times of high demand for compute resources. For example, if a maximum average search duration was 4 hours, but after completing all searches, the average search duration was 4.5 hours, it may be prudent to continue the evaluation, since a resubmission may require regeneration of millions of templates and several thousand repeated searches.
Note: In March 2023, NIST lowered the number of searches from 100 to 25, with all 25 probes depicting a distal phalanx. It also doubled the average quantity of impressions per subject by combining previously separate plain and rolled impressions for each subject. ELFT does not mandate the strategy in which multiple impressions of the same reference finger are stored or searched in the enrollment database, but it does impose search time maximums on a per-subject basis, not per-impression. This means that in Table 2.5 and Figure 2.4, there may be average search durations significantly higher than the evaluation permitted maximum for implementations submitted prior to March 2023.
Note: In October 2023, NIST discovered a probe image in the search dataset contained invalid resolution information. This probe has been removed from the test, reducing the number of searches to 24. Evaluations run prior to this change will show total elapsed time measures including the now-omitted search.
Mated? | Min | 25% | Median | Mean | 75% | Maximum | Failures | Searches |
---|---|---|---|---|---|---|---|---|
False | 1 394 | 2 488 | 3 364 | 3 285 | 3 817 | 5 455 | 0 | 24 |
True | 1 416 | 2 558 | 3 422 | 3 371 | 4 120 | 5 543 | 0 | 24 |
2.5.2 Search Memory Consumption
Figure 2.5 shows the amount of RAM consumed by the single testing process as a function of time during the search procedure, including RAM consumed by the NIST testing apparatus. Implementations were permitted to use up to 300 GB of RAM (of a total available 384 GB) to load their enrollment database, the rest of which was stored on a local solid-state storage device. Note the different scales on each panel—implementations that do not change the contents of RAM may not show variation.
The complete set of advertised CPU flags is
fpu
,vme
,de
,pse
,tsc
,msr
,pae
,mce
,cx8
,apic
,sep
,mtrr
,pge
,mca
,cmov
,pat
,pse36
,clflush
,dts
,acpi
,mmx
,fxsr
,sse
,sse2
,ss
,ht
,tm
,pbe
,syscall
,nx
,pdpe1gb
,rdtscp
,lm
,constant_tsc
,art
,arch_perfmon
,pebs
,bts
,rep_good
,nopl
,xtopology
,nonstop_tsc
,cpuid
,aperfmperf
,pni
,pclmulqdq
,dtes64
,monitor
,ds_cpl
,vmx
,smx
,est
,tm2
,ssse3
,sdbg
,fma
,cx16
,xtpr
,pdcm
,pcid
,dca
,sse4_1
,sse4_2
,x2apic
,movbe
,popcnt
,tsc_deadline_timer
,aes
,xsave
,avx
,f16c
,rdrand
,lahf_lm
,abm
,3dnowprefetch
,cpuid_fault
,epb
,cat_l3
,cdp_l3
,invpcid_single
,intel_ppin
,ssbd
,mba
,ibrs
,ibpb
,stibp
,ibrs_enhanced
,tpr_shadow
,vnmi
,flexpriority
,ept
,vpid
,ept_ad
,fsgsbase
,tsc_adjust
,bmi1
,avx2
,smep
,bmi2
,erms
,invpcid
,cqm
,mpx
,rdt_a
,avx512f
,avx512dq
,rdseed
,adx
,smap
,clflushopt
,clwb
,intel_pt
,avx512cd
,avx512bw
,avx512vl
,xsaveopt
,xsavec
,xgetbv1
,xsaves
,cqm_llc
,cqm_occup_llc
,cqm_mbm_total
,cqm_mbm_local
,dtherm
,ida
,arat
,pln
,pts
,pku
,ospke
,avx512_vnni
,md_clear
,flush_l1d
,arch_capabilities
↩