Face Analysis Technology Evaluation (FATE) Age Estimation & Verification
Test Plan and API | Latest Report | Participation Agreement | Validation | Encryption | Submit
Status
[2023-05-27] The FATE AEV track is open. Developers should submit no more frequently than every four months, and use the FATE Submission Form to do so.
[2014-03-31] NIST published its first evaluation of AE performance as NISTIR 7995 on Performance of Automated Age Estimation.
Overview
Facial age verification has recently been mandated in legislation in a number
of jurisdictions. These laws are typically intended to protect minors from
various harms by verifying that the individual is above a certain age, within
an age range, and, less commonly below a certain age. Other use-cases seek
only to determine actual age. While the mechanism for estimating age is usually
not specified in legislation, software-based face analysis is an attractive
approach when a photograph can be captured.
FATE AEV is an ongoing evaluation of software algorithms that inspect photos and videos of a face to produce an age estimate. The output of this track is a set of reports on the accuracy and computational efficiency of algorithms. AEV is open to a worldwide community of developers. This evaluation is conducted on an ongoing basis in that the evaluation remains open indefinitely such that developers may submit their algorithms to NIST whenever they are ready, but no more frequently than four calendar months.
AE Performance
Results: [last updated: 2024-05-28]
The table shows accuracy measures for 2 common applications, Challenge-25 age-restriction and children age 13-16 online chatroom access. False positive rates quantify how often people outside the allowed age range are estimated to be within required category. True positive rates quantify how often children within age 13-16 are estimated to be within that age range.
- Challenge 25:
-
- [Dataset: Application (good quality), Border (medium quality)]
- Application - False Positive Rate (FPR) values are averages of 12 FPR estimates from two sexes and six regions-of-birth for subjects aged 17.
- Border - False Positive Rate (FPR) values are averages of 12 FPR estimates from two sexes and six regions-of-birth for subjects aged 17.
- Child Online Safety (13-16):
-
- [Dataset: Visa (good quality)]
- Age 13-16 - Mean Absolute Errors (MAE) are averages of four MAE values for age group 13, 14, 15, and 16.
- Age 13-16 - True Positive Rates (TPR) are proportions of subjects aged 13 to 16 and whose age is estimated from 13 to 16 (below 17).
- Age < 13 - False Positive Rates (FPR) are proportions of subjects aged below 13 but whose age is estimated from 13 to 16 (below 17).
- Age ≥ 17 - False Positive Rates (FPR) are proportions of subjects aged 17 or older but whose age is estimated from 13 to 16.
- Resource Usage:
-
- AE Time (msec) values are the median duration for one AE function, in milliseconds.
- Config (MB) values are the total size of the neural network model and configuration data of the algorithm.
- Lib (MB) values are the total size of the libraries of the algorithm.
How to Participate
To participate in this evaluation, developers need to submit a participation
agreement to NIST, wrap their software behind the published C++ API, run their
libraries through the provided validation package (which creates a submission
package), encrypt the package, and provide a download link for the encrypted
submission package.
[Participation agreement] FATE is conducted by NIST, an agency of the United States Government. Participation is free of charge. FATE is open to a global audience of face recognition developers. All organizations who seek to participate in FATE must sign all pages of this Participation Agreement and submit it with their algorithm submission using the Submission Form. [last update: 2023-08-17]
[API] General and common information shared between all tracks of the FRTE/FATE evaluations are documented in a General Evaluation Specifications, which includes hardware and operating system environment, software requirements, reporting, and common data structures that support the APIs. [last update: 2023-08-17]
[Validation] A validation package has been published. All participants must run their software through the validation package prior to submission. The purpose of validation is to ensure consistent algorithm output between your execution and NIST’s execution. [last update: 2023-08-17]
[Encryption] All submissions must be properly encrypted and signed before transmission to NIST. This must be done according to these instructions using the FATE Ongoing public key linked from this page. Participants must email their public key to NIST. The participant’s public key must correspond to the participant’s public-key fingerprint provided on the signed Participation Application. [last update: 2022-07-03]
[Submission] All algorithm submissions must be submitted through the Submission Form, which requires encrypted files be provided as a download link from a generic http server (e.g., Google Drive). We cannot accept Dropbox links. NIST will not register, or establish any kind of membership, on the provided website. Participants can submit their algorithm(s), participation agreement, and GPG key at the same time via the submission form. [last update: 2023-07-03]
Participants must subscribe to the evaluation mailing list to receive emails when new reports are published or announcements are made.