CLAIMBOUND / SOURCE-PUBLICATION EVIDENCE
Facial recognition
Every card identifies what it proves, what it does not prove, its exact quote, locator, source hash, and local rerun command.
1. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 5
Narrow public claim
We restrict the analysis to 24 countries in 7 distinct global regions that have seen lower levels of long-distance immigration.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 5- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t01_primary_claims.json --source-file <local-raw-pdf>Exact source quote
We restrict the analysis to 24 countries in 7 distinct global regions that have seen lower levels of long-distance immigration.
2. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 6
Narrow public claim
We supplement this report with more than 1200 pages of charts contained in seventeen annexes that include exhaustive reporting of results for each algorithm.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 6- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t01_primary_claims.json --source-file <local-raw-pdf>Exact source quote
We supplement this report with more than 1200 pages of charts contained in seventeen annexes that include exhaustive reporting of results for each algorithm.
3. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 6
Narrow public claim
This report includes recommendations for research in developing and evaluating the value, costs, and benefits of potential mitigation techniques - see sections 8 and 9.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 6- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t01_primary_claims.json --source-file <local-raw-pdf>Exact source quote
This report includes recommendations for research in developing and evaluating the value, costs, and benefits of potential mitigation techniques - see sections 8 and 9.
4. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 8
Narrow public claim
Other algorithms implement “fast-search” techniques [2,19,21,26] that avoid many of the N comparisons and are therefore highly economical [17].
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 8- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t01_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Other algorithms implement “fast-search” techniques [2,19,21,26] that avoid many of the N comparisons and are therefore highly economical [17].
5. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 8
Narrow public claim
In such cases, the false positive identification rate is 100% because any search of someone not in the database will still yield candidates.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 8- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t01_primary_claims.json --source-file <local-raw-pdf>Exact source quote
In such cases, the false positive identification rate is 100% because any search of someone not in the database will still yield candidates.
6. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 10
Narrow public claim
Note that in real-time cooperative applications, false negatives can often be remedied by making second attempts. ⊿ False negative error rates vary strongly by algorithm, from below 0.5% to above 10%.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 10- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t01_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Note that in real-time cooperative applications, false negatives can often be remedied by making second attempts. ⊿ False negative error rates vary strongly by algorithm, from below 0.5% to above 10%.
7. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 10
Narrow public claim
See Figure 22 and Annex 12 . ⊿ In domestic mugshots, false negatives are higher in Asian and American Indian individuals, with error rates above those in white and black faces.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 10- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t01_primary_claims.json --source-file <local-raw-pdf>Exact source quote
See Figure 22 and Annex 12 . ⊿ In domestic mugshots, false negatives are higher in Asian and American Indian individuals, with error rates above those in white and black faces.
8. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 11
Narrow public claim
See Figure 14 and Figure 15. ⊿ With mugshot images, the highest false positives are in American Indians, with elevated rates in African American and Asian populations; the relative ordering depends on sex and varies with algorithm.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 11- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t01_primary_claims.json --source-file <local-raw-pdf>Exact source quote
See Figure 14 and Figure 15. ⊿ With mugshot images, the highest false positives are in American Indians, with elevated rates in African American and Asian populations; the relative ordering depends on sex and varies with algorithm.
9. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 11
Narrow public claim
There are, however, recent prior tests of verification algorithms whose results comport with ours regarding demographic differentials between races. ⊿ Using four verification algorithms applied to domestic mugshots, the Florida Institute of Technology and its collaborators showed [23] simultaneously elevated false positives and reduced false negatives in African Americans vs.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 11- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t01_primary_claims.json --source-file <local-raw-pdf>Exact source quote
There are, however, recent prior tests of verification algorithms whose results comport with ours regarding demographic differentials between races. ⊿ Using four verification algorithms applied to domestic mugshots, the Florida Institute of Technology and its collaborators showed [23] simultaneously elevated false positives and reduced false negatives in African Americans vs.
10. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 12
Narrow public claim
Along these lines, it has been suggested [41] that NISTs tests using standards-compliant images “don’t translate to everyday scenarios”.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 12- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t01_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Along these lines, it has been suggested [41] that NISTs tests using standards-compliant images “don’t translate to everyday scenarios”.
11. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 12
Narrow public claim
For example, the main criminal justice application is supported by the FBI and others being proactive in the 1990s in establishing portrait capture standards, and then promulgating them. ⊿ Use wild images: We did not use image data from the Internet nor from video surveillance.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 12- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t02_primary_claims.json --source-file <local-raw-pdf>Exact source quote
For example, the main criminal justice application is supported by the FBI and others being proactive in the 1990s in establishing portrait capture standards, and then promulgating them. ⊿ Use wild images: We did not use image data from the Internet nor from video surveillance.
12. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 14
Narrow public claim
We supplement this report with more than 1200 pages of charts contained in 17 Annexes which include exhaustive reporting of results for each algorithm.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 14- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t02_primary_claims.json --source-file <local-raw-pdf>Exact source quote
We supplement this report with more than 1200 pages of charts contained in 17 Annexes which include exhaustive reporting of results for each algorithm.
13. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 18
Narrow public claim
The figure shows a notional face recognition pipeline consisting of a capture subsystem, primarily a camera, followed by a presentation attack detection (PAD) module intended to detect impersonation attempts, a quality acceptance (QA) step aimed at checking portrait standard compliance, then the recognition components of feature extraction and 1:1 or 1:N comparison, the output of which may prompt human involvement.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 18- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t02_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The figure shows a notional face recognition pipeline consisting of a capture subsystem, primarily a camera, followed by a presentation attack detection (PAD) module intended to detect impersonation attempts, a quality acceptance (QA) step aimed at checking portrait standard compliance, then the recognition components of feature extraction and 1:1 or 1:N comparison, the output of which may prompt human involvement.
14. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 18
Narrow public claim
A poor image can undermine detection or recognition, and it is possible that certain demographics yield photographs ill-suited to face recognition e.g. young children [28], or very tall individuals.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 18- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t02_primary_claims.json --source-file <local-raw-pdf>Exact source quote
A poor image can undermine detection or recognition, and it is possible that certain demographics yield photographs ill-suited to face recognition e.g. young children [28], or very tall individuals.
15. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 19
Narrow public claim
Relevant here is that it produced a better algorithm 2, and examined whether skin tone itself drove gender classification accuracy [30, 31] - in short, “skin type by itself has a minimal effect on the classification decision”.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 19- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t02_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Relevant here is that it produced a better algorithm 2, and examined whether skin tone itself drove gender classification accuracy [30, 31] - in short, “skin type by itself has a minimal effect on the classification decision”.
16. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 19
Narrow public claim
Indeed over the last five years there have been great advances in this respect due to the adoption of deep convolutional neural networks which demonstrate remarkable tolerance to very sub-standard photographs i.e. those that deviate from formal portrait standards most prominently ISO/IEC 39794-5 and its law-enforcement equivalent ANSI/NIST ITL 1-2017.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 19- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t02_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Indeed over the last five years there have been great advances in this respect due to the adoption of deep convolutional neural networks which demonstrate remarkable tolerance to very sub-standard photographs i.e. those that deviate from formal portrait standards most prominently ISO/IEC 39794-5 and its law-enforcement equivalent ANSI/NIST ITL 1-2017.
17. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 20
Narrow public claim
7See recent results for verification algorithms in the FRVT reports, and for identification algorithms in NIST Interagency Report 8271 [17].
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 20- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t02_primary_claims.json --source-file <local-raw-pdf>Exact source quote
7See recent results for verification algorithms in the FRVT reports, and for identification algorithms in NIST Interagency Report 8271 [17].
18. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 22
Narrow public claim
The study found lower similarity scores in those identifying as Black or African American, comporting with [22] but contrary to the best ageing study [3].
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 22- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t02_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The study found lower similarity scores in those identifying as Black or African American, comporting with [22] but contrary to the best ageing study [3].
19. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 22
Narrow public claim
The study should be discounted for two reasons: First the algorithms give high error rates at very modest false match rates: the best FNMR = 0.06 at FMR = 0.01.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 22- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t02_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The study should be discounted for two reasons: First the algorithms give high error rates at very modest false match rates: the best FNMR = 0.06 at FMR = 0.01.
20. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 23
Narrow public claim
The ISO/IEC 19795-1 performance testing and reporting standard requires different metrics to be reported for identification and verification implementations.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 23- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t02_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The ISO/IEC 19795-1 performance testing and reporting standard requires different metrics to be reported for identification and verification implementations.
21. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 25
Narrow public claim
If we express that as decrease of TPIR from, say 98.5% to 97%, then we mentally have to invert the scale to see a doubling in costs.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 25- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t03_primary_claims.json --source-file <local-raw-pdf>Exact source quote
If we express that as decrease of TPIR from, say 98.5% to 97%, then we mentally have to invert the scale to see a doubling in costs.
22. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 25
Narrow public claim
More subtly, readers don’t perceive differences in numbers near 100% well, becoming inured to the “high nineties” effect where numbers close to 100 are perceived indifferently.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 25- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t03_primary_claims.json --source-file <local-raw-pdf>Exact source quote
More subtly, readers don’t perceive differences in numbers near 100% well, becoming inured to the “high nineties” effect where numbers close to 100 are perceived indifferently.
23. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 30
Narrow public claim
Ltd meiya-001 Zhuhai Yisheng Electronics Technology yisheng-004 yisheng-0 iQIYI Inc iqface-000 iSAP Solution Corporation isap-001 Table 6: Algorithms evaluated in this report.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 30- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t03_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Ltd meiya-001 Zhuhai Yisheng Electronics Technology yisheng-004 yisheng-0 iQIYI Inc iqface-000 iSAP Solution Corporation isap-001 Table 6: Algorithms evaluated in this report.
24. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 32
Narrow public claim
CNN model #OutputProvided to1ScoreSystem logs2DecisionUser Output is a similarity score, S Access ControlNon-repudiationRoleAfford access of a person to a physical or logical resource.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 32- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t03_primary_claims.json --source-file <local-raw-pdf>Exact source quote
CNN model #OutputProvided to1ScoreSystem logs2DecisionUser Output is a similarity score, S Access ControlNon-repudiationRoleAfford access of a person to a physical or logical resource.
25. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 33
Narrow public claim
Same country and age” the mean is taken over 24 within-country times 5 within age-group times 4 within and cross sex FMR estimates, i.e.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 33- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t03_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Same country and age” the mean is taken over 24 within-country times 5 within age-group times 4 within and cross sex FMR estimates, i.e.
26. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 33
Narrow public claim
The vertical line shows a nominal FMR value of 0.00003 obtained by setting the threshold on randomly associated i.e. zero-effort pairs of mugshot images.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 33- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t03_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The vertical line shows a nominal FMR value of 0.00003 obtained by setting the threshold on randomly associated i.e. zero-effort pairs of mugshot images.
27. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 34
Narrow public claim
This re-iterates the results of the previous section, and shows it applies globally. ⊿ Country-of-birth matters: For many of the different levels of demographic pairing there is between one and two orders of magnitude between the 24 countries represented in this dataset.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 34- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t03_primary_claims.json --source-file <local-raw-pdf>Exact source quote
This re-iterates the results of the previous section, and shows it applies globally. ⊿ Country-of-birth matters: For many of the different levels of demographic pairing there is between one and two orders of magnitude between the 24 countries represented in this dataset.
28. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 34
Narrow public claim
For example when imposters are from the same sex and country but of any age, the algorithm gives FMR of 0.000046 on Polish faces and 0.0024 on Vietnamese, a fifty fold increase. ⊿ Regions with highest and lowest FMR: Across algorithms often the lowest FMR is observed in Eastern European populations and the highest in East Asian populations.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 34- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t03_primary_claims.json --source-file <local-raw-pdf>Exact source quote
For example when imposters are from the same sex and country but of any age, the algorithm gives FMR of 0.000046 on Polish faces and 0.0024 on Vietnamese, a fifty fold increase. ⊿ Regions with highest and lowest FMR: Across algorithms often the lowest FMR is observed in Eastern European populations and the highest in East Asian populations.
29. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 35
Narrow public claim
For this reason NIST has been reporting “matched-covariate” accuracy results in its FRVT evaluation of face verification algorithms [16].
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 35- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t03_primary_claims.json --source-file <local-raw-pdf>Exact source quote
For this reason NIST has been reporting “matched-covariate” accuracy results in its FRVT evaluation of face verification algorithms [16].
30. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 35
Narrow public claim
Thresholds increase because they are determined from non-mate scores, s, via the quantile function Q, as that value, T , which gives a proportion, FMR, at or above threshold: T = Q(s, 1− FMR) (9) and the set of demographically matched scores is smaller than if all possible comparisons is used.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 35- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t03_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Thresholds increase because they are determined from non-mate scores, s, via the quantile function Q, as that value, T , which gives a proportion, FMR, at or above threshold: T = Q(s, 1− FMR) (9) and the set of demographically matched scores is smaller than if all possible comparisons is used.
31. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 37
Narrow public claim
The thresholds were computed over a set of 93 070 400 imposter comparisons made using a different set of images, namely the law enforcement mugshots detailed in Annex 1 .
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 37- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t04_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The thresholds were computed over a set of 93 070 400 imposter comparisons made using a different set of images, namely the law enforcement mugshots detailed in Annex 1 .
32. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 37
Narrow public claim
Given many comparisons with the same demographic pairing, we can produce a measurement of FMR when comparing individuals from two demographic groups, for example Polish men over the age of 65 with Mexican women between 20 and 35.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 37- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t04_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Given many comparisons with the same demographic pairing, we can produce a measurement of FMR when comparing individuals from two demographic groups, for example Polish men over the age of 65 with Mexican women between 20 and 35.
33. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 41
Narrow public claim
When viewed on a computer the figure has very high resolution and zooming in reveals full detail; when printed it will likely just show coarse trends. ⊿ Nominal FMR in Eastern Europe: For many algorithms, FMR within Eastern Europe is close to the nominal target false match rate i.e. a grey color,−5≤ log10 FMR≤− 4.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 41- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t04_primary_claims.json --source-file <local-raw-pdf>Exact source quote
When viewed on a computer the figure has very high resolution and zooming in reveals full detail; when printed it will likely just show coarse trends. ⊿ Nominal FMR in Eastern Europe: For many algorithms, FMR within Eastern Europe is close to the nominal target false match rate i.e. a grey color,−5≤ log10 FMR≤− 4.
34. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 41
Narrow public claim
The sample size may be small for this study, but not absolutely small: The Somalia-Somalia FMR measurement is obtained from 1 733 116 comparisons involving 2632 images of 1974 males.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 41- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t04_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The sample size may be small for this study, but not absolutely small: The Somalia-Somalia FMR measurement is obtained from 1 733 116 comparisons involving 2632 images of 1974 males.
35. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 43
Narrow public claim
That is, the log10 FMR values are +2 higher corresponding to FMR that is of order 100 times larger than the de-facto baseline.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 43- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t04_primary_claims.json --source-file <local-raw-pdf>Exact source quote
That is, the log10 FMR values are +2 higher corresponding to FMR that is of order 100 times larger than the de-facto baseline.
36. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 43
Narrow public claim
A shift of 1.4 on the logarithmic scale corresponds to a factor of 25 increase, for example. ⊿ Anomalies in the figures: The cross-country heatmaps for the SIAT-004, Panasonic PSL-001, and Sensetimealgorithms are mostly red, indicating high false match rates for all comparisons.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 43- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t04_primary_claims.json --source-file <local-raw-pdf>Exact source quote
A shift of 1.4 on the logarithmic scale corresponds to a factor of 25 increase, for example. ⊿ Anomalies in the figures: The cross-country heatmaps for the SIAT-004, Panasonic PSL-001, and Sensetimealgorithms are mostly red, indicating high false match rates for all comparisons.
37. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 49
Narrow public claim
Indian M−Asian M−Black M−White Demographics of enrollee Demographics of impostor −6 −5 −4 −3 −2 −1 0 Dataset: MUGSHOTS log10 FMR Figure 12: For mugshot photos tagged with one of four race labels and a sex label, the heatmaps show false positive rates for comparison of randomly selected photos from the groups identified in the respective rows and columns.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 49- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t04_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Indian M−Asian M−Black M−White Demographics of enrollee Demographics of impostor −6 −5 −4 −3 −2 −1 0 Dataset: MUGSHOTS log10 FMR Figure 12: For mugshot photos tagged with one of four race labels and a sex label, the heatmaps show false positive rates for comparison of randomly selected photos from the groups identified in the respective rows and columns.
38. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 50
Narrow public claim
The magnitude of this difference is lower with mugshots than with application photos. ⊿ Highest FMR in American Indians: First, the highest FMR occurs in images of American Indians 11.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 50- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t04_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The magnitude of this difference is lower with mugshots than with application photos. ⊿ Highest FMR in American Indians: First, the highest FMR occurs in images of American Indians 11.
39. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 50
Narrow public claim
For the Imperial-002 algorithm featured in Figure 12 the FMR for American Indian women is 0.0068, i.e. a fold increase over the FMR of 0.0001 in white males.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 50- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t04_primary_claims.json --source-file <local-raw-pdf>Exact source quote
For the Imperial-002 algorithm featured in Figure 12 the FMR for American Indian women is 0.0068, i.e. a fold increase over the FMR of 0.0001 in white males.
40. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 52
Narrow public claim
The threshold was computed over a set of 93 070 400 imposter comparisons made using a different set of images, namely the mugshots detailed in Annex 1 .
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 52- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t04_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The threshold was computed over a set of 93 070 400 imposter comparisons made using a different set of images, namely the mugshots detailed in Annex 1 .
41. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 54
Narrow public claim
This, obviously, is an aggregate result; it will generally be possible to find some individuals from different age groups who produce high imposter scores but this will be increasingly difficult as the age difference increases. ⊿ Highest FMR in the oldest age group: For women from all most countries, comparison of images of individuals in the 65-and-over age group produce the highest false match rates.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 54- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t05_primary_claims.json --source-file <local-raw-pdf>Exact source quote
This, obviously, is an aggregate result; it will generally be possible to find some individuals from different age groups who produce high imposter scores but this will be increasingly difficult as the age difference increases. ⊿ Highest FMR in the oldest age group: For women from all most countries, comparison of images of individuals in the 65-and-over age group produce the highest false match rates.
42. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 54
Narrow public claim
For men this is often true also. ⊿ High FMR in the youngest age group: For both sexes, but men in particular, comparison of images of persons in the12−20 age group produce high false match rates.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 54- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t05_primary_claims.json --source-file <local-raw-pdf>Exact source quote
For men this is often true also. ⊿ High FMR in the youngest age group: For both sexes, but men in particular, comparison of images of persons in the12−20 age group produce high false match rates.
43. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 57
Narrow public claim
The line within each box is the median over those algorithms; the box itself spans the interquartile range (26 algorithms) and the lines here extend to minimum and maximum values.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 57- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t05_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The line within each box is the median over those algorithms; the box itself spans the interquartile range (26 algorithms) and the lines here extend to minimum and maximum values.
44. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 57
Narrow public claim
We make the following observations. ⊿ FNMR is absolutely low: In one-to-one verification of mugshots, the best algorithms give FNMR below 0.5% at the reasonably stringent FMR criterion of 0.00001.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 57- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t05_primary_claims.json --source-file <local-raw-pdf>Exact source quote
We make the following observations. ⊿ FNMR is absolutely low: In one-to-one verification of mugshots, the best algorithms give FNMR below 0.5% at the reasonably stringent FMR criterion of 0.00001.
45. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 59
Narrow public claim
Indian Asian Black White Figure 19: For one algorithm verifying mugshot images, the error tradeoff characteristics show false non-match vs. false match rates.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 59- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t05_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Indian Asian Black White Figure 19: For one algorithm verifying mugshot images, the error tradeoff characteristics show false non-match vs. false match rates.
46. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 59
Narrow public claim
Note that this is a marginal effect - perhaps 98% of women are still correctly verified - so the effect is confined to fewer than 2% of comparisons where algorithms fail to verify.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 59- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t05_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Note that this is a marginal effect - perhaps 98% of women are still correctly verified - so the effect is confined to fewer than 2% of comparisons where algorithms fail to verify.
47. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 61
Narrow public claim
The columns of the figure are sorted in order of mean FNMR from the 50 most accurate algorithms - this statistic was chosen so that high FNMR estimates from poor algorithms did not skew the results.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 61- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t05_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The columns of the figure are sorted in order of mean FNMR from the 50 most accurate algorithms - this statistic was chosen so that high FNMR estimates from poor algorithms did not skew the results.
48. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 61
Narrow public claim
For example, the Visionlabs-007 algorithm has outliers only for Liberian and Somali women under the age of 45, for whom FNMR is below 1.4%. • Lower variation across countries: For the more accurate algorithms, false non-match rates generally range by a factor of two or three from the left side of Figure 22 to the right i.e.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 61- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t05_primary_claims.json --source-file <local-raw-pdf>Exact source quote
For example, the Visionlabs-007 algorithm has outliers only for Liberian and Somali women under the age of 45, for whom FNMR is below 1.4%. • Lower variation across countries: For the more accurate algorithms, false non-match rates generally range by a factor of two or three from the left side of Figure 22 to the right i.e.
49. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 63
Narrow public claim
The FNMR value is the mean over men/women and over/under age 45, so represents FNMR in situations where those four populations were balanced.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 63- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t05_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The FNMR value is the mean over men/women and over/under age 45, so represents FNMR in situations where those four populations were balanced.
50. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 64
Narrow public claim
They were conceived of to isolate specific demographic factors as follows. ⊿ Sex: We construct a gallery containing 800 000 white men, and 800 000 white women, aged 20 - 40.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 64- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t05_primary_claims.json --source-file <local-raw-pdf>Exact source quote
They were conceived of to isolate specific demographic factors as follows. ⊿ Sex: We construct a gallery containing 800 000 white men, and 800 000 white women, aged 20 - 40.
51. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 64
Narrow public claim
The general metric FNIR (N, R, T) covers both as follows: ⊿ Investigation: For investigators willing to traverse long candidate lists in pursuit of a lead, the metric FNIR(N, R, 0) is the proportion of missed mates when searching an N-enrollee gallery and considering the R most similar candidates without applying a threshold (T = 0).
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 64- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t06_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The general metric FNIR (N, R, T) covers both as follows: ⊿ Investigation: For investigators willing to traverse long candidate lists in pursuit of a lead, the metric FNIR(N, R, 0) is the proportion of missed mates when searching an N-enrollee gallery and considering the R most similar candidates without applying a threshold (T = 0).
52. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 65
Narrow public claim
3.2 and Table 12, 14, 16 IDRankScoreUsually correct response is empty list as most searches are non-mated IDRankScorePat13.142Usually the correct response is for one entry as most searches will be mated Figure 23: Identification applications and relevant metrics.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 65- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t06_primary_claims.json --source-file <local-raw-pdf>Exact source quote
3.2 and Table 12, 14, 16 IDRankScoreUsually correct response is empty list as most searches are non-mated IDRankScorePat13.142Usually the correct response is for one entry as most searches will be mated Figure 23: Identification applications and relevant metrics.
53. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 66
Narrow public claim
Given well documented twinning rates14 [27], we expect twins to be in the data, both identical and, more commonly, fraternal.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 66- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t06_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Given well documented twinning rates14 [27], we expect twins to be in the data, both identical and, more commonly, fraternal.
54. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 66
Narrow public claim
Siblings will be expected to give elevated similarities along the same lines. ⊿ Higher false positive identification rates in black women: The lines connecting points of fixed threshold are often long and slanted in the error tradeoff plots in the center column of the bottom row - see Figure 24, for example.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 66- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t06_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Siblings will be expected to give elevated similarities along the same lines. ⊿ Higher false positive identification rates in black women: The lines connecting points of fixed threshold are often long and slanted in the error tradeoff plots in the center column of the bottom row - see Figure 24, for example.
55. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 67
Narrow public claim
Female Male Figure 24: For mugshot identification, the top row shows false negative identification “miss” rates as a function of rank, a metric appropriate to investigators traversing candidate lists for lead generation.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 67- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t06_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Female Male Figure 24: For mugshot identification, the top row shows false negative identification “miss” rates as a function of rank, a metric appropriate to investigators traversing candidate lists for lead generation.
56. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 67
Narrow public claim
The four thresholds are chosen to give FPIR of {0.0003, 0.003, 0.03, 0.3} respectively for one baseline demographic, here white males.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 67- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t06_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The four thresholds are chosen to give FPIR of {0.0003, 0.003, 0.03, 0.3} respectively for one baseline demographic, here white males.
57. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 68
Narrow public claim
Female Male Figure 25: For mugshot identification, the top row shows false negative identification “miss” rates as a function of rank, a metric appropriate to investigators traversing candidate lists for lead generation.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 68- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t06_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Female Male Figure 25: For mugshot identification, the top row shows false negative identification “miss” rates as a function of rank, a metric appropriate to investigators traversing candidate lists for lead generation.
58. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 69
Narrow public claim
It is typical in verification access control to target false match rates (FMR) between 0.00001 and 0.001, i.e. between one per hundred thousand and one per thousand.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 69- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t06_primary_claims.json --source-file <local-raw-pdf>Exact source quote
It is typical in verification access control to target false match rates (FMR) between 0.00001 and 0.001, i.e. between one per hundred thousand and one per thousand.
59. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 69
Narrow public claim
The result of that is returned in either of two ways: The system will return an operator-specified number of candidates, or it will return however many candidates are above an operatorspecified threshold 15.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 69- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t06_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The result of that is returned in either of two ways: The system will return an operator-specified number of candidates, or it will return however many candidates are above an operatorspecified threshold 15.
60. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 70
Narrow public claim
Contrast the behavior here with that in Figure 27 and the corresponding figures for developers Aware, Idemia, NEC, Tevian, and Toshiba that are included in Annex 17 . applications where the prior probability of a searched person actually being is in the database is low, e.g. cardsharp surveillance in a casino, or soccer hooligans at a sports game 16.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 70- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t06_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Contrast the behavior here with that in Figure 27 and the corresponding figures for developers Aware, Idemia, NEC, Tevian, and Toshiba that are included in Annex 17 . applications where the prior probability of a searched person actually being is in the database is low, e.g. cardsharp surveillance in a casino, or soccer hooligans at a sports game 16.
61. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 70
Narrow public claim
The lack of an effect on false positive identification rates is evident in Figure 25 where the grey lines join points of equal threshold.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 70- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t07_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The lack of an effect on false positive identification rates is evident in Figure 25 where the grey lines join points of equal threshold.
62. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 71
Narrow public claim
The uniformity of the scores across demographic groups is in contrast to that evident in Figure 26 and many others in the Annex 17 compendium.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 71- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t07_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The uniformity of the scores across demographic groups is in contrast to that evident in Figure 26 and many others in the Annex 17 compendium.
63. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 72
Narrow public claim
Over time this may influence the human, though one earlier study [12] looked at cognitive bias issues in the human review of fingerprint search results, without demographic effects, and found scant evidence that scores influence the reviewer.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 72- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t07_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Over time this may influence the human, though one earlier study [12] looked at cognitive bias issues in the human review of fingerprint search results, without demographic effects, and found scant evidence that scores influence the reviewer.
64. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 73
Narrow public claim
This motivates the following two research and development possibilities. ⊿ Improved standards compliance: The ISO/IEC 19794-5 standard includes requirements regulating geometry and exposure.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 73- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t07_primary_claims.json --source-file <local-raw-pdf>Exact source quote
This motivates the following two research and development possibilities. ⊿ Improved standards compliance: The ISO/IEC 19794-5 standard includes requirements regulating geometry and exposure.
65. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 73
Narrow public claim
The ISO/IEC SC 37 biometrics subcommittee has recently initiated work on quality (ISO/IEC 29794-5 and 24357). ⊿ Face-aware cameras: The same ISO/IEC committee has recently initiated work on specifications for capture subsystems that may require real-time face detection, pose estimation, and exposure measurement.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 73- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t07_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The ISO/IEC SC 37 biometrics subcommittee has recently initiated work on quality (ISO/IEC 29794-5 and 24357). ⊿ Face-aware cameras: The same ISO/IEC committee has recently initiated work on specifications for capture subsystems that may require real-time face detection, pose estimation, and exposure measurement.
66. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 74
Narrow public claim
This affirms a prior finding of an “other-race effect” for algorithms [33] though that paper did not separate false positive from false negative shifts.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 74- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t07_primary_claims.json --source-file <local-raw-pdf>Exact source quote
This affirms a prior finding of an “other-race effect” for algorithms [33] though that paper did not separate false positive from false negative shifts.
67. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 74
Narrow public claim
Similarities in faces in fraternal twins [17] are expected to extend also to siblings (which also share half of the genes), and then to more distant relatives.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 74- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t07_primary_claims.json --source-file <local-raw-pdf>Exact source quote
Similarities in faces in fraternal twins [17] are expected to extend also to siblings (which also share half of the genes), and then to more distant relatives.
68. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 75
Narrow public claim
The texture of the human iris is known to have a structure that when imaged and processed by published feature extraction algorithms [11, 29] will correctly discriminate between identical twins [40] - something that contemporary marketplace face algorithms do not [17].
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 75- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t07_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The texture of the human iris is known to have a structure that when imaged and processed by published feature extraction algorithms [11, 29] will correctly discriminate between identical twins [40] - something that contemporary marketplace face algorithms do not [17].
69. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 75
Narrow public claim
However genetics research [25] does show iris textures have some genetic linkage, so a better characterisation of the tails of the impostor distribution is needed, at least for large scale one-to-many identification.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 75- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t07_primary_claims.json --source-file <local-raw-pdf>Exact source quote
However genetics research [25] does show iris textures have some genetic linkage, so a better characterisation of the tails of the impostor distribution is needed, at least for large scale one-to-many identification.
70. NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects · PDF page 76
Narrow public claim
The explanation uses a graphical construct based on the error tradeoff characteristics shown in Figure 28. ⊿ We start with a target false match rate FMR POLICY that has been set to implement some security objective.
What this card proves
The exact quoted statement occurred in the named official PDF fetched on 2026-08-04, and the complete PDF matched the recorded SHA-256.
What this card does not prove
It does not independently establish the underlying model capability, safety, comparison, or real-world truth. This is a retrospective single-maintainer source-publication audit.
- Primary source
- NIST IR 8280 Face Recognition Vendor Test Part 3: Demographic Effects
- Section locator
PDF page 76- Captured
- 2026-08-04
- Source SHA-256
5eb60e39d34a4436121aa83fc8e8d6492c86636518407b1ebe08d87fdd94048c- Reproduction
- Maintainer run only; no independent operator rerun is registered.
Repeat this exact check locally
Download the PDF to a local file, then run the committed verifier. A changed PDF is reported as drift rather than silently accepted.
python scripts/verify_primary_public_claim_group.py artifacts/cb7k_dom008_t07_primary_claims.json --source-file <local-raw-pdf>Exact source quote
The explanation uses a graphical construct based on the error tradeoff characteristics shown in Figure 28. ⊿ We start with a target false match rate FMR POLICY that has been set to implement some security objective.