Fingerprint vendors trip over NIST test results

Federal government testing of digital fingerprint interoperability has evolved into an eligibility test for the federal employee identification card initiative, industry executives said.

And that much-broadened scope, made official only in the final three months of testing, has stirred some concerns about how to interpret the results.

The Minutiae Interoperability Exchange Test (Minex) large-scale testing was announced in August 2004 by the National Institute of Standards and Technology. Minex tested the compatibility of mathematical algorithms for fingerprint minutiae templates, which are computerized descriptions of fingerprint characteristics that indicate the position and orientation of ridges and other distinctive features.

Last month, Minex certified as interoperable 14 vendor products, including extraction and matching products from Bioscrypt Inc., Cogent Systems Inc., Dermalog Identification Systems GMBH and NEC Corp.

NIST also said, based on guidelines in its Special Publication 800-76 released in February, that the 14 products meet performance requirements for use in the federal identity card initiative under Homeland Security Presidential Directive-12.

The initiative is a government push to get IDs into the hands of government workers and contractors. Billions of dollars could be spent on the effort.

Overall, 15 vendors participated in the Minex, and eight earned certification for both interoperability and personal identity verification.

However, one of the best known biometric companies submitting algorithms to be tested, Identix Inc., did not receive a certification. The company has a $27 million blanket purchasing agreement to provide biometric services and equipment to the Homeland Security Department, and has sold $6 million in facial recognition biometric applications to the State Department.

Left behind

Several industry executives met with NIST officials April 27 to air concerns about the timing, scope and interpretation of the fingerprint minutiae tests, especially in their context as a qualification for meeting HSPD-12 requirements.

To set standards for products meeting the directive's goals, the Federal Information Processing Standard 201 was released in February 2005. Addition of the personal identity verification certification followed Special Publication 800-76 in February 2006.

The Minex tests, initially conducted for a specific purpose, have evolved into a broader pre-qualification of vendors for facets of the federal personal identity cards to be issued, starting in October, for as many as 12 million workers.

That has caused some headaches for vendors that did not participate in Minex, because they did not realize the eventual broad scope of the program, which developed at a late stage of the testing, said Walter Hamilton, chairman of the International Biometric Industry Association.

"Companies that submitted [to the Minex] didn't know the criticality of the submission," Hamilton said. "They did not realize then the importance of participation. If they had, then some companies that did not participate would have done so.

"The NIST testing was pre-FIPS 201," Hamilton said. "Never was it represented that the [NIST] tests could get a company on the magical list of products for FIPS-201, but that is exactly what has happened."

NIST has acted to remedy the situation and to let more vendors participate, Hamilton said. Shortly before the April 27 meeting, NIST announced it would be conducting ongoing tests of vendor products for Minex and PIV certifications. A NIST official did not respond to a phone call requesting comment.

"NIST has addressed most of our concerns," Hamilton said.

An Identix spokesman said the company intends to resubmit a new algorithm.
The algorithm that failed to win NIST certification was "older" and "not one of our top performers," Identix spokesman Damon Wright said.

"I don't believe the [NIST] list is final in any way, shape or form," he said. "We are in the process of submitting our latest BioEngine algorithm, and we feel very confident it will be certified."

Identix has reason to be optimistic. In March, in a separate test of accuracy, NIST identified BioEngine as one of three top-performing algorithms of 21 tested.

All in the timing

Another point raised at the industry meeting with NIST was concern about the timing of the tests.

The Minex testing, while effective, was "a snapshot in time and is now dated," states an April 26 document, "The International Biometric Industry Association Perspective," circulated among industry officials and NIST.

The algorithms tested in Minex were for a relatively new standard: the International Committee for Information Technology Standards' 378 fingerprint template standard.

"It was inevitable that there would be some difficulties in vendor interpretation and implementation to the new standard," according to the document. "Therefore, it is important that reviewers of the Minex test report, not interpret, the results as reflective of today's vendor performance."

A third area of concern discussed at the meeting was NIST's apparent recommendation to use two fingerprints for greatest accuracy.

In announcing its Minex results, NIST said that standardized digital formats for fingerprint minutiae don't perform as well in matching fingerprints as do proprietary formats. Some of the reduced accuracy in the standard minutiae templates can be compensated for by matching two fingerprints instead of one, the institute said.
But the biometric industry group said using two fingerprints is impractical, inconvenient and unnecessary in real-world applications.

"If you are holding a coffee cup in one hand, and a briefcase in the other hand, how are you going to do two fingers from two separate hands?" Hamilton asked.

Furthermore, the industry group recommended that NIST use operational testing in real-life kinds of scenarios to do further tests to get the most accurate measure of how the technology would perform in the field.

For one thing, in an analogy to a real-life scenario, if their first attempt did not work, most people would try two or three times to place a dollar bill in a vending machine properly, the document said. In the NIST tests, if the first attempt for a fingerprint match failed, it was considered a complete failure.

"Scenario, or transaction-based test protocols that more closely mimic operational environments, measure accuracy based on multiple match attempts and from different enrolled fingers," according to the association's document.

Because of the manner in which NIST conducted the Minex test, "it was not possible for NIST to simulate or calculate false rejections based on two or three attempts since those images were not available," the document said.

Staff Writer Alice Lipowicz can be reached at alipowicz@postnewsweektech.com.