Identity confirmed

Armin Reuter explains how D4FLY has been exploring the border controls of the future

An increasingly connected world with rising passenger numbers at airports requires innovative approaches for a comfortable travel experience with the highest security standards. The D4FLY research project opens the gates to border controls and the digital traveller journey of tomorrow.
In addition to the many benefits that freedom of travel brings, it also creates a growing number of security risks that authorities must respond to. While state actors need to know very precisely who is intending to cross their borders, passengers increasingly expect to experience less hassle at either end of a journey, and for their trips to be optimised and personalised to their needs.
A decisive factor here is the verification of identity documents and a person’s identity by border control authorities. The D4Fly research project, funded by the EU, focused on new technological solutions for efficient verification of travel documents and identities using advanced biometric technologies. The combination of these technologies aimed towards an acceleration of border controls while maintaining highest security levels, thus helping authorities to meet those challenges.
The project, launched in 2019, with Veridos as the overall coordinator, brought together a total of 19 different partners from 11 countries, including research institutes, universities, border control authorities and industrial companies. In August 2022, the consortium declared D4FLY successfully completed. The technologies developed and analysed in the project come with different maturity levels. Some are already in use on end-user side, some are close to productisation and some still require some basic research activities. Overall, the partners will continue the development on almost all of the project’s results.
Producing travel documents that are as forgery-proof as possible comes with many challenges, where a core one is to design, select and implement a set of security elements that are very hard to forge on one hand, but relatively easy to verify on the other. The complexity of the verification process increases with the many innovative security features in use. Different printing technologies, special colours and highly sophisticated optical effects sometimes also require technological upgrades on the part of the border control – especially since counterfeiters also have a wide variety of tools at their disposal for manipulation. Automated document verification technologies, which have been developed in the D4FLY project, help border officers to verify security elements and detect potential document fraud.
As the image in any ID document is the most prevalent means to check a person’s identity for a human as it constitutes the basis of the face recognition technology used in automated systems like eGates, it is very important that the images in passports are authentic. For example, two faces can be digitally combined into one without much effort using image morphing techniques. The manipulated passport picture thus contains the biometric features of two people who can use the same document. Closing this security gap was also a work package in D4FLY, where new software solutions were developed to detect such manipulations in passport images.
In general, border controls and especially the processes for establishing identity and verifying documents are becoming increasingly digital. Recent challenges, such as the Covid-19 pandemic, demand further developments in these areas. To not only maintain but also increase security standards, sophisticated technologies are needed to help officials to manage borders more effectively, even in challenging times. New solutions, like biometric technologies in particular have proven to be effective but still have a large potential to be further improved especially in a border check context.
One of the core problems the D4FLY project sought to solve was guiding travellers through the necessary border controls, including identity checks, in the shortest possible time. For spatially limited locations, such as airports or cruise ship terminals, a so-called biometric corridor can be a suitable solution, which uses biometric technologies to identify the travellers while they move through a specific area. The technologies that were researched in D4FLY were 3D facial recognition, iris verification and somatotype analyses, which uses a full body image to match a reference image against a live image.
In the future, biometric corridors will guarantee border controls ‘on the move’; passengers will no longer have to wait in long queues. The traveller will be able to dispense with several points of contact with staff and authorities. A prerequisite would be that they voluntarily enrol their reference data, which the system compares with the images acquired in the corridor. The D4FLY consortium developed a kiosk prototype for this purpose, through which biometric and biographic data can be captured.
The self-service offering is an important aspect of modern travel for passengers. The easier it is for passengers to submit their personal data in advance, the more are ready to use those services and the less waiting time this process will take further down the line. To this end, approaches such as e-kiosks in airports have proved to be highly successful already today. Therefore, the approach was also integrated into the D4Fly project.
Here, too, technologies that can detect possible attempts for manipulation and enrolment of false identities have been developed. While there has been great progress in facial recognition in general, and this approach is considered to be quite mature, “iris on the move” solutions are still challenging. The specific challenge here is to capture an image of the eye well enough at a great distance and in motion. As the iris biometric is a very differentiating one, which also hardly changes over the human lifetime, it is desirable to be able to use this biometric as easily and effectively as face recognition. It is therefore ideally suited as a consistent and undoubtedly assignable biometric feature.
Progress is also being made in the field of 3D face recognition. In a 3D image there is much more information captured than in a classic 2D image, therefore one of the promises of 3D face recognition technology is to reach an even higher level of accuracy. On the other hand, a 3D face image is more challenging to acquire, especially if an object is moving. In the D4FLY project, good results have been achieved combining innovative light field cameras from project partner Raytrix, which can capture a 3D image in a single shot, and newly developed matching algorithms, based on neural networks.
Research on these innovative technologies paves the way to contactless border controls, shorter waiting times and secure identity checks – the ultimate goals of biometric corridors.
Border controls do not always take place under the optimal and controlled conditions like those offered at an airport. Border officials often have to check people’s identities on cramped and poorly lit buses or trains. The D4FLY project investigated solutions for these scenarios. A smartphone app was developed, which officers can use to verify the identity of passengers within buses, without the passengers having to leave the vehicle. The processing can thus be carried out much more efficiently and the entire check can be completed quicker.
Here, too, the prerequisite is that passengers register beforehand, scan their passports and take photos. While the system transmits the required information to the border official’s smartphone ahead of the passengers’ arrival, they only have to take live pictures of the passengers to match the data and confirm their identities. In the process, the app can also display further information on the person, such as the passport data entered.
Advancing new solutions in the area of border control is important in several respects. On the one hand, new technologies can significantly simplify travel for passengers and provide greater convenience. On the other, they support border authorities in their difficult tasks, such as checking an increasing number of travellers in an acceptable period of time. Above all, border controls based on advanced technologies help make borders more secure and detect illegal activities such as forged documents or identity theft at an early stage.
Neural networks and machine learning are already in use to implement these solutions, which require large amounts of data for highly accurate results. The standards for collecting and using this data are currently widely discussed and an important topic in Europe, also because different countries apply different data protection laws. Europe is also one of the regions with the strictest legal regulations in that regard.
Therefore, not only was the correct handling of the data and the compliance to the applicable data protection regulations constantly monitored over the course of the project, but special workshops were conducted in the course of the D4FLY project to assess the potential impact of deployment of these types of automated systems for border control, taking into account ethical, societal and legal perspectives.
One thing became clear in the process: in order to continuously develop new technologies, close cooperation between research institutes, universities, industry and players from the real world is required. Only in this way can research results be brought together with the needs of border officials, as was successfully done in the D4FLY project.

Armin Reuter is Director of Innovation Projects at Veridos.