Swipe Facial Recognition
App • Ux • Ui • Artificial Intelligence • AWS
The Challenge
The world over visitors are asked to sign in at front desks when they arrive to visit an employee of a business. This process can be cumbersome at best, and frustrating at worst resulting in mounds of paper log books that need to be stored for many years.
Swipe's rapid prototyping team set itself the challenge to see if it could devise a cloud based solution that would ease the frustration from a user perspective and optimise the process for enterprise clients who require a record of visitors.
Hypothesis
According to a recent study by the National Institute for Standards and Technology (NIST), facial recognition technologies outperforms human facial recognition capabilities. The technology has been around for decades - in fact Woodrow Wilson Bledsoe an AI pioneer was experimenting with it in 1964 - however the usage has become more ubiquitous and accessible in the last few years as artificial intelligence and in-particular machine learning has improved.
Services such as AWS Rekognition makes it possible to add image and video analysis to applications with ease. Using our internal ideation and continuous development process the Swipe team brainstormed how to build a faster and more efficient cloud-based sign in solution that uses facial recognition to identify new and returning customers and in so doing eliminate the need for front desk legacy paper-based log books altogether.
Solution
Swipe Facial Recognition: a facial recognition sign-in system that uses AWS Rekognition and signs users in in less than a minute.
From a visitor's perspective, the sign in process is fast, intuitive and user-friendly. Powered by a native app installed on a device such as an Android tablet, the visitor presents him/herself to the screen upon arrival. Harnessing the onboard camera and edge machine learning, the app recognises that there is a face in view and initiates the recognition process to the cloud. If the user is a first-time visitor he/she only needs to complete the full sign in process once.
Once all the necessary fields (customisable) are completed, the user has the option to email him or herself a copy of the NDA upon acceptance. Once complete, he/she can select the name of the employee they are there to see and the employee will be notified that the visitor is at the front desk.
The next time the person returns to the premises, he/she only has to approach the tablet, wait for the “confirmation of recognition” screen (+-1 second), which includes a personal welcome, and then select who they are there to see. The recognition process takes less than 5 seconds and requires no human intervention.
Most importantly, the user’s data is secure and the user can ask to be “forgotten”. In so doing the system complies with GDPR data privacy regulations.
Architecture
The goal of the architecture was to keep the cost low and manageable. To reach these goals, the platform was built using a serverless architecture from the ground up and leveraging SaaS services offered by AWS wherever possible.
API Gateway serves as the REST API for both the mobile and web applications. Authentication is done with two authorisers, one custom authoriser for all resources used by the mobile app using JWT tokens, and one authoriser against Cognito for the web portal. API Gateway integrates with Lambda as the compute platform to process all the incoming requests and the Lambda functions are written in Golang.
Usage plans are leveraged on API Gateway to control access rate limits and quotas per application. Each application is assigned an API key which in turn is linked to a usage plan. API Gateway is also deployed on the edge to speed up latency to the API endpoints.
Data is stored using DynamoDB and access to DynamoDB is controlled with fine-grained IAM policies to allow each Lambda function only access to the tables it requires, and the tasks it needs to perform on a specific table. All dynamoDB tables are also encrypted at rest.
Rekognition is used to index and recognise faces from images sent from the mobile app. Images are sent directly to Rekognition tied with a unique user ID during training and recognition. The image data is not stored and is thrown away after training and/or recognition.
The infrastructure and Lambda function deployments are managed using Terraform and is deployed in a continuous delivery fashion. Application logs and infrastructure monitoring is done using CloudWatch.
App
Swipe iX set out to build an application that is fast, user-friendly, cost-effective and white label friendly.
The initial application is focused on Android tablets for their cost-effectiveness and is written as a native Android application. The application does not store any user information, however API access tokens are stored using Realm where encryption provided by Realm and the device key-store is leveraged to keep the tokens safe on the device.
The most prominent feature of the app is the camera interface. The camera interface is designed to always be active and there is no need for the user to initiate an image to be taken for recognition.
Machine learning is used on the device with a video pipeline to continuously monitor the camera feed for any faces entering the feed. This is done on the device to save bandwidth cost to process the video feed in the cloud, and for responsiveness. Once a face has been detected in the feed, an image is automatically taken and then sent to the platform to process. The recognition response is then used to navigate to the correct section of the sign-in process on the device depending on if the user was recognised or not.
Portal
The goal for the web portal was to be lightweight, to be served with no web servers and to be white label friendly. Polymer was chosen as the framework for the portal and was written as a progressive web app. The web components architecture offered by Polymer were leveraged to provide a flexible interface that is easily customise-able and style-able. The application is hosted on S3 behind a CloudFront distribution.
The portal uses Cognito for account and access management. Cognito groups are further used for access permissions on the portal.
Features
The product ultimately simplifies access management and lowers costs of existing business resources:
- Offering an affordable, secure, non-invasive and seamless sign in process.
- Collects, tracks and retrieves relevant visitor information e.g. review logs to see who entered the building on a specific date or time.
- Keeps a digital record of signed NDAs.
- Assigns and tracks visitor badges.
- Add customised notification preferences for employees e.g. an SMS, Slack or email integration.
- Secure administrator roles
Manage employee access:
- Assign permissions e.g. giving security and/or the reception staff access to a view in which they can see “active” visitor badges and log returned badges once a visitor exits add or remove a visitor badge.
- Register or deregister devices for multiple locations using a QR code.
- View analytics to reveal visitor trends.
Tools
Networking & Content Delivery
Compute
Database
Security Identity and Compliance
Storage
Developer Tools
Management & Goverance
Machine Learning
The Result
Swipe Facial Recognition is the first product released by the Swipe iX team and we are very proud of the result. Using the latest AI technology at our disposal, we built a light weight, cost effective tool that can be customised and white labeled according to a client's individual needs. As a white label product, its potential is endless. Even though the initial POC was designed to be deployed as a sign-in system at a reception desk, other use cases could include, but are not limited to:
Access control at :
- Gymnasiums e.g. Virgin Active
- Residential blocks
- Parking lots
- Member clubs
Recognition for:
- Loyalty programmes e.g. Vitality, PnP Smartshopper
- Roll call
- Employee management
- Event detection
- Crowd counting and more.
With the recognition process completed in less than 5 seconds and with all the requisite security requirements for the protection of Personal Identifiable Information in place, we look forward to taking this product from strength to strength.
M-Net Voting
Work with us
We collaborate with ambitious brands and people. We’d love to build something great together.
Get in touch
COPYRIGHT 2022 - SWIPE iX
ALL RIGHTS RESERVED