MaixPy #16: Maixduino Face Recognition

October 30, 2022 Tiago Maixduino Series

MaixPy #16: Maixduino Face Recognition | AI MicroPython

Table of Contents

A facial recognition system is a technology capable of matching a human face from a digital image or a video frame against a database of faces. Such a system is typically employed to authenticate users through ID verification services, and works by pinpointing and measuring facial features from a given image.

Since their inception, facial recognition systems have seen wider uses in recent times on smartphones and in other forms of technology, such as robotics. Because computerized facial recognition involves the measurement of a human’s physiological characteristics, facial recognition systems are categorized as biometrics. Although the accuracy of facial recognition systems as a biometric technology is lower than iris recognition and fingerprint recognition, it is widely adopted due to its contactless process like:

Advanced human–computer interaction
Video surveillance
Automatic indexing of images

Their effectiveness varies, and some systems have previously been scrapped because of their ineffectiveness. The use of facial recognition systems has also raised controversy, with claims that the systems violate citizens’ privacy, commonly make incorrect identifications, encourage gender norms and racial profiling, and do not protect important biometric data. The appearance of synthetic media such as deepfakes has also raised concerns about its security.

The facial recognition system analyzes a face image. It maps and reads face geometry. Identifies key facial landmarks to distinguish a face from other objects. Facial recognition technology often looks for:

Distance between eyes
Distance from forehead to chin
Distance between nose and mouth
Depth of eye lenses
Cheekbones shape
Lip, ear and chin contour

Then, the system converts the facial recognition data into a sequence of numbers or dots called a face print. Each person has a unique facial print, similar to a fingerprint. The information used by facial recognition can also be used in reverse, to digitally reconstruct someone’s face.

In this tutorial, I am going to deploy an AI model at the edge to detect and recognise faces in a live video feed.

What Will We Need

Maixduino development board: https://s.click.aliexpress.com/e/_DBd6fTF

MaixPy IDE: https://www.edgemicrotech.com/maixpy-1-maixduino-first-steps/

Maixduino Face Recognition Algorithm Steps

Step 1 – Detect the face in our video stream

It is basicly the Face detection algoritm, which uses the same model as the previous Maixduino Face Detection post (Link)

Step 2 – Cut the face out of the video feed to a standart picture format and align it to the front position

Cut out the face, find the eyes, nose and mouth of the face, to a picture of 128x128
Rotate the face in the face image to the standard position

Step 3 – Extract information from the face and store it

Use the feature extraction model to extract the feature values of the face: Position of the eyes, nose and mouth.
Store the face information to be used after

Step 4 – Then just the algoritm and try to recognize some faces

Maixduino Face Recognition (Extra Step)

If you have had problems trying to get this to work, we can use the method when everything fails, erase the data from the chip and start fresh. Go to this page link to make the procedure: https://www.edgemicrotech.com/maixpy-15-maixduino-erase-the-chip-data-micropython/

Maixduino Face Recognition Files

Just get them from Maixhub or in the links in end of the post page (Maixduino Experiment Materials).

Two ways to make this work (The changes can be made in the main MaixPy code):

Copy the files to the memory space(s) of the Maixduino chip (I am going to use this method)
Copy the files to the Sdcard

Upload [FaceDetection.smodel] to the [0x300000] memory address

Upload [FaceLandmarkDetection.smodel] to the [0x400000] memory address

Upload [FeatureExtraction.smodel] to the [0x500000] memory address

Upload the firmware [maixpy_v0.6.2_84_g8fcd84a58_openmv_kmodel_v4_with_ide_support.bin] to the [0x00000] memory address

Maixduino Face Recognition Code

import sensor
import image
import lcd
import KPU as kpu
import time
from Maix import FPIOA, GPIO
import gc
from fpioa_manager import fm
from board import board_info
import utime

fm.register(board_info.LED_R, fm.fpioa.GPIO0, force=True)
fm.register(board_info.LED_G, fm.fpioa.GPIO1, force=True)
led_r = GPIO(GPIO.GPIO0, GPIO.OUT)
led_g = GPIO(GPIO.GPIO1, GPIO.OUT)
led_r.value(1)
led_g.value(1)

#Load the AI models from Memory or SD Card

#Face Detection model
task_fd = kpu.load(0x300000)
#Facial Features model (Left eye, Right eye, Nose, Left mouth corner, Right mouth corner)
task_ld = kpu.load(0x400000)
#Facial Features model 192 Dimensions
task_fe = kpu.load(0x500000)

#Load the AI models from SD Card
#task_fd = kpu.load("/sd/FaceDetection.smodel")
#task_ld = kpu.load("/sd/FaceLandmarkDetection.smodel")
#task_fe = kpu.load("/sd/FeatureExtraction.smodel")

#To get the frame rate
clock = time.clock()

#Setup the Boot Button function to be used to store the face recogniton parameters
fm.register(board_info.BOOT_KEY, fm.fpioa.GPIOHS0)
key_gpio = GPIO(GPIO.GPIOHS0, GPIO.IN)
start_processing = False
BOUNCE_PROTECTION = 50
def set_key_state(*_):
global start_processing
start_processing = True
utime.sleep_ms(BOUNCE_PROTECTION)
key_gpio.irq(set_key_state, GPIO.IRQ_RISING, GPIO.WAKEUP_NOT_SUPPORT)

#Set up the LCD and Camera
lcd.init()
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.set_hmirror(1)
sensor.set_vflip(1)
sensor.run(1)

#Create variables to store stuff
#Anchor for face detect
anchor = (1.889, 2.5245, 2.9465, 3.94056, 3.99987, 5.3658, 5.155437,
6.92275, 6.718375, 9.01025)
#Standard face key point position
dst_point = [(44, 59), (84, 59), (64, 82), (47, 105),
(81, 105)]
a = kpu.init_yolo2(task_fd, 0.5, 0.3, 5, anchor)
img_lcd = image.Image()
img_face = image.Image(size=(128, 128))
a = img_face.pix_to_ai()
record_ftr = []
record_ftrs = []
names = ['Mr.1', 'Mr.2', 'Mr.3', 'Mr.4', 'Mr.5',
'Mr.6', 'Mr.7', 'Mr.8', 'Mr.9', 'Mr.10']

ACCURACY = 85

#Main loop
while (1):
img = sensor.snapshot()
clock.tick()
code = kpu.run_yolo2(task_fd, img)
if code:
for i in code:
# Cut face and resize to 128x128
a = img.draw_rectangle(i.rect())
face_cut = img.cut(i.x(), i.y(), i.w(), i.h())
face_cut_128 = face_cut.resize(128, 128)
a = face_cut_128.pix_to_ai()
# a = img.draw_image(face_cut_128, (0,0))
# Landmark for face 5 points
fmap = kpu.forward(task_ld, face_cut_128)
plist = fmap[:]
le = (i.x() + int(plist[0] * i.w() - 10), i.y() + int(plist[1] * i.h()))
re = (i.x() + int(plist[2] * i.w()), i.y() + int(plist[3] * i.h()))
nose = (i.x() + int(plist[4] * i.w()), i.y() + int(plist[5] * i.h()))
lm = (i.x() + int(plist[6] * i.w()), i.y() + int(plist[7] * i.h()))
rm = (i.x() + int(plist[8] * i.w()), i.y() + int(plist[9] * i.h()))
a = img.draw_circle(le[0], le[1], 4)
a = img.draw_circle(re[0], re[1], 4)
a = img.draw_circle(nose[0], nose[1], 4)
a = img.draw_circle(lm[0], lm[1], 4)
a = img.draw_circle(rm[0], rm[1], 4)
# align face to standard position
src_point = [le, re, nose, lm, rm]
T = image.get_affine_transform(src_point, dst_point)
a = image.warp_affine_ai(img, img_face, T)
a = img_face.ai_to_pix()
#Display the matched face feature image on the screen
# a = img.draw_image(img_face, (128,0))
del (face_cut_128)
# calculate face feature vector
fmap = kpu.forward(task_fe, img_face)
feature = kpu.face_encode(fmap[:])
reg_flag = False
scores = []
for j in range(len(record_ftrs)):
score = kpu.face_compare(record_ftrs[j], feature)
scores.append(score)
max_score = 0
index = 0
for k in range(len(scores)):
if max_score < scores[k]:
max_score = scores[k]
index = k
if max_score > ACCURACY:
a = img.draw_string(i.x(), i.y(), ("%s :%2.1f" % (
names[index], max_score)), color=(0, 255, 0), scale=2)
led_r.value(1)
led_g.value(0)
else:
a = img.draw_string(i.x(), i.y(), ("X :%2.1f" % (
max_score)), color=(255, 0, 0), scale=2)
led_r.value(0)
led_g.value(1)
#Store the current feature to the face feature list
if start_processing:
record_ftr = feature
record_ftrs.append(record_ftr)
start_processing = False

break
#Calculate the frame rate
fps = clock.fps()
#Print the frame rate
print("%2.1f fps" % fps)
a = lcd.display(img)
gc.collect()
# kpu.memtest()

# a = kpu.deinit(task_fe)
# a = kpu.deinit(task_ld)
# a = kpu.deinit(task_fd)