Achieving DoppelGanger with Face Recognition
How to make DoppelGanger using OpenCV and dlib
In this article, we will explore how to achieve DoppelGanger using face recognition techniques. The process involves three key steps: Enrollment, Testing, and Inference.
Enrollment
In enrollment we take training dataset of images of the individual and pass through the neural network to obtain 128-dimension feature descriptor of each image.
Testing
In testing we take test image of the each individual and evaluate the system accuracy. To test the accuracy we calculate distance between test and enrolled feature descriptor if distance is less than the threshold value. then we say test images belongs to same person
Inference
In inference we take real-time data and check the accuracy and performance of the system
Implementing DoppelGanger
Enrollment
In enrollment we take all the images and compute feature descriptors and assign index with name.
We save the descriptor and index in separate file newceleb_descriptor.npy and index.pkl
faceDetector = dlib.get_frontal_face_detector()
shapePredictor = dlib.shape_predictor(DATA_PATH + 'models/shape_predictor_68_face_landmarks.dat')
faceRecognizer = dlib.face_recognition_model_v1(DATA_PATH +'dlap_week4/models/dlib_face_recognition_resnet_model_v1.dat')
faceDatasetFolder = 'celeb_mini'
# process faceDatasetFolder to create imagePaths and nameLabelMap
index = {}
i = 0
faceDescriptors = None
for imagePath in imagePaths:
im = cv2.imread(imagePath)
imDlib = cv2.cvtColor(im,cv2.COLOR_BGR2RGB)
faces = faceDetector(imDlib)
for k, face in enumerate(faces):
shape = shapePredictor(imDlib, face)
landmarks = [(p.x,p.y) for p in shape.parts()]
faceDescriptor = faceRecognizer.compute_face_descriptor(im, shape)
faceDescriptorList = [x for x in faceDescriptor]
faceDescriptorNdarray = np.asarray(faceDescriptorList, dtype=np.float64)
faceDescriptorNdarray = faceDescriptorNdarray[np.newaxis,:]
if faceDescriptors is None:
faceDescriptors = faceDescriptorNdarray
else:
faceDescriptors = np.concatenate((faceDescriptors, faceDescriptorNdarray), axis=0)
index[i] = nameLabelMap[imagePath]
i+=1
np.save("newceleb_descriptor.npy", faceDescriptors)
with open("index.pkl",'wb') as f:
cPickle.dump(index,f)
Testing
In testing we take test images and compute feature descriptor then find the Euclidean distance between test image and enrolled descriptor. if distance is less then the threshold then we say image belongs to same person.
# read image
testImages = glob.glob('test-images/*.jpg')
import _pickle as cPickle
index = np.load("index.pkl", allow_pickle=True)
faceDescriptorsEnrolled = np.load("newceleb_descriptor.npy", allow_pickle=True)
for test in testImages:
im = cv2.imread(test)
imDlib = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)
faces = faceDetector(imDlib)
celeb_name = ""
imCeleb = np.zeros((200,200,3))
for face in faces:
shape = shapePredictor(imDlib, face)
faceDescriptor = faceRecognizer.compute_face_descriptor(imDlib, shape)
faceDescriptorList = [m for m in faceDescriptor]
faceDescriptorNdarray = np.asarray(faceDescriptorList, dtype=np.float64)
faceDescriptorNdarray = faceDescriptorNdarray[np.newaxis,:]
distances = np.linalg.norm(faceDescriptorsEnrolled - faceDescriptorNdarray, axis=1)
argmin = np.argmin(distances)
minDistance = distances[argmin]
THRESHOLD = 0.6
if minDistance <= THRESHOLD:
celeb_name = index[argmin]
celeb_folder = os.path.join(faceDatasetFolder, celeb_name)
image_files = glob.glob(os.path.join(celeb_folder, '*'))
imCeleb = cv2.imread(image_files[0])
imCeleb = cv2.cvtColor(imCeleb, cv2.COLOR_BGR2RGB)
else:
celeb_name = 'unknown'
imCeleb = np.zeros((200,200,3))
# Display Images
plt.subplot(121)
plt.imshow(imDlib)
plt.title("test img")
plt.subplot(122)
plt.imshow(imCeleb)
plt.title("Celeb Look-Alike={}".format(celeb_name))
plt.show()
Conclusion
By following these steps, we can implement an effective face recognition system capable of identifying DoppelGangers. The process ensures accurate enrollment, testing, and inference, forming a robust pipeline for face recognition tasks.