教程：使用深度学习和OpenCV的自拍过滤器（面部标记检测）

# Just load the recent model using load_my_CNN_model and use it to predict keypoints on any face data

my_model = load_my_CNN_model('my_model')

'''

第2步：初始化Stuff

我们现在创建一个新的python文件 --shades.py，我们在其中编写代码来读取网络摄像头输入，检测面部，使用我们构建的CNN模型等。

from my_CNN_model import *

import cv2

import numpy as np

# Load the model built in the previous step

my_model = load_my_CNN_model('my_model')

# Face cascade to detect faces

face_cascade = cv2.CascadeClassifier('cascades/haarcascade_frontalface_default.xml')

# Define the upper and lower boundaries for a color to be considered "Blue"

blueLower = np.array([100, 60, 60])

blueUpper = np.array([140, 255, 255])

# Define a 5x5 kernel for erosion and dilation

kernel = np.ones((5, 5), np.uint8)

# Define filters

filters = ['images/sunglasses.png', 'images/sunglasses_2.png', 'images/sunglasses_3.jpg', 'images/sunglasses_4.png', 'images/sunglasses_5.jpg', 'images/sunglasses_6.png']

filterIndex = 0

# Load the video - O for webcam input

camera = cv2.VideoCapture(0)

第3步：检测输入中的面部

一旦开始从网络摄像头读取输入，使用我们之前初始化的Cascade Classifier对象检测输入中的脸。

# Keep reading the input

while True:

(grabbed, frame) = camera.read()

frame = cv2.flip(frame, 1)

frame2 = np.copy(frame)

# Convert to HSV and GRAY for convenience

hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

# Detect faces using the haar cascade object

faces = face_cascade.detectMultiScale(gray, 1.25, 6)

第4步：创建过滤器切换触发器

在这里，我使用蓝色瓶盖作为过滤开关。为了检测蓝色瓶盖，我们编写代码以找到图像中的蓝色轮廓。我们传递了在步骤2中定义的blueLower和blueUpper hsv范围。

# Add the 'Next Filter' button to the frame

frame = cv2.rectangle(frame, (500,10), (620,65), (235,50,50), -1)

cv2.putText(frame, "NEXT FILTER", (512, 37), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)

# Determine which pixels fall within the blue boundaries

# and then blur the binary image to remove noise

blueMask = cv2.inRange(hsv, blueLower, blueUpper)

blueMask = cv2.erode(blueMask, kernel, iterations=2)

blueMask = cv2.morphologyEx(blueMask, cv2.MORPH_OPEN, kernel)

blueMask = cv2.dilate(blueMask, kernel, iterations=1)

一旦我们创建了blueMask，它应该在视频中找到蓝色的东西，我们使用OpenCV的cv2.findContours（）方法来查找轮廓。

# Find contours (bottle cap in my case) in the image

(_, cnts, _) = cv2.findContours(blueMask.copy(), cv2.RETR_EXTERNAL,

cv2.CHAIN_APPROX_SIMPLE)

center = None

# Check to see if any contours were found

if len(cnts) > 0:

# Sort the contours and find the largest one -- we

# will assume this contour correspondes to the area of the bottle cap

cnt = sorted(cnts, key = cv2.contourArea, reverse = True)[0]

# Get the radius of the enclosing circle around the found contour

((x, y), radius) = cv2.minEnclosingCircle(cnt)

# Draw the circle around the contour

cv2.circle(frame, (int(x), int(y)), int(radius), (0, 255, 255), 2)

# Get the moments to calculate the center of the contour (in this case Circle)

M = cv2.moments(cnt)

center = (int(M['m10'] / M['m00']), int(M['m01'] / M['m00']))

# Once

if center[1] <= 65:

if 500 <= center[0] <= 620: # Next Filter

filterIndex += 1

filterIndex %= 6

continue

步骤5：使用模型检测面部关键点

我们使用我们之前初始化的面部级联来定位框架中的面。然后，我们遍历每个面部以预测面部关键点。

# Loop over all the faces found in the frame

for (x, y, w, h) in faces:

# Make the faces ready for the model (normalize, resize and stuff)

gray_face = gray[y:y+h, x:x+w]

color_face = frame[y:y+h, x:x+w]

# Normalize to match the input format of the model - Range of pixel to [0, 1]

gray_normalized = gray_face / 255

# Resize it to 96x96 to match the input format of the model

original_shape = gray_face.shape # A Copy for future reference

face_resized = cv2.resize(gray_normalized, (96, 96), interpolation = cv2.INTER_AREA)

face_resized_copy = face_resized.copy()

face_resized = face_resized.reshape(1, 96, 96, 1)

# Predict the keypoints using the model

keypoints = my_model.predict(face_resized)

# De-Normalize the keypoints values

keypoints = keypoints * 48 + 48

# Map the Keypoints back to the original image

face_resized_color = cv2.resize(color_face, (96, 96), interpolation = cv2.INTER_AREA)

face_resized_color2 = np.copy(face_resized_color)

# Pair the keypoints together - (x1, y1)

points = []

for i, co in enumerate(keypoints[0][0::2]):

points.append((co, keypoints[0][1::2][i]))

在我们将检测到的面部传递给模型之前，我们必须对输入进行归一化，因为归一化图像是我们模型训练的图像，将其大小调整为96 x 96图像，因为这是我们的模型所期望的。上面的代码也是一样的。

第6步：使用面部关键点将阴影置于面部

一旦我们检测到面部关键点，我们就可以使用它们来做各种很酷的事情。例如，您可以使用鼻子关键点添加胡子，嘴唇关键点，以便为它们添加颜色等。

# Add FILTER to the frame

sunglasses = cv2.imread(filters[filterIndex], cv2.IMREAD_UNCHANGED)

sunglass_width = int((points[7][0]-points[9][0])*1.1)

sunglass_height = int((points[10][1]-points[8][1])/1.1)

sunglass_resized = cv2.resize(sunglasses, (sunglass_width, sunglass_height), interpolation = cv2.INTER_CUBIC)

transparent_region = sunglass_resized[:,:,:3] != 0

face_resized_color[int(points[9][1]):int(points[9][1])+sunglass_height, int(points[9][0]):int(points[9][0])+sunglass_width,:][transparent_region] = sunglass_resized[:,:,:3][transparent_region]

# Map the face with shades back to its original shape

frame[y:y+h, x:x+w] = cv2.resize(face_resized_color, original_shape, interpolation = cv2.INTER_CUBIC)

# Add KEYPOINTS to the frame2

for keypoint in points:

cv2.circle(face_resized_color2, keypoint, 1, (0,255,0), 1)

# Map the face with keypoints back to the original image (a separate one)

frame2[y:y+h, x:x+w] = cv2.resize(face_resized_color2, original_shape, interpolation = cv2.INTER_CUBIC)

在这里，我们使用了4个关键点来配置脸部阴影的宽度和高度。并且，face_resized_color有阴影，并且映射回frame，face_resized_color2有关键点，并映射回frame2。然后我们将它们显示出来。

# Show the frame and the frame2

cv2.imshow("Selfie Filters", frame)

cv2.imshow("Facial Keypoints", frame2)

# If the 'q' key is pressed, stop the loop

if cv2.waitKey(1) & 0xFF == ord("q"):

break

# Cleanup the camera and close any open windows

camera.release()

cv2.destroyAllWindows()

最后，我们只是清理stuff。

执行

1.下载数据

从此处下载数据（https://www.kaggle.com/c/facial-keypoints-detection/data）并将其放在项目目录中名为“data”的文件夹中。确保'data'文件夹包含 - training.csv和test.csv

2.构建CNN模型

> python model_builder.py

这会调用my_CNN_model.py，因此请确保您的项目目录中包含该文件。

4.运行引擎文件

> python shades.py

5.拿一个蓝色瓶盖测试

结论

在本教程中，我们构建了一个深度卷积神经网络模型，对面部关键点数据进行了训练。然后，我们使用该模型来预测在输入网络摄像头数据中检测到的面部的面部关键点。我们还使用轮廓创建了一个开关，它允许我们使用手势迭代其他过滤器。我鼓励您调整模型架构并亲自了解它如何影响关键点检测。

这个数据只有15个关键点，其他几个数据集在面上标有超过30个关键点。

面部关键点检测的应用:

您可以在此处访问完整的项目代码：

akshaychandra21 / Selfie_Filters_OpenCV (https://github.com/akshaychandra21/Selfie_Filters_OpenCV/)