Give you a Santa hat using Python

Christmas is coming. Although we can’t celebrate this Western festival, we still have to join in the fun. I believe there are already a lot of Christmas hat-related peripherals circulating. Today we will do it ourselves and add a Christmas hat to the avatar

Basic knowledge preparation

In computers, images are saved in the form of a matrix, rows first and columns second. Therefore, an image with width × height × color channel = 480 × 256 × 3 will be stored in a 256 × 480 × 3 three-dimensional tensor. Image processing is also calculated according to this idea (including image processing under OpenCV), that is, height × width × color channel.

Digital image

For a digital image, what we see is a real picture visible to the naked eye, but to the computer, this image is just a bunch of different brightnesses. point. An image of size M × N can be represented by an M × N matrix. The value of the matrix element represents the brightness of the pixel at this position. Generally speaking, the larger the pixel value, the brighter the point.

Generally speaking, grayscale images are represented by 2-dimensional matrices, and color (multi-channel) images are represented by 3-dimensional matrices (M × N × 3).

Image channel

Describes a pixel. If it is grayscale, then only one value is needed to describe it, which is a single channel. If a pixel has three colors, RGB, to describe it, it has three channels. A four-channel image is R, G, B plus an A channel, indicating transparency. Generally called alpha channel, indicating transparency.

ROI and mask

Setting Region of Interest (ROI), translated into vernacular as, setting the region of interest. Mask is an image masking process, which is equivalent to covering the parts we don't care about, leaving the ROI part. The alpha mentioned above can be used as a mask.

Matrix (Numpy) knowledge

Matrix indexing, slicing, etc., I don’t know much about them here, so I won’t go into details. Friends can learn by themselves.

Environment preparation

After we have the basic knowledge, let’s take a brief look at the code.

First install the OpenCV and dlib libraries you need to use, use pip to install them respectively

pip install python-opencv

pip install dlib

Then manually download the data model file shape_predictor_5_face_landmarks.dat from the Internet, address As follows: http://dlib.net/files/, download and put it in the project directory.

Interested students can play with shape_predictor_68_face_landmarks.dat, which recognizes as many as 68 key points on faces.

Code processing

Hat processing

The first thing we have to do is to process the hat. The pictures we use are as follows

First extract the rgb and alpha values ​​of the hat image

# 帽子Give you a Santa hat using Python
hat_img3 = cv2.imread("hat.png", -1)
r, g, b, a = cv2.split(hat_img3)
rgb_hat = cv2.merge((r, g, b))
cv2.imwrite("rgb_hat.jpg", rgb_hat)
cv2.imwrite("alpha.jpg", a)
print(a)
print(hat_img3.shape)
print(rgb_hat.shape)

The effect we get is as follows:

rgb image

alpha graph

The printed value of a is as follows:

[[0 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]
 ...
 [0 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]]

Face detection

The following is face detection, using dlib processing.

# 人脸检测
dets = self.detector(img, 1)
x, y, w, h = dets[0].left(), dets[0].top(), dets[0].right() - dets[0].left(), dets[0].bottom() - dets[0].top()
# 关键点检测
shape = self.predictor(img, dets[0])
point1 = shape.parts()[0]
point2 = shape.parts(2)
# 求两点中心
eyes_center = ((point1.x + point2.x) // 2, (point1.y + point2.y) // 2)

The next step is to scale down the picture of the hat

# 帽子和人脸转换比例
hat_w = int(round(dets[0].right()/1.5))
hat_h = int(round(dets[0].bottom() / 2))
if hat_h > y:
hat_h = y - 1
hat_newsize = cv2.resize(rgb_hat, (hat_w, hat_h))
mask = cv2.resize(a, (hat_w, hat_h))
mask_inv = cv2.bitwise_not(mask)
dh = 0
dw = 0

bg_roi = img[y+dh-hat_h:y+dh,(eyes_center[0]-hat_w//3):(eyes_center[0]+hat_w//3*2)]

ROI extraction

Perform ROI extraction

# 用alpha通道作为mask
mask = cv2.resize(a, (resized_hat_w, resized_hat_h))
mask_inv = cv2.bitwise_not(mask)

mask variable takes out the area of ​​​​the hat.

mask_inv variable is used to remove the area where the hat is installed in the face image.

Next, take out the area (ROI) where the hat is installed in the face picture

# 原图ROI
# bg_roi = img[y+dh-resized_hat_h:y+dh, x+dw:x+dw+resized_hat_w]
bg_roi = img[y + dh - resized_hat_h:y + dh,
 (eyes_center[0] - resized_hat_w // 3):(eyes_center[0] + resized_hat_w // 3 * 2)]

Then next, in the face picture Take out the hat-shaped area

# 原图ROI中提取放帽子的区域
bg_roi = bg_roi.astype(float)
mask_inv = cv2.merge((mask_inv, mask_inv, mask_inv))
alpha = mask_inv.astype(float) / 255
# 相乘之前保证两者大小一致（可能会由于四舍五入原因不一致）
alpha = cv2.resize(alpha, (bg_roi.shape[1], bg_roi.shape[0]))
# print("alpha size: ",alpha.shape)
# print("bg_roi size: ",bg_roi.shape)
bg = cv2.multiply(alpha, bg_roi)
bg = bg.astype('uint8')

Here is to convert the default uint8 type of the picture into a float type for operation, and finally convert it back.

Synthesized picture

The black part is where we want to place the hat.

Extract the hat part from the hat picture.

# 提取帽子区域
hat = cv2.bitwise_and(resized_hat, resized_hat, mask=mask)

Use the hat image you just resized to extract.

可以看到，除了帽子部分，其他区域已经掩模处理了。

以上就是提取ROI的过程，比较难懂，需要好好琢磨，尤其是矩阵的切片、mask处理部分。

合成Give you a Santa hat using Python

最后一步就是把人脸Give you a Santa hat using Python与帽子合成到一起了，也就是把人脸空余帽子部分的Give you a Santa hat using Python区域和帽子只展示帽子区域的Give you a Santa hat using Python区域（有点拗口）合并在一起。

# 相加之前保证两者大小一致（可能会由于四舍五入原因不一致）
hat = cv2.resize(hat, (bg_roi.shape[1], bg_roi.shape[0]))
# 两个ROI区域相加
add_hat = cv2.add(bg, hat)

效果如下：

刚刚好，完美叠加Give you a Santa hat using Python。

最后把这个片段放回人脸原图中，展示Give you a Santa hat using Python

img[y+dh-hat_h:y+dh, (eyes_center[0]-hat_w//3):(eyes_center[0]+hat_w//3*2)] = add_hat

美美的Give you a Santa hat using Python就出来啦！

我们再尝试几张不同的Give you a Santa hat using Python。

整体效果还不错哦，需要注意的是，在测试的时候，我们尽量选择人脸占比比较大的Give you a Santa hat using Python来合成，效果要好很多哦~

The above is the detailed content of Give you a Santa hat using Python. For more information, please follow other related articles on the PHP Chinese website!