How to implement an online face recognition system using WebSocket and JavaScript

Limited by humans' innate visual processing capabilities, human visual recognition systems cannot be compared with computers in many aspects. For example, human brain power is not enough to recognize a large number of objects in a short time. face. However, in today's advanced computer technology, facial recognition technology has become increasingly mature. Using a combination of computer vision and artificial intelligence, we have been able to develop a variety of facial recognition technologies, one of the most important being online face recognition systems. This article aims to introduce how to use WebSocket and JavaScript to implement an online face recognition system.

First you need to understand what WebSocket is. WebSocket is a network communication protocol based on TCP protocol. It provides a persistent connection between the browser and the server, enabling two-way communication. In this article, we will use WebSocket to send image and recognition information from the client to the server, and to send the recognition results from the server back to the client.

The first step is to create a WebSocket connection. In the client, create a WebSocket connection with the following code snippet:

let socket = new WebSocket("ws://localhost:8080/");
socket.onopen = function() {
   console.log("WebSocket连接已打开");
};
socket.onmessage = function(event) {
   console.log(event.data);
};

This will open a WebSocket connection on localhost, connecting to port 8080. When the WebSocket connection is opened, the log "WebSocket connection is opened" will be output. When a message is received from the server, the message data is output to the console.

Now we need to implement the function of the client sending image information to the server. There are several methods for capturing images, including the "" element in HTML5 and the "getUserMedia" API. One of the easiest ways to capture camera data is using the "getUserMedia" API:

let video = document.querySelector('video');
navigator.mediaDevices.getUserMedia({video: true})
  .then(function (stream) {
    video.srcObject = stream;
  });

It is now possible to draw the captured image into a element using the HTML5 Canvas API:

let canvas = document.getElementById('canvas');
let context = canvas.getContext('2d');
context.drawImage(video, 0, 0, canvas.width, canvas.height);

The image data can be extracted as a Base64 string and then sent to the server via WebSocket:

let dataUrl = canvas.toDataURL('image/jpeg', 1.0);
socket.send(dataUrl);

The server will use OpenCV and Python to process and recognize the received image. The following is a simple Python script that uses OpenCV to cut faces out of images:

import cv2

def detect_faces(image):
    face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.3, 5)
    return faces

def extract_faces(image_path):
    image = cv2.imread(image_path)
    faces = detect_faces(image)
    for i, (x, y, w, h) in enumerate(faces):
        face_image = image[y:y+h, x:x+w]
        cv2.imwrite('face_{}.jpg'.format(i), face_image)

As you can see, this script uses the face detector in OpenCV to detect faces in the image, And cut out the face and save it in the file "face_{}.jpg".

On the server side, you can use Python to write the following WebSocket program.

import asyncio
import cv2
import base64
import io

from aiohttp import web

async def index(request):
    return web.Response(text="WebSocket服务器已启动！")

async def websocket_handler(request):
    ws = web.WebSocketResponse()
    await ws.prepare(request)
    
    while True:
        data = await ws.receive()
        if data.type == web.WSMsgType.TEXT:
            await ws.send_str("接收到了一张新的图像，请稍候……")
            img_data = data.data[23:]  # 截取“data:image/jpeg;base64,”之后的数据
            try:
                img_bytes = base64.b64decode(img_data)
                img_stream = io.BytesIO(img_bytes)
                img_np = cv2.imdecode(np.frombuffer(img_stream.read(), np.uint8), cv2.IMREAD_UNCHANGED)
                
                # 图像识别代码
                # ...
                
                # 向客户端发送识别结果
                await ws.send_str("这是一个人脸。")
            except:
                await ws.send_str("出错了，无法处理该图像。")

        elif data.type == web.WSMsgType.ERROR:
            print("WebSocket连接发生错误！ Code：{}".format(ws.exception()))
            break

    return ws

app = web.Application()
app.router.add_get('/', index)
app.router.add_get('/ws', websocket_handler)  # /ws是WebSocket路径，亦可为其他路径
web.run_app(app, port=8080)

When the WebSocket connection is opened, the websocket_handler function will automatically run and continue to listen for messages from the client. When a new image is received, the Base64 encoding is parsed and processed using OpenCV. After the data is processed, the results are returned to the client.

So far, we have successfully implemented an online face recognition system. The complete client and server side code looks like this:

Client:

<html>
<head>
    <meta charset="UTF-8">
    <title>人脸识别</title>
</head>
<body>
    <h1>人脸识别</h1>
    <video width="320" height="240" autoplay></video>
    <canvas id="canvas" width="320" height="240"></canvas>
    <script>
        let socket = new WebSocket("ws://localhost:8080/");
        socket.onopen = function() {
            console.log("WebSocket连接已打开");
        };
        socket.onmessage = function(event) {
            console.log(event.data);
        };

        let video = document.querySelector('video');
        navigator.mediaDevices.getUserMedia({video: true})
            .then(function (stream) {
                video.srcObject = stream;
            });

        let canvas = document.getElementById('canvas');
        let context = canvas.getContext('2d');

        setInterval(function() {
            context.drawImage(video, 0, 0, canvas.width, canvas.height);
            let dataUrl = canvas.toDataURL('image/jpeg', 1.0);
            socket.send(dataUrl);
        }, 500);
    </script>
</body>
</html>

Server side:

import asyncio
import cv2
import base64
import io

from aiohttp import web

async def index(request):
    return web.Response(text="WebSocket服务器已启动！")

async def websocket_handler(request):
    ws = web.WebSocketResponse()
    await ws.prepare(request)
    
    while True:
        data = await ws.receive()
        if data.type == web.WSMsgType.TEXT:
            await ws.send_str("接收到了一张新的图像，请稍候……")
            img_data = data.data[23:]  # 截取“data:image/jpeg;base64,”之后的数据
            try:
                img_bytes = base64.b64decode(img_data)
                img_stream = io.BytesIO(img_bytes)
                img_np = cv2.imdecode(np.frombuffer(img_stream.read(), np.uint8), cv2.IMREAD_UNCHANGED)
                
                # 图像识别代码
                # ...
                
                # 向客户端发送识别结果
                await ws.send_str("这是一个人脸。")
            except:
                await ws.send_str("出错了，无法处理该图像。")

        elif data.type == web.WSMsgType.ERROR:
            print("WebSocket连接发生错误！ Code：{}".format(ws.exception()))
            break

    return ws

app = web.Application()
app.router.add_get('/', index)
app.router.add_get('/ws', websocket_handler)
web.run_app(app, port=8080)

I hope this article can help you understand how to use WebSocket and JavaScript implements an online face recognition system and quickly builds a feasible system.

The above is the detailed content of How to implement an online face recognition system using WebSocket and JavaScript. For more information, please follow other related articles on the PHP Chinese website!