Table of Contents
1. Project effect
2. Core process
3. Code process
1. Read the camera video and draw a rectangle
2. Import mediapipe to process finger coordinates
3. Position calculation
Full code
Home Backend Development Python Tutorial How to achieve the effect of dragging virtual squares with Python+OpenCV

How to achieve the effect of dragging virtual squares with Python+OpenCV

May 15, 2023 pm 07:22 PM
python opencv

1. Project effect

2. Core process

1. OpenCV reads the video stream and draws a rectangle on each frame of the picture.

2. Use mediapipe to obtain finger key point coordinates.

3. Based on the coordinate position of the finger and the coordinate position of the rectangle, determine whether the finger point is on the rectangle. If so, the rectangle will follow the finger movement.

3. Code process

Environment preparation:

python: 3.8.8

opencv: 4.2.0.32

mediapipe: 0.8 .10.1

Note:

1. If the opencv version is too high or too low, there may be some problems such as the camera not being able to open, crashing, etc. The python version affects the selectable version of opencv.

2. OpenCV may not be able to be used normally after pip install mediapipe. Uninstall it and download it again. Just get used to it.

1. Read the camera video and draw a rectangle

import cv2
import time
import numpy as np
 
 
# 调用摄像头 0 默认摄像头 
cap = cv2.VideoCapture(0)
 
# 初始方块数据
x = 100
y = 100
w = 100
h = 100
 
# 读取一帧帧照片
while True:
    # 返回frame图片
    rec,frame = cap.read()
    
    # 镜像
    frame = cv2.flip(frame,1)
    
    # 画矩形 
    cv2.rectangle(frame, (x, y), (x + w, y + h), (255, 0, 255), -1)
 
    # 显示画面
    cv2.imshow('frame',frame)
    
    # 退出条件
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
    
cap.release()
cv2.destroyAllWindows()
Copy after login

This is a very basic step. At this time, when we run this code and the camera is turned on, we will be surprised to see our handsome face. face, and there is a 100*100 purple rectangle in the upper left corner.

2. Import mediapipe to process finger coordinates

pip install mediapipe
Copy after login

Some problems may occur at this time, such as openCV suddenly becoming unavailable. It doesn’t matter, uninstall it and download it again.

mediapipe details: Hands - mediapipe (google.github.io)

How to achieve the effect of dragging virtual squares with Python+OpenCV

In short, it will return us 21 finger key points Coordinates, that is, its position ratio in the video screen (0~1), we multiply it by the width and height of the corresponding screen to get the coordinates corresponding to the finger.

This time I used the tips of my index finger and middle finger, which are numbers 8 and 12.

2.1 Configure some basic information

import cv2
import time
import numpy as np
import mediapipe as mp
 
 
mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
mp_hands = mp.solutions.hands
 
hands =  mp_hands.Hands(
    static_image_mode=True,
    max_num_hands=2,
    min_detection_confidence=0.5)
Copy after login

2.2 When processing each frame of image, add

    frame.flags.writeable = False
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    # 返回结果
    results = hands.process(frame)
 
    frame.flags.writeable = True
    frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
Copy after login

When we are in When reading each frame of picture in the video stream, it is converted from BGR to RGB and supplied to the hands object generated by mediapipe for reading. It will return the information of the key points of the fingers in this picture. We only need to continue to draw on it. on every frame of the picture.

    # 如果结果不为空
    if results.multi_hand_landmarks:
 
        # 遍历双手(根据读取顺序,一只只手遍历、画画)
        for hand_landmarks in results.multi_hand_landmarks:
            mp_drawing.draw_landmarks(
                frame,
                hand_landmarks,
                mp_hands.HAND_CONNECTIONS,
                mp_drawing_styles.get_default_hand_landmarks_style(),
                mp_drawing_styles.get_default_hand_connections_style())
Copy after login

2.3 The complete code of this step

import cv2
import time
import numpy as np
import mediapipe as mp
 
 
mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
mp_hands = mp.solutions.hands
 
hands =  mp_hands.Hands(
    static_image_mode=True,
    max_num_hands=2,
    min_detection_confidence=0.5)
 
 
# 调用摄像头 0 默认摄像头 
cap = cv2.VideoCapture(0)
 
# 方块初始数组
x = 100
y = 100
w = 100
h = 100
 
 
# 读取一帧帧照片
while True:
    # 返回frame图片
    rec,frame = cap.read()
    
    # 镜像
    frame = cv2.flip(frame,1)
    
    
    
    frame.flags.writeable = False
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    # 返回结果
    results = hands.process(frame)
 
    frame.flags.writeable = True
    frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
    
    
    # 如果结果不为空
    if results.multi_hand_landmarks:
 
        # 遍历双手(根据读取顺序,一只只手遍历、画画)
        # results.multi_hand_landmarks n双手
        # hand_landmarks 每只手上21个点信息
        for hand_landmarks in results.multi_hand_landmarks:
            mp_drawing.draw_landmarks(
                frame,
                hand_landmarks,
                mp_hands.HAND_CONNECTIONS,
                mp_drawing_styles.get_default_hand_landmarks_style(),
                mp_drawing_styles.get_default_hand_connections_style())
    
    
    # 画矩形 
    cv2.rectangle(frame, (x, y), (x + w, y + h), (255, 0, 255), -1)
 
    # 显示画面
    cv2.imshow('frame',frame)
    
    # 退出条件
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
    
cap.release()
cv2.destroyAllWindows()
Copy after login

3. Position calculation

Our experiment requires dragging the square, so there must be some that do not drag time, so we might as well obtain the positions of the index finger (8) and middle finger (12) tips according to the previous step. If they are close, we will change the coordinates of the block according to the position of the fingers when they coincide with the block.

How to achieve the effect of dragging virtual squares with Python+OpenCV

Full code

How to achieve the effect of dragging virtual squares with Python+OpenCV

import cv2
import time
import math
import numpy as np
import mediapipe as mp
 
# mediapipe配置
mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
mp_hands = mp.solutions.hands
hands =  mp_hands.Hands(
    static_image_mode=True,
    max_num_hands=2,
    min_detection_confidence=0.5)
 
 
# 调用摄像头 0 默认摄像头 
cap = cv2.VideoCapture(0)
 
# cv2.namedWindow("frame", 0)
# cv2.resizeWindow("frame", 960, 640)
 
 
# 获取画面宽度、高度
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
 
 
# 方块初始数组
x = 100
y = 100
w = 100
h = 100
 
L1 = 0
L2 = 0
 
on_square = False
square_color = (0, 255, 0)
 
# 读取一帧帧照片
while True:
    # 返回frame图片
    rec,frame = cap.read()
    
    # 镜像
    frame = cv2.flip(frame,1)
    
    
    
    frame.flags.writeable = False
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    # 返回结果
    results = hands.process(frame)
 
    frame.flags.writeable = True
    frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
    
    
    # 如果结果不为空
    if results.multi_hand_landmarks:
 
 
        # 遍历双手(根据读取顺序,一只只手遍历、画画)
        # results.multi_hand_landmarks n双手
        # hand_landmarks 每只手上21个点信息
        for hand_landmarks in results.multi_hand_landmarks:
            mp_drawing.draw_landmarks(
                frame,
                hand_landmarks,
                mp_hands.HAND_CONNECTIONS,
                mp_drawing_styles.get_default_hand_landmarks_style(),
                mp_drawing_styles.get_default_hand_connections_style())
            
            # 记录手指每个点的x y 坐标
            x_list = []
            y_list = []
            for landmark in hand_landmarks.landmark:
                x_list.append(landmark.x)
                y_list.append(landmark.y)
                
            
            # 获取食指指尖
            index_finger_x, index_finger_y = int(x_list[8] * width),int(y_list[8] * height)
 
            # 获取中指
            middle_finger_x,middle_finger_y = int(x_list[12] * width), int(y_list[12] * height)
 
 
            # 计算两指尖距离
            finger_distance = math.hypot((middle_finger_x - index_finger_x), (middle_finger_y - index_finger_y))
 
            # 如果双指合并(两之间距离近)
            if finger_distance < 60:
 
                # X坐标范围 Y坐标范围
                if (index_finger_x > x and index_finger_x < (x + w)) and (
                        index_finger_y > y and index_finger_y < (y + h)):
 
                    if on_square == False:
                        L1 = index_finger_x - x
                        L2 = index_finger_y - y
                        square_color = (255, 0, 255)
                        on_square = True
 
            else:
                # 双指不合并/分开
                on_square = False
                square_color = (0, 255, 0)
 
            # 更新坐标
            if on_square:
                x = index_finger_x - L1
                y = index_finger_y - L2
            
            
 
    # 图像融合 使方块不遮挡视频图片
    overlay = frame.copy()
    cv2.rectangle(frame, (x, y), (x + w, y + h), square_color, -1)
    frame = cv2.addWeighted(overlay, 0.5, frame, 1 - 0.5, 0)
    
 
    # 显示画面
    cv2.imshow(&#39;frame&#39;,frame)
    
    # 退出条件
    if cv2.waitKey(1) & 0xFF == ord(&#39;q&#39;):
        break
    
cap.release()
cv2.destroyAllWindows()
Copy after login

The above is the detailed content of How to achieve the effect of dragging virtual squares with Python+OpenCV. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Repo: How To Revive Teammates
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

How to download deepseek Xiaomi How to download deepseek Xiaomi Feb 19, 2025 pm 05:27 PM

How to download DeepSeek Xiaomi? Search for "DeepSeek" in the Xiaomi App Store. If it is not found, continue to step 2. Identify your needs (search files, data analysis), and find the corresponding tools (such as file managers, data analysis software) that include DeepSeek functions.

How do you ask him deepseek How do you ask him deepseek Feb 19, 2025 pm 04:42 PM

The key to using DeepSeek effectively is to ask questions clearly: express the questions directly and specifically. Provide specific details and background information. For complex inquiries, multiple angles and refute opinions are included. Focus on specific aspects, such as performance bottlenecks in code. Keep a critical thinking about the answers you get and make judgments based on your expertise.

How to search deepseek How to search deepseek Feb 19, 2025 pm 05:18 PM

Just use the search function that comes with DeepSeek. Its powerful semantic analysis algorithm can accurately understand the search intention and provide relevant information. However, for searches that are unpopular, latest information or problems that need to be considered, it is necessary to adjust keywords or use more specific descriptions, combine them with other real-time information sources, and understand that DeepSeek is just a tool that requires active, clear and refined search strategies.

How to program deepseek How to program deepseek Feb 19, 2025 pm 05:36 PM

DeepSeek is not a programming language, but a deep search concept. Implementing DeepSeek requires selection based on existing languages. For different application scenarios, it is necessary to choose the appropriate language and algorithms, and combine machine learning technology. Code quality, maintainability, and testing are crucial. Only by choosing the right programming language, algorithms and tools according to your needs and writing high-quality code can DeepSeek be successfully implemented.

How to use deepseek to settle accounts How to use deepseek to settle accounts Feb 19, 2025 pm 04:36 PM

Question: Is DeepSeek available for accounting? Answer: No, it is a data mining and analysis tool that can be used to analyze financial data, but it does not have the accounting record and report generation functions of accounting software. Using DeepSeek to analyze financial data requires writing code to process data with knowledge of data structures, algorithms, and DeepSeek APIs to consider potential problems (e.g. programming knowledge, learning curves, data quality)

How to access DeepSeekapi - DeepSeekapi access call tutorial How to access DeepSeekapi - DeepSeekapi access call tutorial Mar 12, 2025 pm 12:24 PM

Detailed explanation of DeepSeekAPI access and call: Quick Start Guide This article will guide you in detail how to access and call DeepSeekAPI, helping you easily use powerful AI models. Step 1: Get the API key to access the DeepSeek official website and click on the "Open Platform" in the upper right corner. You will get a certain number of free tokens (used to measure API usage). In the menu on the left, click "APIKeys" and then click "Create APIkey". Name your APIkey (for example, "test") and copy the generated key right away. Be sure to save this key properly, as it will only be displayed once

Major update of Pi Coin: Pi Bank is coming! Major update of Pi Coin: Pi Bank is coming! Mar 03, 2025 pm 06:18 PM

PiNetwork is about to launch PiBank, a revolutionary mobile banking platform! PiNetwork today released a major update on Elmahrosa (Face) PIMISRBank, referred to as PiBank, which perfectly integrates traditional banking services with PiNetwork cryptocurrency functions to realize the atomic exchange of fiat currencies and cryptocurrencies (supports the swap between fiat currencies such as the US dollar, euro, and Indonesian rupiah with cryptocurrencies such as PiCoin, USDT, and USDC). What is the charm of PiBank? Let's find out! PiBank's main functions: One-stop management of bank accounts and cryptocurrency assets. Support real-time transactions and adopt biospecies

Quantitative currency trading software Quantitative currency trading software Mar 19, 2025 pm 04:06 PM

This article explores the quantitative trading functions of the three major exchanges, Binance, OKX and Gate.io, aiming to help quantitative traders choose the right platform. The article first introduces the concepts, advantages and challenges of quantitative trading, and explains the functions that excellent quantitative trading software should have, such as API support, data sources, backtesting tools and risk control functions. Subsequently, the quantitative trading functions of the three exchanges were compared and analyzed in detail, pointing out their advantages and disadvantages respectively, and finally giving platform selection suggestions for quantitative traders of different levels of experience, and emphasizing the importance of risk assessment and strategic backtesting. Whether you are a novice or an experienced quantitative trader, this article will provide you with valuable reference

See all articles