Get the x and y coordinates of the video picker to resize and drag based on the video resolution

Question

I'm using @use-gesture/react and @react-spring/web to create a draggable and resizable selector overlay over a video. I'm also able to get the x and y coordinates of the selector relative to the video element, but since I set the height and width for the video tag to fit the video to my screen, the x and y coordinates I'm getting are in the full resolution video are not in exactly the same position. I need to pass x and y coordinates to ffmpeg to blur a specific part of the video.

Answer 1

To achieve the same coordinate transformation in a more manual way, you can use raw JavaScript to do it:

// 获取原始视频尺寸
const originalWidth = ...; // 获取视频的原始宽度
const originalHeight = ...; // 获取视频的原始高度

// 获取缩放后视频元素的当前宽度和高度
const scaledVideoWidth = ...; // 获取缩放后视频元素的当前宽度
const scaledVideoHeight = ...; // 获取缩放后视频元素的当前高度

// 计算缩放因子
const scaleX = originalWidth / scaledVideoWidth;
const scaleY = originalHeight / scaledVideoHeight;

// 用于获取选择器坐标的事件监听器
function handleSelectorDrag(x, y) {
  const originalX = x * scaleX;
  const originalY = y * scaleY;

  // 现在可以使用originalX和originalY执行诸如使用FFmpeg进行模糊处理等操作
  // 示例：运行FFmpeg命令以在视频的某个部分进行模糊处理
  const ffmpegCommand = `ffmpeg -i input.mp4 -vf "crop=w=100:h=100:x=${originalX}:y=${originalY},boxblur=10" output.mp4`;

  // 使用您喜欢的方法（例如，在Node.js中使用child_process）运行FFmpeg命令
}

// 示例拖动事件的事件监听器
const selectorElement = document.getElementById('selector'); // 请替换为实际的选择器元素
selectorElement.addEventListener('drag', (event) => {
  const x = event.clientX;
  const y = event.clientY;
  handleSelectorDrag(x, y);
});

Please replace selectorElement in the example with the actual selector element, and adjust the event listener and capture of selector coordinates according to your specific implementation.

Please note that running FFmpeg commands directly in the browser using JavaScript may not be straightforward due to security restrictions. Typically, FFmpeg commands are executed server-side, and your frontend sends requests to the backend to trigger these commands.