is there any way to send webrtc frame to python script?

I created first web app(python and django) which shows client’s webcam frames
This is my video.js

'use strict';

// On this codelab, you will be streaming only video (video: true).
const mediaStreamConstraints = {
  video: true,
};

// Video element where stream will be placed.
const localVideo = document.querySelector('video');

// Local stream that will be reproduced on the video.
let localStream;

// Handles success by adding the MediaStream to the video element.
function gotLocalMediaStream(mediaStream) {
  localStream = mediaStream;
  localVideo.srcObject = mediaStream;
}

// Handles error by logging a message to the console with the error message.
function handleLocalMediaStreamError(error) {
  console.log('navigator.getUserMedia error: ', error);
}

// Initializes media stream.
navigator.mediaDevices.getUserMedia(mediaStreamConstraints)
  .then(gotLocalMediaStream).catch(handleLocalMediaStreamError);

However, I want to use client’s webcam frame as an input to my machine learning script(python .py) file.
In local, it was easily done via opencv and numpy.
But in web, I cannot feed frames to ML model.
Any suggestions?

Answer

webrtc is a peer-to-peer connection so clients can be connected to the most direct route possible, without server interference (when client-server connections are needed, webrtc is not as useful as sockets or gRPC). so if you want to redirect the data from one peer to ML model you should connect it directly to ML model as the other peer. to do so, one way would be creating a python native webrtc client for your ML model controller,in that case, you may find this GitHub repository useful.

another way for real-time connection would be a proxy controller which benefits the old-school UDP socket connection or gRPC streaming between your webrtc controller in browser and ML controller in processing service

we still have the old and battle proved socket connection (django channels) or newer cilent server options like gRPC streaming.