TransWikia.com

Reading data from Twilio video streams

Data Science Asked by Aayush Gupta on March 4, 2021

Tl;dr: I would like to read video streams coming from all the participants from this https://tfvideo-5708-dev.twil.io/video.html in python

Details: I have a live tutoring website that uses Twilio Video API to carry out the video class. I want to run the object detection and other computer vision models on the input stream. I am stuck at how to get this input video stream on which i can run ml models

  • using Python. I can read by open cv2.VideoCapture(….) method and then run my inference using some pre-trained model or using inbuilt computer vision models from open-cv. I want to understand what should be the parameter under brackets of cv2.VideoCapture().
  • using Tensorflow.js API – by integrating the tf.js code in the website’s Javascript and running the inference on that stream. Is it feasible?

I am having difficulty fetching that stream data to use for my Ml models

To test, I have created a web chat application that leverages Twilio Video . here is the link to the original page and here is the github repo. Following is the link that you can join for video web chat here if you open this link, you can join the room (I will have to deploy first before you can join) and then my requirement is to get access to the streams of all the participants that join.

Here are some links that I have gone through:

https://github.com/twilio/twilio-video.js/issues/333

https://www.twilio.com/blog/build-video-chat-application-python-javascript-twilio-programmable-video

https://www.infoq.com/articles/video-stream-analytics-opencv/

Can anyone help me with this? Is there any other way?

If the question is not clear, let me know what else is needed. I can give more details.

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP