'Real time video stream cropping & re compositing: your thoughts on this?

I'm digging some information for an upcoming personal/hobby project and I thought Stack Overflow would be a nice place to discuss the feasibility of the project.

In a few words:

I got a video stream (from an HDMI capture card), I would like to take a few parts of that stream and re-arrange them into a new video.

In a few more words:

  • be able to selectively crop some areas (hardcoded locations could do the trick) from a video stream
  • use the cropped video parts (of the original video) for compositing a new video by rearranging those elements and maybe adding a few other things
  • being able to do this with minimal latency (Less than a second could be okay)
  • being able to preview locally the output video in "real time"
  • be able to switch between "scenes" (different video cropping areas etc.)

As pictures speak better than confused explanations, here is one:

schema

Is it something in the realm of the possibilities for a senior dev (backend and mobile) with small experience in media manipulation ? or is it a really big challenge right here ? is the "real time" constraints bearable for this kind of task ?

what kind language/tool would you direct to ?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source