Merge multiple audio and video files

Question

I am new to using ffmpeg and what I am trying to do is combine multiple video and audio files and have them being displayed in a mosaic.

What i currently have:

2 video files and 2 audio files, where 1 audio file belongs to 1 video file while the other audio file belongs to the other video file

input1.mkv
input1.mka
input2.mkv
input2.mka

What i'm trying to accomplish

1. Combine input1.mkv with input1.mka and input2.mkv with input2.mka
2. I want to display input1.mkv and input2.mkv in a horizontal mosaic (https://stackoverflow.com/questions/11552565/vertically-or-horizontally-stack-mosaic-several-videos-using-ffmpeg)
3. Input2.mkv should start 120 seconds after input1.mkv in output.mkv

Way of combining that works via multiple calls

ffmpeg -i input1.mkv -i input1.mka -map 0:0 -map 1:0 input1.mkv
ffmpeg -i input2.mkv -i input2.mka -map 0:0 -map 1:0 input2.mkv   

Can the above be done in a single call?

What i currently have in terms of command line argument:

ffmpeg -i input1.mkv -i input1.mka -itsoffset 120 -i input2.mkv -i input2.mka -filter_complex hstack=inputs=2 -map 0:0 -map 1:0 -map 2:0 -map 3:0 -acodec copy -vcodec copy output.mkv

Error i'm getting:

Streamcopy requested for output stream 0:0, which is fed from a complex filtergraph. Filtering and streamcopy cannot be used together.

Am i going about it correctly in the task i'm trying to accomplish? Should i execute multiple ffmpegs call from C# to combine each input's audio and video file separately? Thank you for your help!

Answer 1

Try this:

ffmpeg -i input1.mkv -i input1.mka -i input2.mkv -i input2.mka -i input3.mka \
  -filter_complex \
    "[2:v]tpad=start_duration=120:color=black[vpad]; \
     [3:a]adelay=120000:all=1[a2pad]; \
     [4:a]adelay=180000:all=1[a3pad]; \
     [0:v][vpad]hstack=inputs=2[vout]; \
     [1:a][a2pad][a3pad]amix=inputs=3:weights=1|1|1[aout]" \
  -map [vout] -map [aout] \
  output.mkv

• To delay the second set of A/V files, you use tpad for video and adelay for audio. Here you see how the 120s and 180s delays are coded. For tpad you can also set the color of the padded video frames.
• [edit] adelay has to specify delay for all channels. Assuming audios are stereo, you need to add all=1 option.
• amix I've already explained in the comments. I've added weights option just in case you decide to change the mixing balance.
• the filtergraph outputs a video stream [vout] and a audio stream [aout]. You then assign them to the output file using -map option, one for each of them.
• BTW, both [vout] and [aout] will last until the longest of 2 streams ends (shorter audio stream drops out while shorter video stream holds its last frame) If this is a problem, describe the situation and desired.
• You cannot copy output streams of a filtergraph. So -vcodec copy and -acodec copy are no goes. I've removed them to use the default codecs for .mkv file, but if you need to customize it, add output options before output.mkv.