Live queries implementation / aproaches in backend

Question

I am working on think, which should be "live". I.e. use web-sockets or SSE to show current data in browser. Source of my data are two and they should be combined with a bit of business logic. Data can be retrieved using http get and they also come as web-hook notifications.

I am able to code needed thing in java + spring, but readability would suffer. I have discovered that using RethinkDB would make my task much more easier. But it seems that given project is not backed by live development.

I would like any java idiomatic approach / library / external SW (like database) to make easy (maintainable ~= less code) algorithm which would for example do thing like this:

2 inputs:

• filesystem tree (git repo)
• list of trees with some processing info in it. Each tree in list does contain:
  • root node with some irrelevant info
  • some number of children nodes
  • leaf nodes with filename from filesystem (with path), duration of action with file and status of file processing

Note: second input can contain for example 20 trees, where it have info about processing single file from filesystem tree 20 times. i.e. it is possible that for getting info about some particular file, we will need to crawl whole list of trees and there is no guarantee that file will have any matching processing info in second input. In this case, we will output "N/A" to resulting tree for given file.

I would like to transform these two inputs to another tree, which will have structure like first input and will contain info about last status (last from array) and sum of duration.

My current approach was not reactive. It have involved a lot of java stream api and using http GET to get actual data from two sources. It was working ok, it was fast enough, but not enough to introduce pooling and make user feel that it is real time.

To make this reactive, it would involve a lot of spaghetti code to keep current algorithms in place. so I have started another approach (from scratch).

I have started to make some nice OOP class, which will receive changes from both inputs and will produce "observable" changes as an output. This would be relatively nice, if my "query", which computes output would be immutable. It is not, due to design changes of business logic.

Can you point me to some approach making this problem implementation easy to maintain?

PS: I was considering using spring cache mechanism for receiving changes (caching methods which make http get calls for inputs and returns parsed, partly processed, input data). But this part of code is a bit small to mane any difference.