'Stream instrumentation data lossless trough non-reliable 4G

I have some data aquisition devices in industrial machinery that have 4G connectivity. Right now I make them to stream the intrumentation data in real time to my server through raw TCP/IP protocol. But this has some problems:

  1. The machinery sometimes work in places where there is low or null mobile connectivity. If there is no connectivity for too long it can happen two things: a) the machine gets shutted down and the tcp/ip buffer it's lost along with the instrumentation data or b) the tcp/ip buffer overflows, which has the same results.
  2. The same as point 1, but for the server side, due to maintenance or if something in the server fails in the weekend when nobody is going to notice it but the machinery can be ON and working. Then we can have data loss in the same way as point 1.
  3. I have to manage authentication and the connection of all the clients into a server single TCP port. I have done some temporary hack that works for the moment but isn't the best. But this is another problem and it's not the reason of this question, so take it only for context.

So, I should code an application layer acknowledge where the server tells the client when a high-level message (not the individual TCP packets) has been received and processed. And in the client side to have a buffer writted in-disk where data is being deleted as is being confirmed by the server. This, to solve points 1 & 2.

But I'm afraid that I'm reinventing the wheel or that I don't know the correct tools, because I think that this problem should be more or less common but I fail to google for it and I can't find a library or tool that does this job.

What I was thinking about is something that in the remote client is listenning in a local TCP port for incomming data from the DAQ software, once it receives a message it streams it to the server and writes it to the local disk. In the server, the tool receives the message and re-streams it over local network to the final server. Then, notifies the client that is able to delete the message from its disk buffer.

So, the question is, there is something already done? I would prefer an already compiled / language agnostic solution because I code in LabView and I know there isn't like that in its ecosystem, but I'm open to everything. If there isn't anything like that, any advice in what to do / to avoid when developing it myself?

Thanks for your time.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source