'How to implement a transformer that takes a sequence of float arrays and outputs a sequence of float array

I have a time-series problem with 966 samples, 139 time-steps, and fixed length float vectors of tunable size n. As output, I need a transformer model that takes a sequence (time-steps) of fixed length float vectors and outputs the same sequence (time-steps) of different length (but still fixed-length) float vectors.

Somewhat more formal problem statement: [a_1, ..., a_n] => [b_1, ..., b_n] where each a = [x1_, ..., x_m] and each b = [y_1, ..., y_p].

My problem is, I do not know how to implement transformers that are float array seq to float array seq (non-NLP/image/audio based). I believe I essentially need to bypass the tokenizer, as the tokenized values of words are essentially int seq to int seq values (to my understanding). Does anyone know how to implement this type of transformer in Python? Or should I just use an LSTM model?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source