'Dot product of two columns containing sparse vectors
I have a pyspark dataframe with two columns containing sparse vectors. I want to create a third column (Column C) containing the element-wise product of these two column
Column A:
(262144,[45252,99197,108625],[1.0,2.0,1.0])
Column B
(262144,[45252,99197,108625],[1.252762968495368,1.6945957207744073,1.252762968495368])
I'm looking to get:
Column C
(262144,[45252,99197,108625],[1/1.252762968495368,2/1.6945957207744073,1/1.252762968495368])
How can I do it in the most scalable way? Thank you.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
