'How to create an edge list using a column of node path of ArrayType in Spark?
I have a Spark Dataset containing a single column of ArrayType which denotes the path from one user to another through their mutual friends
| path |
|---|
| ["Amy","John","Wally"] |
| ["Beth","Sally","Tim","Jacob"] |
What I would like to achieve in the end is a table that explicitly lists the edges in the paths. (i.e. an edgelist)
| src | dest |
|---|---|
| "Amy" | "John" |
| "John" | "Amy" |
| "John" | "Wally" |
| "Beth" | "Sally" |
| "Sally" | "Tim" |
| "Tim" | "Sally" |
| "Tim" | "Jacob" |
| "Jacob" | "Tim" |
How should I go about trying to transform the former table into the latter one?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|