'spark agg multi collect_list. Can we guarantee that the index of multiple columns in the same row is the same?

Is it possible to ensure that the value at the same index of each Collect_set is on a single line of the original dataframe?

        ("a",1)
        ,("b",2)
    ).toDF("c1","c2")
    df.agg(collect_list($"c1"),collect_list($"c2")).show()
    // now print
    // +----------------+----------------+
    //|collect_list(c1)|collect_list(c2)|
    //+----------------+----------------+
    //|          [a, b]|          [1, 2]|
    //+----------------+----------------+

    //Is it possible for the following situation to occur
    // +----------------+----------------+
    //|collect_list(c1)|collect_list(c2)|
    //+----------------+----------------+
    //|          [a, b]|          [2, 1]|
    //+----------------+----------------+

apache-spark

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source

'spark agg multi collect_list. Can we guarantee that the index of multiple columns in the same row is the same?

Sources

Related Questions