how to find the sql string being passed to hive table from spark apis like filter

Question

I am facing issues due to length of string being passed to hive table when I form a filter query. Something like:

val filtered = df.filter(col("name").isin(list))

now when I do filter.show it fails as it hits the char limit of 2500 characters.

What I want to find is : given filtered dataframe, how to know what query in form of string will be passed internally.

Answer 1

You can inspect this in the execution plan: filtered.explain()