How to remove duplicates from a list of lists? [duplicate]

Question

I wanted to remove duplicates from a list of lists. I know the method in which we use a set and add our element lists as tuples as tuples are hashable. ex:

arr=[[1,2,4],[4,9,8],[1,2,4],[3,2,9],[1,4,2]]
ans=set()

for i in arr:
   ans.add(set(i))
print(ans)

when we print(ans) we get {(1,2,4),(4,9,8),(3,2,9),(1,4,2)} this method removes the extra duplicates of [1,2,4] but not [1,4,2] as it is different. can anyone suggest a method in which I can remove [1,,4,2] as a duplicate of [1,2,4]?

Thank you

Answer 1

You can use a frozenset as a hashable set:

arr=[[1,2,4],[4,9,8],[1,2,4],[3,2,9],[1,4,2]]
ans=set()

for i in arr:
    ans.add(frozenset(i))
print(ans)

Or, functional version:

set(map(frozenset, arr))

output: {frozenset({2, 3, 9}), frozenset({1, 2, 4}), frozenset({4, 8, 9})}

To get back a list of lists:

list(map(list,set(map(frozenset,arr))))

output: [[9, 2, 3], [1, 2, 4], [8, 9, 4]]

NB. the order of the lists and items in sub-lists is not guaranteed!

Answer 2

arr=[[1,2,4],[4,9,8],[1,2,4],[3,2,9],[1,4,2]]
ans=set()

for i in arr:
  i.sort()
  ans.add(tuple(i))
print(ans)