'Spark Scala - Create an object with element from my dataframe

For a Scala Project I am working on, I had to create a class Table as below :

class Table(PbddName : String , PTableName: String, POutputFilename: String="", PFilteringFlag: Boolean=true) {
  var bddName : String = PbddName
  var TableName: String = PTableName
  var OutputFilename: String = POutputFilename
  var FilteringFlag: Boolean =PFilteringFlag
}

To create a Table, I use this code in a Scala object :

        val randomName = new Table(bddName, blablaTableName, blablaOutputFilename,blablaFilteringFlag)
        list_tables+=randomName

I also add it to list_Tables that obviously list all the tables.

But as I have a lot of "Tables" to create, I made a csv where each line contains the needed values to create a Table (matching the values of the class). I loaded my CSV file in a DataFrame called empDF that matches this :

|bdd_name|    table_name|     file_name|filtering_flag|
+--------+--------------+--------------+--------------+
|    bdd1|name1tablename| name1Filename|          true|
|    bdd2|name2tablename| name2Filename|          true|
|    bdd3|name3tablename| name3Filename|         false|
|    bdd4|name4tablename| name4Filename|          true|
+--------+--------------+--------------+--------------+

I would like to know how to automate the creation of my "Tables" elements ?(and add the created table to my list_Tables). I am not sure, but I think I might have to browse my dataframe somehow to create the element at each line that is readden.

Also, each val name (here randomName) must be different, but can be random.

If you have any idea, or any clue on how I could do this, that would help me a lot.

Thank's for your help.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source