How to iterate though each dictionary from the list in Scala?

Question

I have a JSON string, which basically is a list of dictionaries. I am trying to use a for loop to iterate through it. Basically in each iteration, I should be able to get the keys such as index, name, source, s3. Can someone please help? So in first iteration I should be able to get index=1,source=a etc.

import scala.util.parsing.json._
class CC[T] { def unapply(a:Any):Option[T] = Some(a.asInstanceOf[T]) }

object M extends CC[Map[String, Any]]
object A extends CC[List[Any]] //for s3
object I extends CC[Double]
object S extends CC[String]
object E extends CC[String]
object F extends CC[String]
object G extends CC[Map[String, Any]]

val jsonString =
    """
    [{
        "index": 1,
      "source": "a",
      "name": "v",
      "s3": [{
        "path": "s3://1",
        "bucket": "p",
        "key": "r"
      }]
     },
     {
        "index": 2,
      "source": "b",
      "name": "b",
      "s3": [{
        "path": "s3://1",
        "bucket": "p",
        "key": "r"
      }]
     }]
     
    """.stripMargin
    
println(List(JSON.parseFull(jsonString)) )

Answer 1

If you want to do it without spark, then you can try this way:

JSON.parseFull(jsonString) match {
    case Some(x) => {
        val json_list = x.asInstanceOf[List[Map[String, Any]]]
        json_list.foreach { json_map =>
            // To extract any base level fields

            // You can do any action on the extracted field. This is just to illustrate the extraction method
            println("index: " + json_map("index"))
            println("source: " + json_map("source"))

            // To extract fields from inside a list: example element at index 0 of list s3

            // You can do any action on the extracted field. This is just to illustrate the extraction method
            val s3_list = json_map("s3").asInstanceOf[List[Any]]
            println("s3: " + s3_list(0))

            // To extract fields from inside a map: example path from 0th element of list s3

            // You can do any action on the extracted field. This is just to illustrate the extraction method
            println("s3.path: " + s3_list(0).asInstanceOf[Map[String, Any]]("path"))

        }
    }
    case None => {
        println("Improper JSON")
    }
}

// Output of the print statements

// Iteration #1:

index: 1.0
source: a
s3: Map(path -> s3://1, bucket -> p, key -> r)
s3.path: s3://1

// Iteration #2:

index: 2.0
source: b
s3: Map(path -> s3://1, bucket -> p, key -> r)
s3.path: s3://1