Scala bulkcopy not working in Azure DAtabricks runtime 7.3LTS and above

Question

My scala code that used to work fine with databricks runtime 5.5LTS is not working with runtime 7.3LTS and above. I have tried upgrading microsoft libraries according to the 7.3/ higher runtimes supporting scala 2.12. still no luck. the code that i am running is as below. I need to do a bulkcopy on the SQLMI, any help would be highly appreciated.

%scala
// import com.microsoft.azure.sqldb.spark.bulkcopy.BulkCopyMetadata
import com.microsoft.azure.sqldb.spark.config.Config
import com.microsoft.azure.sqldb.spark.connect._

val Datawriter = spark.sql("select * from writer")
val bulkCopyConfig = Config(Map(
                              "url"             -> jdbcHost,
                              "databaseName"    -> jdbcDatabase,
                              "user"            -> jdbcUsername,
                              "password" -> jdbcPassword,
                              "dbTable" -> jdbcTableName,
                              "bulkCopyBatchSize" -> "2000",
                              "bulkCopyTablelock" -> "true",
                              "bulkCopyTimeout" -> "6000"))