'How to mock and test Databricks Pyspark notebooks Locally

How can I mock existing Azure Databricks PySpark codes of a project (written by others) and run them locally on windows machine/Anaconda to test and practice?

Is it possible to mock the codes or I need to create a new cluster on Databricks for my own testing purposes?

how I can connect to storage account, use the Databricks Utilities, etc? I only have experience with Python & GCP and just joined a Databricks project and need to run the cells one by one to see the result and modify if required.

Thanks

Solution 1:^[1]

You can test/run PySpark code from your IDE by installing PySpark on your local computer.

Now, to use Databricks Utilities, in fact you would need a Databricks instance and it's not available on local. You can try Databricks community Editionfor free but with some limitation

To acess a cloud storage account, it can be done locally from your computer or from your own Databricks instance. In both cases your will have to set up the end point of this storage account using its secrets.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	Luiz Viola

'How to mock and test Databricks Pyspark notebooks Locally

Solution 1:[1]

Sources

Related Questions

Solution 1:^[1]