'How to read a pandas dataframe in Databricks?
I don't know why, but my file located at "FileStore/tables/train.csv" is not readable using pandas in the Databricks platform. I tried :
pd.read_csv("/dbfs/FileStore/tables/train.csv")
and got
FileNotFoundError: [Errno 2] File /dbfs/FileStore/tables/train.csv does not exist: '/dbfs/FileStore/tables/train.csv'
Solution 1:[1]
Sorry for the late reply. You can read microsoft docs HERE
It will help you in copying the file uploaded in "FileStore" to "Tmp" location. This should enable your code like:
import pandas as pd
pd.read_csv(path)
and
import pyspark.pandas as ps
ps.read_csv(path)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Balram |
