How to store Dataframe data to Firebase Storage?

Question

Given a pandas Dataframe which contains some data, what is the best to store this data to Firebase?

Should I convert the Dataframe to a local file (e.g. .csv, .txt) and then upload it on Firebase Storage, or is it also possible to directly store the pandas Dataframe without conversion? Or are there better best practices?

Update 01/03 - So far I've come with this solution, which requires writing a csv file locally, then reading it in and uploading it and then deleting the local file. I doubt however that this is the most efficient method, thus I would like to know if it can be done better and quicker?

import os
import firebase_admin
from firebase_admin import db, storage

cred   = firebase_admin.credentials.Certificate(cert_json)
app    = firebase_admin.initialize_app(cred, config)
bucket = storage.bucket(app=app)

def upload_df(df, data_id):
    """
    Upload a Dataframe as a csv to Firebase Storage
    :return: storage_ref
    """

    # Storage location + extension
    storage_ref = data_id + ".csv"

    # Store locally
    df.to_csv(data_id)

    # Upload to Firebase Storage
    blob    = bucket.blob(storage_ref)
    with open(data_id,'rb') as local_file:
        blob.upload_from_file(local_file)

    # Delete locally
    os.remove(data_id)

    return storage_ref

Answer 1

if you just want to reduce code length and the steps of creating and deleting files, you can use upload_from_string:

import firebase_admin
from firebase_admin import db, storage

cred   = firebase_admin.credentials.Certificate(cert_json)
app    = firebase_admin.initialize_app(cred, config)
bucket = storage.bucket(app=app)

def upload_df(df, data_id):
    """
    Upload a Dataframe as a csv to Firebase Storage
    :return: storage_ref
    """
    storage_ref = data_id + '.csv'
    blob = bucket.blob(storage_ref)
    blob.upload_from_string(df.to_csv())

    return storage_ref

https://googleapis.github.io/google-cloud-python/latest/storage/blobs.html#google.cloud.storage.blob.Blob.upload_from_string

Answer 2

After figuring out for hours, the following solution works for me. You need to convert your csv file to bytes & then upload it.

import pyrebase
import pandas

firebaseConfig = {
   "apiKey": "xxxxx",
   "authDomain": "xxxxx",
   "projectId": "xxxxx",
   "storageBucket": "xxxxx",
   "messagingSenderId": "xxxxx",
   "appId": "xxxxx",
   "databaseURL":"xxxxx"
};

firebase = pyrebase.initialize_app(firebaseConfig)

storage = firebase.storage()

df = pd.read_csv("/content/Future Prices.csv")

# here is the magic. Convert your csv file to bytes and then upload it
df_string = df.to_csv(index=False)
db_bytes = bytes(df_string, 'utf8')

fileName = "Future Prices.csv"

storage.child("predictions/" + fileName).put(db_bytes)

That's all Happy Coding!