'Importing pandas dataframe into elasticsearch eland,espandas or other methods?
Having some issues loading a pandas dataframe into elasticsearch. Any advice would help.
from elasticsearch import Elasticsearch
import eland as ed
import pandas as pd
from re import sub
es = Elasticsearch('http://localhost:9200/')
## Create a Pandas Dataframe of Data to be Loaded into Elasticsearch
df = cuny_campus_locations
## Replace NaN (null) Values with Zero
df.fillna(0, inplace=True)
# Rename the Columns to be Camel Case
def camel_case_string(string):
string = sub(r"(_|-)+", " ", string).title().replace(" ", "")
string = string[0].lower() + string[1:]
return string
df.columns = [camel_case_string(x) for x in df.columns]
## Save the Data into Elasticsearch
df = ed.pandas_to_eland(
pd_df=df,
es_client=es,
# Where the data will live in Elasticsearch
es_dest_index="test",
# Type overrides for certain columns, the default is keyword
# name has been set to free text and year to a date field.
es_type_overrides={
"id":"text"
},
# If the index already exists replace it
es_if_exists="replace",
# Wait for data to be indexed before returning
es_refresh=True,
)
Are there easier methods to import dataframes into elasticsearch?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|

