Pandas Dynamic Groupby and then using something like to_html() with Flask

Question

I am trying to display a grouped object into html with the to_hmtl(), however I am having trouble wrapping the object through render_template() and then displaying all the group dataframes with flask on my html page.

Python:

groups = pandas_table.groupby(pandas_table.City)
#pandas_table is my dataframe and City is a column
#also in my code pandas_table will be variable in size potentially containing a lot of
#different cities

So, imagine the groups object now has 3 separate dataframes with 3 keys being the city names (ex. Boston, New York, Los Angeles) with other columns like population and gdp or something else. I know to display this is in the terminal I'd do:

for city, city_df in groups:
    print(city, city_df)

And it would print the key (i.e. Boston, New York, etc.) and then each key's dataframe with city_name, city_population, city_gdp, etc. as columns in the dataframes. But, how can I pass this group through flask's render_template() and then write html and jinja to display each key's dataframe in the group.

Because something pseudo like:

{% for city, city_df in groups%}
<h2>{{city}}</h2>
{{city_df}}
{% endfor %}

Does not work. I want to use to_html(), but I do not think you can with grouped objects. My project involves a dynamic pandas dataframe dependent on the user, so the group size is variable and revolves around say...a user inputting a series of cities, and then scraping the web to get data on each city.

So what can I do?

Answer 1

Jinja probably generates the templates beforehand so my solution should be avoided if you're dealing with huge data which might cause memory problems. Also be careful using the safe filter. All in all it's definitely doable and it can be tweaked depending on what you are trying to achieve.

# app.py

import pandas
from flask import Flask, render_template

app = Flask(__name__, template_folder=".")

@app.route("/")
def main():
    d = {"col1": [1, 2, 3, 4, 5, 6], "col2": [1, 2, 2, 3, 3, 3]}
    df = pandas.DataFrame(d)
    ggs = df.groupby(df.col2)

    ggs = (gg.to_html() for _, gg in ggs)
    return render_template("example.html", ggs=ggs)
    

if __name__ == "__main__":
    app.run(port=5001, debug=True)

<!-- example.html -->

{% for gg in ggs %}
{{ gg|safe }}
{% endfor %}