Latex expressions in pandas dataframes not rendering in vscode

Question

I am trying to set some labels and the caption of my dataframe using mathjax, but it doesn't render in vscode.

For example, when I do

import pandas as pd
test = pd.DataFrame(index=["$\sigma$"])
test

I get this as output in my jupyter notebook:

$\sigma$

Instead of the letter σ. I get the right results if I do the same thing in JupyterLab (in Google Chrome).

Note: Other packages that use mathjax seem to be working fine. Printing latex expressions in matplotlib, sympy and even cells marked with '%%latex' work fine.