Why am I getting an error in df.transform when I should be able to execute arbitrary functions?

Question

I believe that df.transform can perform arbitrary functions. For example df.transform(lambda x: x + 10) etc.

However, a simple math.ceil() gives me an error. This can be solved with np.ceil(), but why is there an error in df.transform, which should be able to execute arbitrary functions?

import math
import numpy as np
import pandas as pd

df = pd.DataFrame({'V':[1, 1.5, 2]})

df.transform(lambda x: x + 10)        #OK
df.transform(lambda x: np.ceil(x))    #OK

df.transform(lambda x: math.ceil(x))
TypeError: cannot convert the series to <class 'float'>  #NG. Why?

Answer 1

The function math.ceil gets a float as an input, and cannot work with pd.Series as input, while np.ceil is a vectorized function that can get either float or series (array) as input.

You should refer to a specific series:

df.V.transform(lambda x: math.ceil(x))

The other works fine since they are vectorized functions and can work on a pd.Series