'How to replace a value of smaller length than one with bigger length in pandas dataframe?
I have a dataset which I read as an pandas dataframe. I wanna tokenize one column of the dataset. My tokenized value is a list with bigger length than my original cell value.
Error message:
ValueError: Replacement lists must match in length. Expecting 1 got 72
How can I do this replacement in my dataframe?
Code where I tried to iterate over and replace my values
trainData = pd.read_csv("data/train.csv")
test = trainData[0:100]
CODEBERTA_LANGUAGE_ID = "huggingface/CodeBERTa-language-id"
tokenizer = RobertaTokenizer.from_pretrained(CODEBERTA_LANGUAGE_ID)
for i in test["code"]:
test["code"] = test["code"].replace([i], tokenizer.encode(i))
Image example of my dataframe
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|

