'No values in the new column
When I type train_set['processed_Review'], there are no values in the column.
The train_set data frame looks like the image below.
punctuations = string.punctuation
stopwords = list(STOP_WORDS)
review = str(" ".join([i.lemma_ for i in doc]))
doc = nlp(review)
spacy.displacy.render(doc, style='ent',jupyter=True)
#Parser for reviews
parser = English()
def spacy_tokenizer(sentence):
mytokens = parser(sentence) # After tokenization, spaCy can parse
mytokens = [ word.lemma_.lower().strip() if word.lemma_ != "-PRON-" else word.lower_ for word in mytokens ]
mytokens = [ word for word in mytokens if word not in stopwords and word not in punctuations ]
mytokens = " ".join([i for i in mytokens])
return mytokens
tqdm.pandas()
train_set['processed_Review'] = train_set['Review'].progress_apply(spacy_tokenizer)
train_set['processed_Review']
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|


