How to remove input from from generated text in GPTNeo?

Question

I'm writing a program to generate text... I need to remove the input from the generated text. How can I do this? The code:

input_ids = tokenizer(context, return_tensors="pt").input_ids
gen_tokens = model.generate(
    input_ids,
    do_sample=True,
    temperature=0.8,
    top_p=0.9)
strs = tokenizer.batch_decode(gen_tokens)[0]

Here the strs contains the input I've given... How to remove that?

Answer 1

The Transformers library does not provide you with a way to do it, but this is something you can easily achieve with 1 line of code:

strs = strs.replace(context,"")

This is actually what I'm doing behind my NLP Cloud API as it uses Transformers behind the hood.