BERT2: How to use GPT2LMHeadModel to start a sentence, not complet it

Question

I am using GPT2LMHeadModel to change the way GPT2 choose the next word in a sentence. At this point, I have to give the initial part of the sentence and GTP2 starts to predict the better next word.

I want GPT2 to read an entire sentence and then start a new one based on that (like it does with translation)

this is an example of how I am using it:

def gera_palavras_candidatas(context, past):

  #global model
  global enc
  global stop_token

  if past == None:
    context = torch.tensor(context).unsqueeze(0)
  else: 
    context = torch.tensor([context[-1]]).unsqueeze(0)

  context = {'input_ids': context} 

  output = context
  prev = context

  with torch.no_grad():
    logits, past = model(**prev,  past_key_values=past, use_cache=True, return_dict=False)
    logits = logits[:, -1, :] 
    probs = F.softmax(logits, dim=-1).tolist()[0]
    probs = sorted(enumerate(probs), key=lambda x: x[1], reverse=True)
  return probs, past

stop_token = [enc.encoder[x] for x in ('<|endoftext|>', '.', '!', '?')]

initial_sentence= "What I am trying to say is"

context = enc.encode(initial_sentence)

candidate_words, past = generate_candidates(context, None)

print('Candidate words to complete the sentence "', initial_sentence, '": ')
print('Word        Probability         Score')
for i in range(0, 10):
  candidate_word = candidate_words[i]
  finalWord = enc.decode(candidate_word[0])
  count = zipf_frequency (finalWord, 'en',wordlist='large')
  print("%-15s" % finalWord , candidate_word[1], str(count))

Is there any kind of parameter that I need to set up in order to make GPT2 start a sentence from zero, not complete an initial one?