Console isn't working even if the code is correct

Code Help Python
1

Question: Frustated that the console is giving error messages only and not the right output, I just copy pasted the tutorial’s code to see if that works. To my surprise that also is not working. To add more, the console is not printing a single response! I don’t know what to do in this situation.

Repl link: https://replit.com/@SaarthakSaxena1/Day-3-Implementing-GPT3-and-Flan-T5

from transformers import TFAutoModelForSeq2SeqLM, AutoTokenizer
import gradio as gr

model = TFAutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-small")
tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-small")



def generate_text_flan(input_string, max_length):
  inputs = tokenizer(input_string, return_tensors="pt")
  outputs = model.generate(**inputs, max_length=max_length)
  final_text = tokenizer.batch_decode(outputs[0], skip_special_tokens=True)

  return (final_text)


def to_gradio():
  demo = gr.Interface(fn=generate_text_flan,
                      inputs=["text", gr.Slider(0, 250)],
                      outputs="text")
  demo.launch(debug=True, share=True)


if __name__ == "__main__":
  to_gradio()
2

Is there any specific reason that you use PyTorch tensors here: (return_tensors="pt" ), but then you import TFAutoModelForSeq2SeqLM from transformers , which is TensorFlow?

3 Likes
4

i dont know to be honest. i am yet to make sense of the code.

5

@SaarthakSaxena1 what errors are you getting? Have you tried using pip install module-name or poetry add module-name (make sure to replace module-name with the name of your module, like gradio for example) for your imports?

1 Like
6

It’s because since you are using two different types of tensor processing it can lead to some errors.
You should use AutoModelForSeq2SeqLM instead of TFAutoModelForSeq2SeqLM if you’re working with PyTorch.

1 Like