Console isn't working even if the code is correct

Question: Frustated that the console is giving error messages only and not the right output, I just copy pasted the tutorial’s code to see if that works. To my surprise that also is not working. To add more, the console is not printing a single response! I don’t know what to do in this situation.

Repl link:

from transformers import TFAutoModelForSeq2SeqLM, AutoTokenizer
import gradio as gr

model = TFAutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-small")
tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-small")

def generate_text_flan(input_string, max_length):
  inputs = tokenizer(input_string, return_tensors="pt")
  outputs = model.generate(**inputs, max_length=max_length)
  final_text = tokenizer.batch_decode(outputs[0], skip_special_tokens=True)

  return (final_text)

def to_gradio():
  demo = gr.Interface(fn=generate_text_flan,
                      inputs=["text", gr.Slider(0, 250)],
  demo.launch(debug=True, share=True)

if __name__ == "__main__":

Is there any specific reason that you use PyTorch tensors here: (return_tensors="pt" ), but then you import TFAutoModelForSeq2SeqLM from transformers , which is TensorFlow?


i dont know to be honest. i am yet to make sense of the code.

@SaarthakSaxena1 what errors are you getting? Have you tried using pip install module-name or poetry add module-name (make sure to replace module-name with the name of your module, like gradio for example) for your imports?

1 Like

It’s because since you are using two different types of tensor processing it can lead to some errors.
You should use AutoModelForSeq2SeqLM instead of TFAutoModelForSeq2SeqLM if you’re working with PyTorch.

1 Like