Unable To Use Google Gemma 7B Huggingface Model

RedCoder · March 8, 2024, 3:39am

Question:
I am unable to use the Google Gemma 7B Huggingface Model. This is the error I get:

2024-03-08 03:34:36.545587: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-03-08 03:34:36.551405: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-03-08 03:34:36.622989: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-08 03:34:37.960448: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Traceback (most recent call last):
  File "/home/runner/[REDACTED]/main.py", line 13, in <module>
    classifier = pipeline('text-classification', model='google/gemma-7b', token=os.environ['API_KEY'])
  File "/home/runner/[REDACTED]/.pythonlibs/lib/python3.10/site-packages/transformers/pipelines/__init__.py", line 905, in pipeline
    framework, model = infer_framework_load_model(
  File "/home/runner/[REDACTED]/.pythonlibs/lib/python3.10/site-packages/transformers/pipelines/base.py", line 292, in infer_framework_load_model
    raise ValueError(
ValueError: Could not load model google/gemma-7b with any of the following classes: (<class 'transformers.models.auto.modeling_tf_auto.TFAutoModelForSequenceClassification'>,). See the original errors:

while loading with TFAutoModelForSequenceClassification, an error is thrown:
Traceback (most recent call last):
  File "/home/runner/[REDACTED]/.pythonlibs/lib/python3.10/site-packages/transformers/pipelines/base.py", line 279, in infer_framework_load_model
    model = model_class.from_pretrained(model, **kwargs)
  File "/home/runner/[REDACTED]/.pythonlibs/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
    raise ValueError(
ValueError: Unrecognized configuration class <class 'transformers.models.gemma.configuration_gemma.GemmaConfig'> for this kind of AutoModel: TFAutoModelForSequenceClassification.
Model type should be one of AlbertConfig, BartConfig, BertConfig, CamembertConfig, ConvBertConfig, CTRLConfig, DebertaConfig, DebertaV2Config, DistilBertConfig, ElectraConfig, EsmConfig, FlaubertConfig, FunnelConfig, GPT2Config, GPT2Config, GPTJConfig, LayoutLMConfig, LayoutLMv3Config, LongformerConfig, MobileBertConfig, MPNetConfig, OpenAIGPTConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoFormerConfig, TapasConfig, TransfoXLConfig, XLMConfig, XLMRobertaConfig, XLNetConfig.

Firepup650 · March 8, 2024, 3:45am

Perhaps this gihub issue?

github.com/huggingface/transformers

Could not load model deepset/minilm-uncased-squad2

opened 10:10AM - 20 Apr 22 UTC

closed 03:01PM - 29 May 22 UTC

IavTavares

bug

### System Info ```shell I'm trying to load the model "deepset/minilm-uncase…d-squad2". On my laptop (Ubuntu 20.4 LTS), there's no problem. This happens when I run the exact same code on a server running Linux(see version below). Here's the output of transformer-cli env command: - `transformers` version: 4.18.0 - Platform: Linux-5.13.0-1021-aws-x86_64-with-glibc2.29 - Python version: 3.8.10 - Huggingface_hub version: 0.5.1 - PyTorch version (GPU?): not installed (NA) - Tensorflow version (GPU?): 2.8.0 (False) - Flax version (CPU?/GPU?/TPU?): not installed (NA) - Jax version: not installed - JaxLib version: not installed - Using GPU in script?: No (I'm manually filling this in) - Using distributed or parallel set-up in script?: No (I'm manually filling this in) Here's the error message: ValueError: Could not load model deepset/minilm-uncased-squad2 with any of the following classes: (<class 'transformers.models.auto.modeling_tf_auto.TFAutoModelForQuestionAnswering'>, <class 'transformers.models.bert.modeling_tf_bert.TFBertForQuestionAnswering'> On this [github issue](https://github.com/huggingface/transformers/issues/353) they point to a memory failure. However, to solve this, I had to download the pytorch version for CPU! ``` ### Who can help? @Rocketknight1, @LysandreJik,@Narsil ### Information - [ ] The official example scripts - [X] My own modified scripts ### Tasks - [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...) - [ ] My own task or dataset (give details below) ### Reproduction model_checkpoint = "deepset/minilm-uncased-squad2" device = -1 model_checkpoint = pipeline('question-answering', model=model_checkpoint, tokenizer=model_checkpoint, device=device) ### Expected behavior ```shell No error output, and correct loading of the model. ```

Seems like you need might PyTorch (CPU)?

RedCoder · March 8, 2024, 3:52am

Attempting now (wow this repl takes up 26.5 GiB of my storage )

Status: Had to delete it

@Firepup650 Is there a way to not install it and use it without an installation?

NuclearPasta0 · March 8, 2024, 5:12am

Generally, installing it using nix packages instead of python packages would be the way to go, but the nix repository doesn’t have all the packages, it probably doesn’t have this package.