You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm having trouble loading Gemma2 2B It, which has bf16 weights, rather than fp16. Is this something easily fixed? Using numpy 1.26.4 and torch 2.2.2+cu121.
Loading checkpoint shards: 100%|█████████████████| 2/2 [00:00<00:00, 11.33it/s]
0%| | 0/74 [00:00<?, ?it/s]C:\users\jim\appdata\local\programs\python\python311\Lib\site-packages\transformers\models\gemma2\modeling_gemma2.py:458: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.)
attn_output = torch.nn.functional.scaled_dot_product_attention(
0%| | 0/74 [00:00<?, ?it/s]
Traceback (most recent call last):
File "C:\cygwin64\home\Jim\chat\repeng\example.py", line 59, in
trippy_vector = ControlVector.train(model, tokenizer, trippy_dataset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\cygwin64\home\Jim\chat\repeng\repeng\extract.py", line 51, in train
dirs = read_representations(
^^^^^^^^^^^^^^^^^^^^^
File "C:\cygwin64\home\Jim\chat\repeng\repeng\extract.py", line 202, in read_representations
layer_hiddens = batched_get_hiddens(
^^^^^^^^^^^^^^^^^^^^
File "C:\cygwin64\home\Jim\chat\repeng\repeng\extract.py", line 293, in batched_get_hiddens
.numpy()
^^^^^^^
TypeError: Got unsupported ScalarType BFloat16
The text was updated successfully, but these errors were encountered:
I'm having trouble loading Gemma2 2B It, which has bf16 weights, rather than fp16. Is this something easily fixed? Using numpy 1.26.4 and torch 2.2.2+cu121.
Loading checkpoint shards: 100%|█████████████████| 2/2 [00:00<00:00, 11.33it/s]
0%| | 0/74 [00:00<?, ?it/s]C:\users\jim\appdata\local\programs\python\python311\Lib\site-packages\transformers\models\gemma2\modeling_gemma2.py:458: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.)
attn_output = torch.nn.functional.scaled_dot_product_attention(
0%| | 0/74 [00:00<?, ?it/s]
Traceback (most recent call last):
File "C:\cygwin64\home\Jim\chat\repeng\example.py", line 59, in
trippy_vector = ControlVector.train(model, tokenizer, trippy_dataset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\cygwin64\home\Jim\chat\repeng\repeng\extract.py", line 51, in train
dirs = read_representations(
^^^^^^^^^^^^^^^^^^^^^
File "C:\cygwin64\home\Jim\chat\repeng\repeng\extract.py", line 202, in read_representations
layer_hiddens = batched_get_hiddens(
^^^^^^^^^^^^^^^^^^^^
File "C:\cygwin64\home\Jim\chat\repeng\repeng\extract.py", line 293, in batched_get_hiddens
.numpy()
^^^^^^^
TypeError: Got unsupported ScalarType BFloat16
The text was updated successfully, but these errors were encountered: