V100上无法直接推理

by MonolithFoundation - opened 14 days ago

14 days ago

如果V100上安装了flashattn2，但是实际上用不了flashatten2，代码便无法推理。
设置了config atten_implementation 为sdpa，代码不认这个还是flashattn2.

10 days ago

This comment has been hidden

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment