Cannot deploy Phi-3-mini-4k-instruct model on AWS Sagemaker
#51
by
MajTad
- opened
I try to run a sagemaker endpoint with the Phi-3-mini-4k-instruct using HuggingFaceModel to load it but I got from Sagemaker logs the following error:
ValueError: Unsupported model type phi3.
Does anyone face the same problem.
It's only made to run on Azure
/s
Seriously, not all models are supported by the HuggingFaceModel feature on SageMaker. You can just deploy it on a regular gpu-powered vm using vLLM or TGI
nguyenbh
changed discussion status to
closed