microsoft

/

Phi-3-mini-128k-instruct-onnx

like 182

Hey guys, I tried running the onnx model using this code "import os
import onnxruntime_genai as og

model_path = os.path.abspath("./models/phi2")

model = og.Model(model_path)

tokenizer = og.Tokenizer(model)

prompt = '''def print_prime(n):
"""
Print all primes between 1 and n
"""'''

tokens = tokenizer.encode(prompt)

params = og.GeneratorParams(model)
params.set_search_options({"max_length":200})
####### Add the following line to enable cuda graph by passing the maximum batch size.
####### params.try_use_cuda_graph_with_max_batch_size(16)
params.input_ids = tokens

output_tokens = model.generate(params)

text = tokenizer.decode(output_tokens)

print("Output:")
print(text)" Not sure why I'm getting this error can someone help with this?"Phi-3-mini-128k-instruct-onnx\cpu_and_mobile\cpu-int4-rtn-block-32\genai_config.json' JSON Error: Unknown value: eos_token_id at line 29 index 27"