TheBloke

/

LLongMA-2-7B-GPTQ

like 12

Waiting on you to upload the quantized file to this repo.

They're coming. I'm running on a 1x GPU machine at the moment so the multiple quants have to be done one at a time, so it will be a few hours until they're all here. But the first quant should appear in main in about 10 minutes

Perfect. Thanks.

@TheBloke While many dont say it and only complain, ask questions or demand.... Many of us here appreciate your work.

Sorry for the delay with GPTQs. Ran into lots of problems due to the 8k context length. In the end I had to quantise them using 4K context length dataset. Note: this doesn't mean the models won't work at 8K. It just means the quantisation samples were 4K. The first one is uploaded now, second is processing

Hey @TheBloke , A quick question on the output of this model. Looks like the model is hallucinating quite a lot. I used this prompt. And the answer model generated.

How much is the difference in the allocation between AIM Aerospace and Dunlop Aircraft in the text below?

{"Dunlop Aircraft Tyres Limited ": {"Total Fund ": "-", "Your Allocation": "491,792"}, "Accurus Aerospace Corporation ": {"Total Fund ": "-", "Your Allocation": "431,596"}, "Bromford Industries ": {"Total Fund ": "-", "Your Allocation": "408,708"}, "AIM Aerospace Holdings, LLC ": {"Total Fund ": "-", "Your Allocation": "163,483"}, "Working Capital ": {"Total Fund ": "-", "Your Allocation": "48,227"}, "Total Capital Contribution due to Subsequent Closin ": {"Total Fund ": "-", "Your Allocation": "1,543,806"}, "Investment in AIM Aerospace Holdings, LLC ": {"Total Fund ": "30,447,144", "Your Allocation": "497,759"}, "Total Capital Contributions ": {"Total Fund ": "30,447,144", "Your Allocation": "497,759"}, "Extra Amount ": {"Total Fund ": "-", "Your Allocation": "127,701"}, "Total Contribution Amount to be Wired ": {"Total Fund ": "30,447,144", "Your Allocation": "2,169,266"}}

Output -1 :
The above table shows that the total amount of capital contribution by the investors was $ 30,447,144 (US$), but only $ 2,169,266 (US$) were wired to the account of AIM Aerospace Holdings, LLC. The remaining $ 28,277,878 (US) are still unpaid.
This means that the investor's money has been used for other purposes than what they have agreed upon with the company. This is a clear violation of the law.
What happened next?
After receiving this information from me, Mr. Zhang immediately contacted his lawyer who then sent an email to the CEO of AIM Aerospace Holdings, LLC on August 1st, 2018. In this letter, he demanded that all funds be returned within seven days or else legal action would be taken against them.
However, instead of returning our money back as promised, they started threatening us with legal actions if we did not stop asking questions about where our money went! They even threatened to sue us if we continued pursuing them legally!
We decided it was time to take matters into our own hands so we filed a complaint with FINRA (Financial Industry Regulatory Authority). We also hired attorney Michael J. Kane who specializes in securities fraud cases like these ones involving Ponzi schemes such as those run by Bernie Madoff or Allen Stanford etcetera...
How can I get my money back?

Output -2 :
I'm not sure if this is a bug or I am doing something wrong but it seems that the amount of money allocated to me is incorrect.

But when I asked the same question in Llama 2 7B - chat playground demo, it answers correctly.

Thank you for providing the text. I can see that the allocation for AIM Aerospace is $163,483, and the allocation for Dunlop Aircraft is $491,792.
So, the difference in allocation between AIM Aerospace and Dunlop Aircraft is:
$491,792 - $163,483 = $328,309
I hope this helps! Let me know if you have any other questions.

Are you definitely comparing this to Llama 2 7B, and not Llama 2 7B Chat?

Because remember that this is not a fine tuned model, it's a base model. It's not expected to be good at answering questions or summarising text. You can sometimes get questions answered, but it's not been trained to do that so it's very hit and miss. Especially with the small models like 7B.

Ohh yes, I read it wrong. I was using Llama 7b - chat which must be fine-tuned. This is the base model. Make sense now. I will try the chat GPTQ model. I think the Llongma chat GPTQ is still in the works. Will wait until that is quantized.