hey all - I'm getting a really weird bug from OpenAI. when I prompt it with something like "please summarize this text" I get repeating words like this: "I'm proficient in Python, SQL, and machine/deep learning, and I'm passionate, and I'm a quick, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and," any ideas?

Any chance you're using Semantic-Kernel? There was a bug in earlier versions that could cause issues like that. If you're not using Semantic-Kernel, I think the root cause was passing an invalid model identifier (eg. not gpt-3.5-turbo or etc) to OpenAI in your API calls

this is coming straight out of the OpenAI api so no Semantic Kernel

Right but could (possibly) have the same root cause of a bad model specifier being sent to OpenAI by the caller

got it. so basically just a typo or something when setting the model?

My guess in the situation I encountered was OpenAI was somehow defaulting to a crappy ancient internal test model (there are around 100? models that can be specified by the API).

hm looks good to me

yeah, that does look sensible

i dont think so

I haven't played with contexts that can overflow the working memory of the GPU. Is it possible you're sending it too big a document (hopefully they return a sensible error but since I've never done that I don't know personally how they handle it)

we're sending emails over so honsetly not crazy

^ pure guess on that last one, just trying to be helpful but probly not succeeding 🙂

yea haha

My guess is Azure's version is more likely to have support than OpenAI (same models and APIs, just supported by a different org and billed through a different company)

ah

(I've also heard the Azure variant has significantly faster response times, but I haven't confirmed that myself)

we're on AWS

The Azure folks probably wont mind an opportunity to try to convince you of their goodness lol

yea haha

OpenAI might have support as well, they're just a newer and smaller company and good support orgs tend to take time to build

yea - im gonna bug my investors to get in front of openai and see if that works lol

This is a hallucination error, modify the prompt to let it know the tone that it needs to use or use CoT prompting. For eg: Summarize the text such that it can be used in my LinkedIn profile, and give reasons for why.

whats CoT?

Chain of thought prompting

ah

Can you try what I suggested and let me know?

yea will have to wait for tomorrow

check your frequency penalty. also are you using chat or completions api? feel like chat has been RLHF'ed more so it's less likely to do this whereas completions is probably closer to the original models but tends to fall into pits like this

omg thank you!!

yesss let's goo!!!