https://cerebralvalley.ai logo
#06-technical-discussion
Title
# 06-technical-discussion
j

Jeevan Gyawali

08/14/2023, 2:27 AM
Folks my Vercel Edge-> LangChainStream -> ChatOpenAI chatbot setup, streaming is bugging out and losing conversation mid-stream after I add a system message with 8k tokens mid chat. Has anyone seen this streaming issue with LangChainStream -> ChatOpenAI packages before? My current hypotheses are: 1. There might be ChatOpenAI issues with heavily loaded context? 2. Is it possible that free tier Vercel is causing this weird behavior? 3. Is adding context via system message mid-chat the issue? Would it be better to start a new chat altogether? Any suggestions/intuitions would be helpful. cc: @Hao Sun
b

bruels

08/17/2023, 8:26 PM
@Jeevan Gyawali Just wondering if you made progress on this. Purely curiosity. My initial gut check was that: • you exhausted the “initial response” time from Edge or • you blew past the context window but you should be catching errors to that effect
2 Views