Please improve AI Assistants Streaming template
I have tested the templates for both Streaming Chatbot and Streaming API.
On the high level, they are working for demo puposes. But for a robust solution, none of them meets the requirements.
1. For AI Chatbot Streaming: it doesn't parse the threadId, therefore, the chat session does not retain the the previous context.
2. For AI Assistant API: The output only shows the message(stream), but no threadId meta too. And the stream part is not json.
I have successfully implemented the AI Assistant in my project. It's robust. Currently exploring the streaming approach. (not the deal breaker, but nice to have for UI experience) ,
The current template, or node only produces a response like :
"stream": {
"_events": {},
"_readableState": {
"highWaterMark": 16384,
"buffer": [],
"bufferIndex": 0,
"length": 0,
"pipes": [],
"awaitDrainWriters": null
}
},
"threadId": "thread_vfGivteoekohrQUgz75OTzan"
}
with the Set Header approach, it produces the actual message, but the threadId is missing.
Please suggest!
Also, I found some of the tutorials from the past by buildship, and they are outdated.
On the high level, they are working for demo puposes. But for a robust solution, none of them meets the requirements.
1. For AI Chatbot Streaming: it doesn't parse the threadId, therefore, the chat session does not retain the the previous context.
2. For AI Assistant API: The output only shows the message(stream), but no threadId meta too. And the stream part is not json.
I have successfully implemented the AI Assistant in my project. It's robust. Currently exploring the streaming approach. (not the deal breaker, but nice to have for UI experience) ,
The current template, or node only produces a response like :
"stream": {
"_events": {},
"_readableState": {
"highWaterMark": 16384,
"buffer": [],
"bufferIndex": 0,
"length": 0,
"pipes": [],
"awaitDrainWriters": null
}
},
"threadId": "thread_vfGivteoekohrQUgz75OTzan"
}
with the Set Header approach, it produces the actual message, but the threadId is missing.
Please suggest!
Also, I found some of the tutorials from the past by buildship, and they are outdated.
Solution
I have finally figured out.
Here is the solution.
In case you don't get the Json output. You need to manually modify the node's code. BuildShip engineers put good comments there.
Just uncomment the for the line stream.push("data: [DONE]\n\n");
and change the parameter to "Server Sent Events".
Here is the solution.
In case you don't get the Json output. You need to manually modify the node's code. BuildShip engineers put good comments there.
Just uncomment the for the line stream.push("data: [DONE]\n\n");
and change the parameter to "Server Sent Events".
