Excessive Token Usage with OpenAI Assistant Functions in WhatsApp Chatbot via Buildship
Original message was deleted
RHello,
I'm currently experiencing an issue with my OpenAI assistant integrated into a WhatsApp chatbot via Buildship. The assistant has a few functions like sending emails and making calls, which are executed through standard API calls. However, I've noticed that when these functions are executed within a conversation, the number of tokens, and consequently the cost, skyrockets.
For instance, in a short conversation of just 8 messages between the user and the assistant, where the function was executed twice, the total token usage was 55,252 tokens—54,866 tokens in and 386 tokens out. I'm unsure if this issue is due to how Buildship sends information to OpenAI within the flow or if it's a matter of how OpenAI processes the function. Regardless, this makes it nearly impossible to use these tools without significant financial resources.
Could you please confirm if this is an error? (i used buildship template for the assistant, and added functions, nothing fancy) In fact, a few weeks ago, during testing, I managed to reach the maximum token limit for the day by myself, simply by sending a few messages and executing the functions. This indicates that the number of tokens used for executing these functions is excessively high.
Any insights or solutions to this problem would be greatly appreciated.
I'm currently experiencing an issue with my OpenAI assistant integrated into a WhatsApp chatbot via Buildship. The assistant has a few functions like sending emails and making calls, which are executed through standard API calls. However, I've noticed that when these functions are executed within a conversation, the number of tokens, and consequently the cost, skyrockets.
For instance, in a short conversation of just 8 messages between the user and the assistant, where the function was executed twice, the total token usage was 55,252 tokens—54,866 tokens in and 386 tokens out. I'm unsure if this issue is due to how Buildship sends information to OpenAI within the flow or if it's a matter of how OpenAI processes the function. Regardless, this makes it nearly impossible to use these tools without significant financial resources.
Could you please confirm if this is an error? (i used buildship template for the assistant, and added functions, nothing fancy) In fact, a few weeks ago, during testing, I managed to reach the maximum token limit for the day by myself, simply by sending a few messages and executing the functions. This indicates that the number of tokens used for executing these functions is excessively high.
Any insights or solutions to this problem would be greatly appreciated.
GHi @ruben.romero, could you please submit your workflow via in-app support form? We'll take a look and will get back, as 55k of token usage seems too much. https://docs.buildship.com/support-messages
A unified resource to start building your backend with low-code. Dive into triggers, nodes, and step-by-step guidance to jumpstart your workflow creation.
RThank you @Gaurav Chadha I already did, i also attached one of the conversations with the excessive amount of tokens, and it only has 15 messages in total!
RHi @Gaurav Chadha
I sent you a response email, but I forgot to mention something important. Let's say I reduce the length of the instructions. Even with shorter instructions, I still reached the token limit by using functions. I remember that all I did was tell the assistant to execute those functions to ensure everything was working, and suddenly, I received a response stating that I had reached the daily token limit for my assistant.
I'm pretty sure the issue is that the large JSON required to execute the functions is being sent as input, and the equally large JSON response from the functions is also being counted as input. This makes it impossible to use functions in a production environment with multiple users if a single user can reach the token limit in one day just by testing.
Even switching the model to ChatGPT-3.5 to reduce costs won’t work if a single user reaches the token limit so quickly.
I sent you a response email, but I forgot to mention something important. Let's say I reduce the length of the instructions. Even with shorter instructions, I still reached the token limit by using functions. I remember that all I did was tell the assistant to execute those functions to ensure everything was working, and suddenly, I received a response stating that I had reached the daily token limit for my assistant.
I'm pretty sure the issue is that the large JSON required to execute the functions is being sent as input, and the equally large JSON response from the functions is also being counted as input. This makes it impossible to use functions in a production environment with multiple users if a single user can reach the token limit in one day just by testing.
Even switching the model to ChatGPT-3.5 to reduce costs won’t work if a single user reaches the token limit so quickly.
GHi @ruben.romero, an update on this - our team investigated this and found that it's the Assistant which is using the excessive tokens, and yes you came to the right conclusion about the Assistant to use function which ended up in having excessive token usage.
Ref - https://community.openai.com/t/assistant-api-way-too-much-input-tokens-used/699661/7.
We'll be updating our docs to highlight this in more details along with guidelines to prevent the excessive usage of token.
Ref - https://community.openai.com/t/assistant-api-way-too-much-input-tokens-used/699661/7.
We'll be updating our docs to highlight this in more details along with guidelines to prevent the excessive usage of token.