Deviant and irregular execution times
This is kind of related to my previous post here.
I've sent an in app support request showing a video on deviant execution times: #182588
We have two BuildShip workflows that we use for an AI Chatbot (a "Trigger" and a "Message" workflow). Our goal is to reach a super low latency response. Though both workflows show extraordinary response times in some cases.
My concerns are these:
1. "Get document" node sometimes takes several hundreds of milliseconds and sometimes even more than a second to retrieve a simple database line from the BuildShip database (even though there is not much data inside)
2. "Execute Workflow" node can also take several hundreds of milliseconds to be executed even though the remote workflow just runs some simple javascript code.
3. "Execute Workflow" node apparently takes up extra time to "connect" to the remote workflow which adds further unnecessary latency. Are you doing an internal REST API call with this node?
4. Several nodes like "Get document" or "Execute workflow" have very deviating execution times even though they always have the same complexity
5. REST API call of a workflow adds several hundreds of milliseconds (see attached images: upper diagram shows REST API call response time and lower diagram shows internal workflow execution time which are manually recorded by extra nodes in the beginning and end of a workflow)
6. Why are some super simple Javascript nodes running several hundreds of a milliseconds sometimes? Is every node basically adding significant delay and in best case scenario we should put most code into one node?
7. Is there something faster than a REST API call that we can do to execute a BuildShip workflow?
8. Do you have several servers running across the globe to allow the clients to connect to the server with the best ping possible?
It would be great if you can take the time and respond to every single point here.
7 Replies
AI Support Bot Information
<@1227983829856157809> you can react on the relevant answer (message) with a ✅ in this thread when you think it has been solved, by the bot or by a human!
Anyone can react on this message with a ❌ if the GPT bot is unhelpful or hallucinating answers.
Please note: Team members will review and answer the questions on best efforts basis.
AI Response
I recognize the complexity you're facing with response times and the execution paths for your BuildShip workflows. Here's a breakdown addressing each of your points based on available information:
1. Variable Execution Times for "Get Document": This could be an issue of workflow configuration on the platform you are running on or possibly network-related. Generally, execution time variations can happen if there are external factors such as network latency or load on the data service involved.
2. Variability in "Execute Workflow" Node Timing: Delays in communication between different workflows can occur due to multiple reasons: server load, network latency, or internal processing delays. It is indeed possible that an internal API call is being made, which can account for the delay you notice.
3. Extra Time in "Execute Workflow" Connections: It's quite possible that inter-workflow communication is treated as an internal web service call, which does introduce some overhead.
4. Execution Time Deviations Across Nodes: This can be due to a range of factors such as the conditional evaluations, the data being processed, and available compute resources.
5. REST API Call Response Time: The addition of REST API calls introduces network latency, as well as time for setting up connections and possible cold starts if you haven't accessed the service recently .
6. Simple JavaScript Nodes Running into Delays: Every node being a virtualized instance can have variable cold start times and execution times which could result in delays.
7. Faster Than REST API Call for Workflow Execution: Other faster means, like direct code access or serverless function calls without REST API overhead, might be present within BuildShip according to your plan . Options like having workflows closer to your backend services or looking into paid plans allowing "no cold start" would help.
8. Globally Distributed Servers?: Usually, SaaS providers like BuildShip leverage multiple data centers to cater to a global audience, but specifics would depend on BuildShip's deployment model (not elaborated in available resources). Adjusting your project to the nearest server region might help with latency depending on your plan.
For a more authoritative and detailed analysis pertinent to your current setup and any possible optimizations available on your particular plan, I'd suggest reaching out directly to BuildShip support for real-time assistance. You can send a support request via the in-app support form as you're also considering.
BuildShip AI Support
Here is another diagram (the "Trigger workflow" as shown in my jam video)
Hey @Brian, let's connect over a call. Also will share an email later covering all these points. Sent you a DM with my calendar.
@Gaurav Chadha Hey I just tested my workflows again and I cannot really see any better results. I updated the "Get Document" node's code and I am also using the new "Execute Workflow" node wherever I have to await its remote workflow execution.
In the attached diagram you can see again that the request took suspiciously long.
I also sent some in app request again: #515450
Hi @Brian quick question: how are you wrapping the API Call function on your client side? Are you using axios, fetch or any other?
The statistic with the purple bar you can see is recorded withint he BuildShip workflow itself. I start a timer at the very beginning of the workflow and stop the timer at the very end before returning it. The BuildShip workflow is actually called by an Azure VM which is doing a python "requests" request.