Error in workflow Running
Hi everyone, I have a workflow which connected to cron trigger and in the workflow i have loop inside loop i have custom node .The custom node logic is it receive the url from loop and make api call to the url and fetched courses then enriched AI content in it and then save it to supabase. But the loop has 2.5k elements and each api call give me an average 500 courses. The issue is now that the workflow trigger and processes the courses from few pages and insert it into supabase too but after 15-20 mints the workflow stops. I put delays too where necessary to overcome rate limits of OPenAI and MapAPI.
2 Replies
AI Support Bot Information
<@1324707169257390131> you can react on the relevant answer (message) with a ✅ in this thread when you think it has been solved, by the bot or by a human!
Anyone can react on this message with a ❌ if the GPT bot is unhelpful or hallucinating answers.
Please note: Team members will review and answer the questions on best efforts basis.
AI Response
Dealing with a workflow that stops after 15-20 minutes despite implementing delays can be frustrating. Here are some potential solutions and considerations:
1. Rate Limiting: Ensure that your delays are sufficient to handle the rate limits imposed by both the OpenAI and MapAPI services. You might need to fine-tune delay settings based on the responses from these APIs.
2. Resource Overload: The large number of elements (2.5k) being processed might be causing resource exhaustion. Consider batching the processing to reduce the load on your system.
3. System Logs: Check the logs for any errors or warnings that occur right before the workflow stops. This can provide insights into potential issues like memory leaks or unhandled exceptions.
4. Timeout Settings: Review any timeout settings in your workflow or API calls that might be causing premature termination.
5. Parallel Processing: If possible, consider implementing parallel processing with a smaller batch size to improve efficiency while remaining within rate limits.
6. Error Handling: Make sure you have robust error handling to manage API errors or failed requests, and implement retries if feasible.
If none of these solutions address the problem, you may need to contact BuildShip support for more detailed diagnostics and assistance.
BuildShip AI Support