mukamuka
mukamuka•7mo ago

mukamuka - Hey guys, how do I fetch all the dat...

Hey guys, how do I fetch all the data from a paginated API that responds with {data, next} ?
5 Replies
Gaurav Chadha
Gaurav Chadha•7mo ago
Hey @mukamuka You can use the API Call node to fetch the data from your paginated API, given the data and nextare served as query parameters, example: /api/posts?data=121&next=true
mukamuka
mukamuka•7mo ago
Hey @Gaurav Chadha thanks for looking into this. So, I'm already using an API Call node to fetch the data. But how do I make it fetch the next URL from the response? I can't figure out how to make a "looping" API Call node, if that makes sense.
Gaurav Chadha
Gaurav Chadha•7mo ago
You can use implement to loop through the api call, returning the the next url each time it loop through it, let me know if it helps.
No description
mukamuka
mukamuka•7mo ago
@Gaurav Chadha So I am calling an external API from BuildShip The external API response is {data, nextURL} I store the data and then make another API Call to the next URL. In other words, I need a while loop for the API Call Node to call the nextURL to fetch the data This loop will run until there is no nextURL in the response from the external API --- In your example, your Loop node takes an Items argument, but you don't know how many URLs you have in advance. Also, the URL in your API Call node has to change to the next URL Calling external APIs that are paginated will be a very common task, so I wonder if there's a best practice setup already 🤔 Unless there is a "Buildship way" with already existing nodes, I think I will have to make a custom node
Gaurav Chadha
Gaurav Chadha•7mo ago
Gotcha, yeah, for while loop, you'll require to make a custom node to support this usecase.