How do I image upload capabilities on OpenAI assistant API on Buildship?
I followed this documentation to set up openai assistant node https://docs.buildship.com/ai-assistant/openai-assistant. It has thread ID and prompt in the body, but how do I add image upload to my assistant?
Solution:Jump to solution
1. Passing an image_url into the API call body.
2. Creating a conditional to check if the image_url is available.
3. If true, you will use the OpenAI Vision API and pass the response to the Assistant AI node.
4. If false, you will take the prompt and pass it to the OpenAI Assistant node....
33 Replies
Hey @Ada, what frontend are you using?
@Henry Moses I’m using flutterflow
You want to
1. Pass in image_url in the body
2. Then pass that through the assistant node
This will allow a user to upload a image on the frontend and you can pass that image through the assistant node
@Henry Moses I passed image_url in the body, but I don’t know how to pass it to the assistant node. I didn’t see any options to image_url key into it. Could you explain step by step? I would really appreciate it.
Yes, you can use the "upload file from URL" node. The URL will be the image_url, and the destination will be {unique-image-name}.png.
@Henry Moses this is how my nodes look like now, do I need to add image_url in the 3rd node (OpenAI assistant node)? sorry I'm very new with buildship
Really appreciate your help.
Hey!
Looking at this now
@Henry Moses thx. is it correct?
What response are you getting? And yes, everything is right, just curious if you're getting the response you need.
@Henry Moses when i test it with an image URL, it doesn't seem to see the image.
You could use the GPT-4 Vision API before the Assistant Node and pass it in the prompt. Use that for now and I can find another option for the future!
@Henry Moses thanks for the suggestion.
if i use gpt-4 vision api, would it still reply as the assistant?
So use the gpt-4 vision api to describe what’s in the image. Then pass the response to the AI assistant node.
@Henry Moses could you explain how to do that?
Can you join this channel
https://discord.gg/dvWJEkYU
I can walk you through it!
Recording a video for you now!