Ethan Tan Comments - BuildShip + Rowy

Ethan Tan

•Created by Ethan Tan on 5/19/2024 in #❓・buildship-help

OpenAI Stream Response is empty

Thanks @nithinrdy - I see the log but for example in this one: - The request is being received correctly (called from Vapi.ai) - I cant tell what if anything is being output from the Stream Response, or the Return node. It looks like nothing? Vapi says they receive nothing from their side How else can we diagnose this? Is there something you can see from your side?

19 replies

BRBuildShip + Rowy

•Created by Ethan Tan on 5/19/2024 in #❓・buildship-help

OpenAI Stream Response is empty

@nithinrdy following up on this

19 replies

BRBuildShip + Rowy

•Created by Ethan Tan on 5/19/2024 in #❓・buildship-help

OpenAI Stream Response is empty

@nithinrdy Is there any way you can see the logs of a particular workflow run?

19 replies

BRBuildShip + Rowy

•Created by Ethan Tan on 5/19/2024 in #❓・buildship-help

OpenAI Stream Response is empty

It may be fine - I'm just attempting to find the problem. The workflow is called correctly now from Vapi, but on the Vapi side they said they don't receive any output from Buildship

19 replies

BRBuildShip + Rowy

•Created by Ethan Tan on 5/19/2024 in #❓・buildship-help

OpenAI Stream Response is empty

Thank you - oh and how should the REST API node be set up please, in terms of the Body/Header etc?

19 replies

BRBuildShip + Rowy

•Created by Ethan Tan on 5/19/2024 in #❓・buildship-help

OpenAI Stream Response is empty

Oh I see - so looking at the code and setup above, does it look like it should be working correctly?

19 replies

BRBuildShip + Rowy

•Created by Ethan Tan on 5/19/2024 in #❓・buildship-help

OpenAI Stream Response is empty

export default async (
  { userRequest, systemPrompt, openaiSecret, model, temperature },
  { logging, req: ctx = {}},
) => {
  // Create a new Readable stream for writing the response
  let writeStream = (ctx.body = new Readable());
  writeStream._read = function () {}; // Make the writeStream readable
  writeStream.pipe(ctx.res); // Pipe the writeStream to the response object to send data to the client
  ctx.type = "text/undefined-content";

  // Set response headers
  Object.assign(ctx.response.headers, {
    "Transfer-Encoding": "chunked",
    Connection: "keep-alive",
  });

  // Initialize OpenAI API client
  const openai = new OpenAI({
    apiKey: openaiSecret,
  });

  try {
    // Make an asynchronous call to OpenAI API to get stream response
    const completion = await openai.beta.chat.completions.stream(
      {
        model,
        temperature,
        messages: [
          {
            role: "system",
            content: systemPrompt,
          },
          {
            role: "user",
            content: userRequest,
          },
        ],
        stream: true,
      },
      {
        responseType: "stream",
      },
    );

    // Stream data from completion to the writeStream
    const response = await streamer(writeStream, completion);
    return response;
  } catch (error) {
    // Handle errors if any
    logging.log(error);
    return "";
  }
};

export default async (
  { userRequest, systemPrompt, openaiSecret, model, temperature },
  { logging, req: ctx = {}},
) => {
  // Create a new Readable stream for writing the response
  let writeStream = (ctx.body = new Readable());
  writeStream._read = function () {}; // Make the writeStream readable
  writeStream.pipe(ctx.res); // Pipe the writeStream to the response object to send data to the client
  ctx.type = "text/undefined-content";

  // Set response headers
  Object.assign(ctx.response.headers, {
    "Transfer-Encoding": "chunked",
    Connection: "keep-alive",
  });

  // Initialize OpenAI API client
  const openai = new OpenAI({
    apiKey: openaiSecret,
  });

  try {
    // Make an asynchronous call to OpenAI API to get stream response
    const completion = await openai.beta.chat.completions.stream(
      {
        model,
        temperature,
        messages: [
          {
            role: "system",
            content: systemPrompt,
          },
          {
            role: "user",
            content: userRequest,
          },
        ],
        stream: true,
      },
      {
        responseType: "stream",
      },
    );

    // Stream data from completion to the writeStream
    const response = await streamer(writeStream, completion);
    return response;
  } catch (error) {
    // Handle errors if any
    logging.log(error);
    return "";
  }
};

19 replies

BRBuildShip + Rowy

•Created by Ethan Tan on 5/19/2024 in #❓・buildship-help

OpenAI Stream Response is empty

This is the code:

import { Readable } from "stream";
import OpenAI from "openai";

// Function to stream data from readStream to writeStream
function streamer(writeStream, readStream) {
  return new Promise(function (resolve, reject) {
    let response = "";
    
    readStream.on("content", (chunk) => {
      // Extracting the payload and removing "data: " from each chunk
      const payloads = chunk.toString();

      for (const payload of payloads) {
        const cleaned = payload.replace("data: ", "");
        // Push cleaned payload to the writeStream
        writeStream.push(cleaned);
        response += cleaned;
      }
    });

    readStream.on("end", () => {
      // Resolve the promise when the readStream ends
      resolve(response);
    });

    readStream.on("error", () => {
      // Handle errors by pushing an error message to the writeStream and resolving the promise
      writeStream.push("err...");
      resolve("");
    });
  });
}

import { Readable } from "stream";
import OpenAI from "openai";

// Function to stream data from readStream to writeStream
function streamer(writeStream, readStream) {
  return new Promise(function (resolve, reject) {
    let response = "";
    
    readStream.on("content", (chunk) => {
      // Extracting the payload and removing "data: " from each chunk
      const payloads = chunk.toString();

      for (const payload of payloads) {
        const cleaned = payload.replace("data: ", "");
        // Push cleaned payload to the writeStream
        writeStream.push(cleaned);
        response += cleaned;
      }
    });

    readStream.on("end", () => {
      // Resolve the promise when the readStream ends
      resolve(response);
    });

    readStream.on("error", () => {
      // Handle errors by pushing an error message to the writeStream and resolving the promise
      writeStream.push("err...");
      resolve("");
    });
  });
}

19 replies

BRBuildShip + Rowy

•Created by Ethan Tan on 5/19/2024 in #❓・buildship-help

OpenAI Stream Response is empty

Thanks @nithinrdy - I understand the node cant be tested individually When I test the workflow though it seems nothing is output from the stream?

19 replies