Starting with 2.1.13 streamed responses not rendered anymore - works fine with 2.1.12

[andnig]

When using the exact sample application as provided in the docs, chat messages are rendered as expected with version 2.1.12. However changing the package to 2.1.13 does not render the messages anymore.
The requests work fine and responses are coming back to application (seen in chrome devtools), but the messages are not rendered anymore.

'use client'

import { useChat } from 'ai/react'

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat()

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role}: {m.content}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <label>
          Say something...
          <input
            value={input}
            onChange={handleInputChange}
          />
        </label>
      </form>
    </div>
  )
}

import { OpenAIStream, StreamingTextResponse } from 'ai'
import { Configuration, OpenAIApi } from 'openai-edge'

// Create an OpenAI API client (that's edge friendly!)
const config = new Configuration({
  apiKey: process.env.OPENAI_API_KEY
})
const openai = new OpenAIApi(config)
 
// IMPORTANT! Set the runtime to edge
export const runtime = 'edge'
 
export async function POST(req: Request) {
  // Extract the `messages` from the body of the request
  const { messages } = await req.json()
 
  // Ask OpenAI for a streaming chat completion given the prompt
  const response = await openai.createChatCompletion({
    model: 'gpt-3.5-turbo',
    stream: true,
    messages
  })
  // Convert the response into a friendly text-stream
  const stream = OpenAIStream(response)
  // Respond with the stream
  return new StreamingTextResponse(stream)
}

[e-roy]

Here to confirm the same with my results. Here's my code using langchain

import { LangChainStream, Message, StreamingTextResponse } from "ai";
import { CallbackManager } from "langchain/callbacks";
import { ChatOpenAI } from "langchain/chat_models/openai";
import {
  AIChatMessage,
  HumanChatMessage,
  SystemChatMessage,
} from "langchain/schema";

export const runtime = "edge";

export async function POST(req: Request) {
  const { messages, model, temperature } = await req.json();

  console.log("messages: ", messages);

  const { stream, handlers } = LangChainStream();

  const llm = new ChatOpenAI({
    modelName: "gpt-3.5-turbo",
    streaming: true,
    callbackManager: CallbackManager.fromHandlers(handlers),
  });

  const chatMessages = (messages as Message[])
    .map((m) => {
      if (m.role === "user") {
        return new HumanChatMessage(m.content);
      }
      if (m.role === "assistant") {
        return new AIChatMessage(m.content);
      }
      if (m.role === "system") {
        return new SystemChatMessage(m.content);
      }
      return undefined; // Default case
    })
    .filter(
      (message): message is Exclude<typeof message, undefined> =>
        message !== undefined
    );

  llm.call(chatMessages).catch(console.error);

  return new StreamingTextResponse(stream);
}

What I'm seeing is the message array in the frontend stays empty
useEffect(() => { console.log("messages", messages); }, [messages]);

This will result in a repeat stream of messages []

I am also seeing a response in devtools

[holdenmatt]

Yep, also seeing this (seems to affect both the chat and completion APIs).

The examples in the docs here don't work with 2.1.13:
https://sdk.vercel.ai/docs/getting-started
https://sdk.vercel.ai/docs/guides/openai

Works after downgrading to 2.1.12.

[jrysana]

Saw the same issue as well

[jrysana]

Seems like this commit is probably the issue, since it's the only commit with actual source code changed between 2.1.12 (works fine) and 2.1.13 (broken hook) fd82961

[cuevaio]

Same issue. The returned value completion of useCompletion is an empty string:

{ completion: "" }

[mavic111]

Same issue with huggingface

[kerkkoh]

I thought I had gone insane when all of the sudden yesterday everything just didn't work. Glad it wasn't just me.

Here's a quick fix/revert that fixes it (React/Next.js, but same idea for other frameworks, just provide an id, you can check the old id generation from this commit fd82961):

import { useId } from 'react';
import { UseChatOptions, useChat } from 'ai/react'

const useChatWrapper = (options?: UseChatOptions) => {
  const id = useId();
  const chat = useChat({...(options ?? {}), id})
  return chat;
};

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChatWrapper()
  ....
}

[willscottrod]

I thought I had gone insane when all of the sudden yesterday everything just didn't work. Glad it wasn't just me.

Here's a quick fix/revert that fixes it (React/Next.js, but same idea for other frameworks, just provide an id, you can check the old id generation from this commit fd82961):

import { useId } from 'react';
import { UseChatOptions, useChat } from 'ai/react'

const useChatWrapper = (options?: UseChatOptions) => {
  const id = useId();
  const chat = useChat({...(options ?? {}), id})
  return chat;
};

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChatWrapper()
  ....
}

This is amazing, this fixed it. I've been trying to fix this for the last 2 hours and was almost losing my mind.

[anhhtca]

quoted...

Amazing ! It's worked.

[kerkkoh]

For anyone wanting to actually fix this with a PR and still use nanoid-based ids, change it so that the id-generating function is not called on each render as it currently clearly is. Likely this was missed during testing as I assume they have an id that they provide to useChat in the tests.

My fix above is just one way of ensuring there's an id that does not change. You can also just use useChat({ id: "hello world" }) and it will work just the same, as in:

import { useChat } from 'ai/react'

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({ id: "hello world" })
  ....
}

[jaredpalmer]

This has been resolved. #264 . Sorry about that!

[kerkkoh]

@jaredpalmer I'm not sure if there was an actual reason for using nanoid based ids, but if there was, at least in react, just placing the id value in the persistent state to avoid running it on each render should suffice, i.e.

import {useState} from 'react'
...
const [chatId,] = useState(id || `chat-${nanoid()}`)

Again, not sure whether there was a reason for the change, and I am not an expert in svelte nor vue, but I assume they have similar mechanisms to avoid such issues.