Using Vercel AI SDK With GeneralCompute: Full Integration Guide

Markdown body

The Vercel AI SDK is the most popular way to add streaming AI responses to Next.js apps. It ships `useChat` and `useCompletion` hooks that handle everything from token streaming to loading states to conversation history, and it connects to any OpenAI-compatible API.

GeneralCompute exposes an OpenAI-compatible API, which means the SDK connects to it with a two-line config change. You get GeneralCompute's inference speeds with the same hooks and components you already know.

This guide walks through:

- Setting up the Vercel AI SDK to use GeneralCompute as its provider
- Building a chat interface with `useChat`
- Building a text generation UI with `useCompletion`
- Deploying your API routes to the Vercel edge runtime for minimum latency

## How the SDK Works

The Vercel AI SDK separates the provider (where you call the model) from the hooks (how you handle the response in your UI). On the server you define a route handler that calls the model and streams back the response. On the client, `useChat` or `useCompletion` hits that route and handles the stream, exposing React state you bind to your UI.

This architecture keeps API keys on the server and makes the client code trivially simple.

## Installation

```bash
npm install ai @ai-sdk/openai
```

The `ai` package is the core SDK. `@ai-sdk/openai` is the official OpenAI provider, which works with any OpenAI-compatible endpoint -- including GeneralCompute's.

## Configuring the Provider

Create a shared provider config you can import into any route handler:

```typescript
// lib/generalcompute.ts
import { createOpenAI } from "@ai-sdk/openai";

export const generalcompute = createOpenAI({
  apiKey: process.env.GENERALCOMPUTE_API_KEY,
  baseURL: "https://api.generalcompute.com/v1",
});
```

The `createOpenAI` factory accepts `baseURL` to override where requests go. From here, use `generalcompute("model-name")` anywhere you'd normally write `openai("gpt-4o")`.

Add your API key to `.env.local`:

```
GENERALCOMPUTE_API_KEY=your_api_key_here
```

## Building a Chat Interface With `useChat`

`useChat` is the SDK's primary hook for multi-turn conversations. It manages message history, handles the streaming response, and exposes loading and error states.

### The Route Handler

First, define the server-side route that the hook will call. In the Next.js App Router, this goes in `app/api/chat/route.ts`:

```typescript
// app/api/chat/route.ts
import { streamText } from "ai";
import { generalcompute } from "@/lib/generalcompute";

export const runtime = "edge";

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: generalcompute("llama-4-maverick"),
    messages,
    system: "You are a helpful assistant. Be concise.",
  });

  return result.toDataStreamResponse();
}
```

`streamText` calls the model and returns a streaming result. `toDataStreamResponse()` formats it as the Vercel AI SDK's data stream protocol, which `useChat` on the client knows how to consume.

The `export const runtime = "edge"` line tells Vercel to deploy this function to its edge network. More on that below.

### The Chat Component

```typescript
// app/chat/page.tsx
"use client";

import { useChat } from "ai/react";

export default function ChatPage() {
  const { messages, input, handleInputChange, handleSubmit, isLoading } =
    useChat({
      api: "/api/chat",
    });

  return (
    <div className="max-w-2xl mx-auto p-4">
      <div className="space-y-4 mb-4 min-h-96">
        {messages.map((message) => (
          <div
            key={message.id}
            className={`p-3 rounded-lg ${
              message.role === "user"
                ? "bg-blue-50 ml-8"
                : "bg-gray-50 mr-8"
            }`}
          >
            <p className="text-sm font-medium text-gray-500 mb-1">
              {message.role === "user" ? "You" : "Assistant"}
            </p>
            <p className="text-gray-800 whitespace-pre-wrap">{message.content}</p>
          </div>
        ))}
        {isLoading && (
          <div className="bg-gray-50 mr-8 p-3 rounded-lg">
            <p className="text-sm text-gray-400">Thinking...</p>
          </div>
        )}
      </div>

      <form onSubmit={handleSubmit} className="flex gap-2">
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Ask something..."
          disabled={isLoading}
          className="flex-1 border border-gray-200 rounded-lg px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-blue-500"
        />
        <button
          type="submit"
          disabled={isLoading || !input.trim()}
          className="px-4 py-2 bg-blue-600 text-white text-sm rounded-lg disabled:opacity-50"
        >
          Send
        </button>
      </form>
    </div>
  );
}
```

`useChat` gives you `messages` (the full conversation array), `input` (controlled input value), `handleInputChange` and `handleSubmit` (event handlers you attach directly to the form), and `isLoading` (true while a response is streaming).

That's the complete chat UI. The hook handles everything else: appending your message to the history, sending the request, accumulating streamed tokens into the message object, and updating React state on each token.

## Building a Text Generation UI With `useCompletion`

`useCompletion` is better suited for single-shot generation tasks: summarization, text transformation, content generation from a prompt. Unlike `useChat`, it doesn't maintain a conversation history.

### The Route Handler

```typescript
// app/api/completion/route.ts
import { streamText } from "ai";
import { generalcompute } from "@/lib/generalcompute";

export const runtime = "edge";

export async function POST(req: Request) {
  const { prompt } = await req.json();

  const result = streamText({
    model: generalcompute("llama-4-scout"),
    prompt,
    maxTokens: 1024,
  });

  return result.toDataStreamResponse();
}
```

### The Completion Component

```typescript
// app/complete/page.tsx
"use client";

import { useCompletion } from "ai/react";

export default function CompletionPage() {
  const { completion, input, handleInputChange, handleSubmit, isLoading } =
    useCompletion({
      api: "/api/completion",
    });

  return (
    <div className="max-w-2xl mx-auto p-4 space-y-4">
      <form onSubmit={handleSubmit} className="space-y-3">
        <textarea
          value={input}
          onChange={handleInputChange}
          placeholder="Enter your prompt..."
          rows={4}
          className="w-full border border-gray-200 rounded-lg px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-blue-500 resize-none"
        />
        <button
          type="submit"
          disabled={isLoading || !input.trim()}
          className="px-4 py-2 bg-blue-600 text-white text-sm rounded-lg disabled:opacity-50"
        >
          {isLoading ? "Generating..." : "Generate"}
        </button>
      </form>

      {completion && (
        <div className="border border-gray-100 rounded-lg p-4 bg-gray-50">
          <p className="text-sm font-medium text-gray-500 mb-2">Output</p>
          <p className="text-gray-800 whitespace-pre-wrap">{completion}</p>
        </div>
      )}
    </div>
  );
}
```

The `completion` string updates token-by-token as the response streams in. You can bind it directly to any UI element.

## Customizing the Hook Behavior

Both hooks accept an `options` object with several useful settings:

```typescript
const { messages, input, handleInputChange, handleSubmit } = useChat({
  api: "/api/chat",

  // Initial messages (useful for pre-seeding a conversation)
  initialMessages: [
    {
      id: "welcome",
      role: "assistant",
      content: "Hi, how can I help you today?",
    },
  ],

  // Run a callback when the stream finishes
  onFinish: (message) => {
    console.log("Final message:", message.content);
    // Save to database, update analytics, etc.
  },

  // Handle errors from the route handler
  onError: (error) => {
    console.error("Chat error:", error);
  },
});
```

If you need to send additional data with each request (a user ID, a selected document, a session token), pass it through `body`:

```typescript
const { messages, handleSubmit } = useChat({
  api: "/api/chat",
  body: {
    userId: session.user.id,
    documentId: selectedDoc,
  },
});
```

The route handler receives this in the request body alongside `messages`.

## Passing Extra Context From the Route Handler

Sometimes you want to send structured data back to the client alongside the stream -- for example, token usage stats or tool call results. The SDK supports this through `data`:

```typescript
// app/api/chat/route.ts
import { streamText, createDataStreamResponse } from "ai";
import { generalcompute } from "@/lib/generalcompute";

export const runtime = "edge";

export async function POST(req: Request) {
  const { messages } = await req.json();

  return createDataStreamResponse({
    execute: async (dataStream) => {
      dataStream.writeData({ startTime: Date.now() });

      const result = streamText({
        model: generalcompute("llama-4-maverick"),
        messages,
        onFinish: ({ usage }) => {
          dataStream.writeData({
            inputTokens: usage.promptTokens,
            outputTokens: usage.completionTokens,
          });
        },
      });

      result.mergeIntoDataStream(dataStream);
    },
  });
}
```

On the client, read it with the `data` field from `useChat`:

```typescript
const { messages, data } = useChat({ api: "/api/chat" });

// data is an array of objects you wrote with dataStream.writeData()
const latestUsage = data?.[data.length - 1];
```

This is useful for displaying token counts, cost estimates, or any metadata that doesn't belong in the message content itself.

## Edge Runtime Deployment

Adding `export const runtime = "edge"` to your route handlers deploys them to Vercel's edge network rather than serverless functions. The practical benefit: edge functions run in the region closest to your user, which cuts the round-trip time before the first token arrives.

For a streaming chat app, TTFT (time to first token) is the most noticeable latency. A user in Frankfurt connecting to a US-east serverless function adds ~100ms of network overhead before the model starts generating. Edge deployment removes most of that.

One constraint: edge functions run in a limited runtime that excludes Node.js-specific APIs. The Vercel AI SDK is designed for this -- `streamText` and `toDataStreamResponse` work fine in the edge runtime. If your route handler uses Node.js APIs (file system, native modules), you'll need to keep it in the default serverless runtime.

To verify your routes work on edge before deploying, test with:

```bash
npx vercel dev
```

Vercel's local dev server emulates the edge runtime and will surface compatibility issues before you push.

## Choosing a Model

GeneralCompute supports several models at different speed and quality points:

| Model | Context | Best for |
|---|---|---|
| `llama-4-maverick` | 128K | General chat, complex reasoning |
| `llama-4-scout` | 128K | Faster responses, lighter tasks, lower cost |
| `qwen3-coder` | 128K | Code generation, technical Q&A |

For a conversational UI where response quality matters, `llama-4-maverick` is a good default. For autocomplete or one-shot generation where speed matters more, `llama-4-scout` gives noticeably faster TTFT.

Swap the model name in your route handler -- no client code changes needed.

## Complete Project Structure

For reference, here's the full file tree for a project using both hooks:

```
my-ai-app/
  app/
    api/
      chat/
        route.ts        # useChat backend
      completion/
        route.ts        # useCompletion backend
    chat/
      page.tsx          # Chat UI
    complete/
      page.tsx          # Completion UI
  lib/
    generalcompute.ts   # Provider config
  .env.local            # GENERALCOMPUTE_API_KEY
  package.json
```

## Getting Started

If you don't have a GeneralCompute API key yet, get one at [generalcompute.com](https://generalcompute.com). The same key works across all SDK integrations -- the `createOpenAI` factory with `baseURL` pointed at GeneralCompute is all you need to route any Vercel AI SDK app through GeneralCompute's infrastructure.

The [Vercel AI SDK docs](https://sdk.vercel.ai) cover the full API surface if you need to go deeper: tool calling, attachments, multi-step agent flows, and React Server Components support are all available once the basic integration is working.