Vercel AI SDK Integration

This guide demonstrates a starter AI-powered browser automation application using Browserless, the Vercel AI SDK, and Next.js. The example shows how an agent can control a browser through natural language instructions for tasks like web scraping, form filling, and content extraction.

Introduction

This integration combines three powerful technologies:

• Browserless: A headless browser service that provides browser automation capabilities
• Vercel AI SDK: A toolkit for building AI-powered applications
• Next.js: A React framework for building web applications

Together, these technologies enable you to create applications where AI can understand and execute browser automation tasks through natural language.

Prerequisites

• Node.js 18 or higher
• Vercel account
• Browserless API token
• OpenAI API key (or other supported LLM provider)

Step 1: Project Setup

Create a Next.js Project

npx create-next-app@latest browserless-ai
cd browserless-ai

Install Required Packages

npm install @vercel/ai @browserless/ai puppeteer-core openai zod prettier

Configure Environment Variables

Create a .env.local file in your project root:

BROWSERLESS_API_KEY=your_browserless_api_key
OPENAI_API_KEY=your_openai_api_key

Step 2: Core Components

Browser Service

Create src/lib/browser.ts:

import puppeteer from 'puppeteer-core';
import { z } from 'zod';

// Schema for browser settings
const BrowserSettingsSchema = z.object({
  viewport: z.object({
    width: z.number().default(1920),
    height: z.number().default(1080),
  }),
  userAgent: z.string().optional(),
  timeout: z.number().default(30000),
});

export type BrowserSettings = z.infer<typeof BrowserSettingsSchema>;

export class BrowserlessService {
  private static instance: BrowserlessService;
  private browser: puppeteer.Browser | null = null;
  private settings: BrowserSettings;

  private constructor(settings: Partial<BrowserSettings> = {}) {
    this.settings = BrowserSettingsSchema.parse(settings);
  }

  static getInstance(settings?: Partial<BrowserSettings>): BrowserlessService {
    if (!BrowserlessService.instance) {
      BrowserlessService.instance = new BrowserlessService(settings);
    }
    return BrowserlessService.instance;
  }

  async getBrowser(): Promise<puppeteer.Browser> {
    if (!this.browser) {
      this.browser = await puppeteer.connect({
        browserWSEndpoint: `wss://production-sfo.browserless.io?token=${process.env.BROWSERLESS_API_KEY}`,
        defaultViewport: this.settings.viewport,
      });
    }
    return this.browser;
  }

  async close() {
    if (this.browser) {
      await this.browser.close();
      this.browser = null;
    }
  }
}

AI Route Handler

Create src/app/api/chat/route.ts:

import { OpenAIStream, StreamingTextResponse } from 'ai';
import { BrowserlessService } from '@/lib/browser';
import OpenAI from 'openai';
import { z } from 'zod';

// Schema for chat messages
const MessageSchema = z.object({
  role: z.enum(['user', 'assistant', 'system']),
  content: z.string(),
});

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

export async function POST(req: Request) {
  try {
    const { messages } = await req.json();
    
    // Validate messages
    const validatedMessages = z.array(MessageSchema).parse(messages);
    
    const browserService = BrowserlessService.getInstance();
    const browser = await browserService.getBrowser();
    const page = await browser.newPage();

    // Process the message and perform browser actions
    const response = await openai.chat.completions.create({
      model: process.env.OPENAI_MODEL ?? 'gpt-4.1-mini',
      stream: true,
      messages: [
        {
          role: 'system',
          content: `You are a browser automation assistant. You can:
          1. Navigate to URLs
          2. Extract information from pages
          3. Fill out forms
          4. Take screenshots
          5. Click elements
          6. Type text
          Always respond with clear instructions for the browser.`
        },
        ...validatedMessages,
      ],
    });

    const stream = OpenAIStream(response, {
      async onCompletion(completion) {
        try {
          // Process the AI's response and perform browser actions
          if (completion.includes('navigate to')) {
            const url = completion.match(/navigate to (https?:\/\/[^\s]+)/)?.[1];
            if (url) {
              await page.goto(url, { waitUntil: 'networkidle0' });
            }
          }
          
          if (completion.includes('click')) {
            const selector = completion.match(/click "([^"]+)"/)?.[1];
            if (selector) {
              await page.click(selector);
            }
          }
          
          if (completion.includes('type')) {
            const [selector, text] = completion.match(/type "([^"]+)" into "([^"]+)"/)?.slice(1) || [];
            if (selector && text) {
              await page.type(selector, text);
            }
          }
          
          // Add more action handlers as needed
        } catch (error) {
          console.error('Error executing browser action:', error);
        }
      },
    });

    return new StreamingTextResponse(stream);
  } catch (error) {
    console.error('Error processing request:', error);
    return new Response(
      JSON.stringify({ error: 'Failed to process request' }),
      { status: 500 }
    );
  }
}

Chat Interface

Create src/app/page.tsx:

'use client';

import { useChat } from 'ai/react';
import { useState } from 'react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat();
  const [isProcessing, setIsProcessing] = useState(false);

  return (
    <div className="flex flex-col w-full max-w-2xl mx-auto p-4">
      <div className="flex-1 overflow-y-auto mb-4">
        {messages.map((message) => (
          <div
            key={message.id}
            className={`p-4 mb-4 rounded-lg ${
              message.role === 'user'
                ? 'bg-blue-100 ml-auto'
                : 'bg-gray-100 mr-auto'
            }`}
          >
            <div className="whitespace-pre-wrap">{message.content}</div>
          </div>
        ))}
      </div>

      <form onSubmit={handleSubmit} className="flex gap-2">
        <input
          className="flex-1 p-2 border border-gray-300 rounded-lg"
          value={input}
          placeholder="Ask the AI to perform browser actions..."
          onChange={handleInputChange}
          disabled={isLoading || isProcessing}
        />
        <button
          type="submit"
          className="px-4 py-2 bg-blue-500 text-white rounded-lg disabled:opacity-50"
          disabled={isLoading || isProcessing}
        >
          Send
        </button>
      </form>
    </div>
  );
}

Step 3: Advanced Features

Session Management

Create src/lib/session.ts:

import { BrowserlessService } from './browser';
import puppeteer from 'puppeteer-core';

export class BrowserSession {
  private static sessions: Map<string, {
    page: puppeteer.Page;
    lastActive: number;
  }> = new Map();
  private static readonly SESSION_TIMEOUT = 30 * 60 * 1000; // 30 minutes

  static async getSession(sessionId: string): Promise<puppeteer.Page> {
    const session = this.sessions.get(sessionId);
    
    if (session && Date.now() - session.lastActive < this.SESSION_TIMEOUT) {
      session.lastActive = Date.now();
      return session.page;
    }

    const browser = await BrowserlessService.getInstance().getBrowser();
    const page = await browser.newPage();
    
    this.sessions.set(sessionId, {
      page,
      lastActive: Date.now(),
    });

    return page;
  }

  static async cleanup() {
    const now = Date.now();
    for (const [sessionId, session] of this.sessions.entries()) {
      if (now - session.lastActive > this.SESSION_TIMEOUT) {
        await session.page.close();
        this.sessions.delete(sessionId);
      }
    }
  }
}

Error Handling Middleware

Create src/middleware.ts:

import { NextResponse } from 'next/server';
import { z } from 'zod';

export async function middleware(request: Request) {
  try {
    // Add rate limiting
    const ip = request.headers.get('x-forwarded-for') || 'unknown';
    // Implement your rate limiting logic here

    // Validate request body
    if (request.method === 'POST') {
      const body = await request.json();
      // Add your validation logic here
    }

    return NextResponse.next();
  } catch (error) {
    console.error('Middleware error:', error);
    return new NextResponse(
      JSON.stringify({ error: 'Internal server error' }),
      { status: 500 }
    );
  }
}

Step 4: Testing

Local Testing

1. Start the development server:

npm run dev

2. Test the API endpoints:

curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Navigate to example.com"}]}'

Production Testing

1. Deploy to Vercel:

vercel

2. Test the deployed endpoints:

curl -X POST https://your-app.vercel.app/api/chat \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Navigate to example.com"}]}'

Step 5: Deployment

Vercel Deployment

1. Push your code to GitHub
2. Import the project in Vercel
3. Add your environment variables:
   • BROWSERLESS_API_KEY
   • OPENAI_API_KEY
4. Deploy!

Environment Configuration

Configure your Vercel project settings:

{
  "buildCommand": "next build",
  "outputDirectory": ".next",
  "framework": "nextjs",
  "installCommand": "npm install",
  "regions": ["sfo1"]
}

Best Practices

1. Resource Management

   • Always close browser pages when done
   • Implement session timeouts
   • Clean up unused resources

2. Error Handling

   • Implement comprehensive error handling
   • Log errors appropriately
   • Provide meaningful error messages

3. Security

   • Validate all inputs
   • Implement rate limiting
   • Use environment variables for sensitive data

4. Performance

   • Use appropriate timeouts
   • Implement caching where possible
   • Optimize browser operations

5. Monitoring

   • Set up error tracking
   • Monitor API usage
   • Track performance metrics

Troubleshooting

Common Issues

1. Connection Errors

   • Check your Browserless API token
   • Verify network connectivity
   • Check firewall settings

2. Timeout Errors

   • Increase timeout values
   • Optimize browser operations
   • Implement retry logic

3. Rate Limit Errors

   • Implement proper rate limiting
   • Monitor API usage
   • Consider upgrading your plan

Debugging Tips

1. Enable verbose logging:

process.env.DEBUG = 'browserless:*';

2. Use the Browserless dashboard to monitor sessions

3. Implement request logging:

console.log('Request:', {
  url: request.url,
  method: request.method,
  headers: Object.fromEntries(request.headers),
});

Additional Resources

• Browserless Documentation
• Vercel AI SDK Documentation
• Next.js Documentation
• OpenAI API Documentation