Next.js

Integrate xseek with Next.js using our middleware solution. Track and optimize your Next.js website for AI search engines and improve visibility.

Overview

Our Next.js middleware integration allows you to add AI SEO tracking capabilities to your Next.js applications with minimal configuration. By leveraging the built-in middleware capabilities of Next.js, you can track AI interactions across your entire site without modifying individual pages. This solution works with both app router and pages router applications and provides complete coverage of your Next.js site.

Features

Seamless integration with Next.js middleware
Automatic tracking of all routes
Real-time AI visibility analytics
Easy installation with npm/yarn/pnpm
Detailed performance insights for Next.js applications
Compatible with app router and pages router
Zero impact on site performance
Tracks all major AI systems including ChatGPT, Claude, and Perplexity

Requirements

Next.js 12.0.0 or higher
API key from xseek
Website ID from your xseek dashboard

Setup Process

1
Create an AI detection module in your Next.js project
2
Add the detection function to your middleware.ts file
3
Deploy your Next.js application to start tracking AI visits

Integration Setup

Environment Variables Required

Add these environment variables to your project:

XSEEK_API_KEY=your_api_keyXSEEK_WEBSITE_ID=your_website_id

Your API key can be found in your account settings. Make sure it has the ai_visits:push privilege.

1Add detection code to your Next.js project

Create a new file (e.g., lib/ai-detection.js) with the following code:

// AI crawler detection - patterns auto-generated from xSeek's bot database
// Add to .env: XSEEK_API_KEY=your_api_key, XSEEK_WEBSITE_ID=your_website_id

const AI_BOTS = [
  { name: 'anthropic-ai', pattern: /anthropic-ai/i },
  { name: 'claudebot', pattern: /ClaudeBot/i },
  { name: 'claude-web', pattern: /claude-web/i },
  { name: 'claude-user', pattern: /Claude-User/i },
  { name: 'claude-searchbot', pattern: /Claude-SearchBot/i },
  { name: 'perplexitybot', pattern: /PerplexityBot/i },
  { name: 'perplexity-user', pattern: /Perplexity-User/i },
  { name: 'grokbot', pattern: /GrokBot(?!.*DeepSearch)/i },
  { name: 'grok-search', pattern: /xAI-Grok/i },
  { name: 'grok-deepsearch', pattern: /Grok-DeepSearch/i },
  { name: 'GPTBot', pattern: /GPTBot/i },
  { name: 'chatgpt-user', pattern: /ChatGPT-User/i },
  { name: 'oai-searchbot', pattern: /OAI-SearchBot/i },
  { name: 'google-extended', pattern: /Google-Extended/i },
  { name: 'applebot', pattern: /Applebot(?!-Extended)/i },
  { name: 'applebot-extended', pattern: /Applebot-Extended/i },
  { name: 'meta-external', pattern: /meta-externalagent/i },
  { name: 'meta-externalfetcher', pattern: /meta-externalfetcher/i },
  { name: 'bingbot', pattern: /Bingbot(?!.*AI)/i },
  { name: 'bingpreview', pattern: /bingbot.*Chrome/i },
  { name: 'microsoftpreview', pattern: /MicrosoftPreview/i },
  { name: 'cohere-ai', pattern: /cohere-ai/i },
  { name: 'cohere-training-data-crawler', pattern: /cohere-training-data-crawler/i },
  { name: 'youbot', pattern: /YouBot/i },
  { name: 'duckassistbot', pattern: /DuckAssistBot/i },
  { name: 'semanticscholarbot', pattern: /SemanticScholarBot/i },
  { name: 'ccbot', pattern: /CCBot/i },
  { name: 'ai2bot', pattern: /AI2Bot/i },
  { name: 'ai2bot-dolma', pattern: /AI2Bot-Dolma/i },
  { name: 'aihitbot', pattern: /aiHitBot/i },
  { name: 'amazonbot', pattern: /Amazonbot/i },
  { name: 'novaact', pattern: /NovaAct/i },
  { name: 'brightbot', pattern: /Brightbot/i },
  { name: 'bytespider', pattern: /Bytespider/i },
  { name: 'tiktokspider', pattern: /TikTokSpider/i },
  { name: 'cotoyogi', pattern: /Cotoyogi/i },
  { name: 'crawlspace', pattern: /Crawlspace/i },
  { name: 'pangubot', pattern: /PanguBot/i },
  { name: 'petalbot', pattern: /PetalBot/i },
  { name: 'semrushbot-ocob', pattern: /SemrushBot-OCOB/i },
  { name: 'semrushbot-swa', pattern: /SemrushBot-SWA/i },
  { name: 'sidetrade-indexer', pattern: /Sidetrade indexer bot/i },
  { name: 'timpibot', pattern: /Timpibot/i },
  { name: 'velenpublicwebcrawler', pattern: /VelenPublicWebCrawler/i },
  { name: 'omgili', pattern: /omgili/i },
  { name: 'omgilibot', pattern: /omgilibot/i },
  { name: 'webzio-extended', pattern: /Webzio-Extended/i },
  { name: 'baiduspider', pattern: /Baiduspider/i }
];

export async function detectAndLogAIBot(req) {
  const userAgent = req.headers.get('user-agent') || '';
  
  let detectedBot = null;
  for (const bot of AI_BOTS) {
    if (bot.pattern.test(userAgent)) {
      detectedBot = bot.name;
      break;
    }
  }
  
  if (detectedBot) {
    const apiKey = process.env.XSEEK_API_KEY;
    const websiteId = process.env.XSEEK_WEBSITE_ID;
    
    if (!apiKey || !websiteId) {
      console.warn('XSEEK_API_KEY or XSEEK_WEBSITE_ID not configured');
      return;
    }
    
    const ip = req.headers.get('x-forwarded-for')?.split(',')[0]?.trim() || '';
    
    await fetch('https://www.xseek.io/api/track-ai-bot', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json', 'x-api-key': apiKey },
      body: JSON.stringify({
        botName: detectedBot,
        userAgent,
        url: req.nextUrl.toString(),
        ip: ip || undefined,
        referer: req.headers.get('referer') || undefined,
        websiteId,
      }),
    });
    
    console.log(`AI crawler detected: ${detectedBot}`);
  }
}

2Call the function in your middleware

Add this code to your middleware.ts file:

// Add this to your middleware.ts
import { NextResponse } from 'next/server';
import { detectAndLogAIBot } from './lib/ai-detection';

export async function middleware(req) {
    await detectAndLogAIBot(req);
    return NextResponse.next();
}

export const config = {
  matcher: ['/((?!api|_next/static|_next/image|favicon.ico).*)']
};

That's it!

Your Next.js app is now set up to track AI crawler visits. Come back to the xseek dashboard to see your data.

Need more help?

Check our comprehensive documentation for detailed instructions.

View documentation