What is ByteDance Doubao's main user agent string?

ByteDance Doubao uses Mozilla/5.0 (compatible; Doubaobot/1.0; +https://www.doubao.com/bot) as its primary user agent. It also uses Bytespider and TikTokSpider.

What makes Doubao unique among Chinese LLMs?

Doubao is integrated with TikTok, Douyin, and ByteDance's content ecosystem. It excels at content generation, video understanding, and recommendations powered by ByteDance's algorithms.

How do I block Doubao from crawling my website?

Add User-agent: Doubaobot, User-agent: Bytespider, and User-agent: TikTokSpider followed by Disallow: / to your robots.txt file to block ByteDance's crawlers.

xSeek/Docs/User agents/Doubao

⌘K

Back to documentation

Doubao user agents, explained.

ByteDance ships multiple crawlers — Doubaobot for AI, Bytespider for general content, TikTokSpider for video. Here's how each one fits into the Doubao stack.

Doubao · ByteDanceUpdated Apr 2026~7 min read3 user agents

TL;DR

ByteDance uses Doubaobot for Doubao-specific AI crawling, Bytespider as its general-purpose crawler across Toutiao and other services, and TikTokSpider for content on TikTok/Douyin. All three feed Doubao.

Overview

Doubao (豆包) is ByteDance's AI assistant, launched in 2023 and now serving 35M+ monthly active users. It specializes in content generation and recommendations — powered by ByteDance's world-class recommendation algorithms.

Doubao is deeply integrated with TikTok (international) and Douyin (China), giving it access to one of the largest short-form content ecosystems on earth.

How to identify the agents

3 crawlers

Doubao identifies itself with specific user-agent strings. Verify each through a published IP range.

Doubaobot

Doubao AI · primary crawler

TrainingAnswers

Mozilla/5.0 (compatible; Doubaobot/1.0; +https://www.doubao.com/bot)

ByteDance's AI-specific crawler for Doubao. Feeds the assistant's knowledge and generation systems.

Verifiabledoubao.com/bot

Bytespider

ByteDance · general crawler

TrainingContent

Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; +https://zhanzhang.toutiao.com/)

ByteDance's general-purpose web crawler. Powers Toutiao, Xigua Video, and feeds content into Doubao's ecosystem. Often the heaviest ByteDance crawler by volume.

Verifiablezhanzhang.toutiao.com

TikTokSpider

TikTok / Douyin · content crawler

Video ecosystem

Mozilla/5.0 (compatible; TikTokSpider/1.0; +https://www.tiktok.com)

Crawls external content linked from TikTok and Douyin. Feeds ByteDance's recommendation engine and indirectly Doubao's content-generation capabilities.

Verifiabletiktok.com

How Doubao accesses your content

3 access patterns, mapped to the agents above.

AI answers

Doubaobot feeds Doubao's generated responses and creative tools.

Content ecosystem

Bytespider powers Toutiao (news) and indexes general web content.

Video surfaces

TikTokSpider crawls content linked from TikTok and Douyin posts.

Heads up. Blocking Bytespider removes you from Toutiao and other ByteDance news surfaces — significant reach in China. Consider blocking only the specific Doubao UA if that's your goal.

Control Doubao's access

robots.txt

You control each agent independently. Common configurations below.

Allow ByteDance crawlersRecommended

Standard setup for visibility across Doubao and TikTok/Douyin.

# Allow ByteDance ecosystem User-agent: Doubaobot Allow: / User-agent: Bytespider Allow: / User-agent: TikTokSpider Allow: /

Block all ByteDance crawlersRestrictive

Removes you from Doubao, Toutiao, and ByteDance recommendations.

# Block every ByteDance crawler User-agent: Doubaobot Disallow: / User-agent: Bytespider Disallow: / User-agent: TikTokSpider Disallow: /

Public pages only

Expose public content, protect private areas.

# Public-only access for ByteDance User-agent: Bytespider User-agent: Doubaobot Disallow: /admin/ Disallow: /private/ Disallow: /api/ Allow: /blog/ Allow: /articles/ Allow: /videos/

Optimize content for Doubao.

Doubao thrives on content optimized for short-form video and social recommendation. Five rules that earn citations.

Optimize for short-form. Short, punchy paragraphs, clear hooks in the first line — Doubao models social content patterns.

Include video and transcripts. ByteDance is video-first. Embed videos with transcripts, captions, and descriptive filenames.

Mobile-first. Nearly all TikTok/Douyin traffic is mobile — desktop-only layouts get ranked down.

Tap trends. Doubao's recommendation engine amplifies topical content. Current phrasing wins over evergreen generic.

Use schema + social links. VideoObject schema and visible TikTok/Douyin profiles reinforce authority signals.

Track Doubao visits with xSeek.

Track every ByteDance crawler hit.

Monitor Doubaobot, Bytespider, and TikTokSpider separately. See how your content flows into ByteDance's recommendation and AI systems.

Start free →

Frequently asked questions

Mozilla/5.0 (compatible; Doubaobot/1.0; +https://www.doubao.com/bot). ByteDance also operates Bytespider for general crawling and TikTokSpider for video ecosystem content.

Source: Doubao's bot page and ByteDance's webmaster center.

PreviousDeepSeek user agents NextAll user agents