robots.txt & XML Sitemap Generator / Validator
Build and test robots.txt crawl rules (including AI bot blocks) or generate and validate spec-compliant sitemap.xml files locally.
How to Configure Crawl Rules & Sitemaps Safely
Pick a Crawl Preset
Start from Allow All, Block AI Crawlers (GPTBot/CCBot/ClaudeBot), or Staging Block All.
Test Paths Live
Use the Live Path Tester to verify whether any URL path is ALLOWED or BLOCKED for a bot.
Validate Sitemap URLs
Paste full https:// URLs in the XML Sitemap tab—invalid URLs are flagged automatically.
Export to Site Root
Download robots.txt and sitemap.xml and deploy both to your domain root (/robots.txt).
Frequently Asked Questions
In-Depth Guides & Tutorials
Formulas, industry standards, and best practices for robots.txt & XML Sitemap Generator.
.env File Syntax Validation, Diff Checking & Kubernetes Secret Conversion: Complete DevOps Guide
Master environment variable hygiene. Learn how to lint .env syntax, compare .env.example against .env.local, sort keys alphabetically, and convert variables to Kubernetes Secrets & Docker Compose in-browser.
How to Write, Read, and Debug Cron Expressions: A Sysadmin & Developer Guide
Master standard crontab syntax, understand the 5 fields, explore common schedule patterns, and avoid production scheduling mistakes.
SVG <path> Commands & Bezier Curves Guide: Inspect, Scale, Shift & Export Vector Coordinates
Master SVG path commands (M, L, H, V, C, S, Q, T, A, Z), visualize Cubic and Quadratic Bezier control points, batch-scale or translate coordinates, and export clean React JSX or CSS clip-paths.
Complementary Tools
Other fast, free, and zero-upload utilities built for your workflow.
HAR File Analyzer, Network Waterfall & Secret Redactor
Inspect HTTP Archive (.har) network waterfalls, slow TTFB requests, and headers locally—plus 1-click redaction of cookies and Bearer tokens.
.env File Validator, Diff Comparer & Kubernetes Secret Converter
Audit .env syntax, compare environment files (.example vs .local), sort variables alphabetically, and convert to Kubernetes Secrets or Docker Compose in-browser.
SQLite Database Viewer, Explorer & SQL Query Runner
Open, explore, query, and export SQLite databases (.sqlite, .db, .sqlite3) entirely in your browser using WebAssembly. Air-gapped privacy with zero data transmission.
JSON Formatter & Schema Diff
Format, repair, diff, and inspect JSON in real time. Convert to TypeScript, Go structs, YAML, CSV tables, and JSON Schema.