DEV3 Free Robots.txt and Sitemap Generator — Source Code (React + Tailwind)

Source code of the DEV3 Robots.txt and Sitemap Generator, builds robots.txt from rule groups and presets, lints it for common mistakes, and turns a pasted URL list into a valid sitemap.xml.

Free download: sign in to any DEV3 account and click Download ZIP. Try it live: https://dev3.tech/tools/robots-sitemap-generator shows the finished tool running.

What is in the download

• A standalone Vite + React + TypeScript + Tailwind CSS project: src/tool holds the tool, shared pieces sit beside it
• package.json with exact versions plus a lockfile, so npm ci works
• README.md (run, build, embed in Next.js or plain React) and an MIT LICENSE

What the code does

• Rule groups with allow, disallow and crawl-delay
• Presets for common site types
• Warnings for risky rules, such as blocking the whole site
• Sitemap XML with escaping and URL normalising

Tech stack

React 19, TypeScript, Vite, Tailwind CSS 4.

Install and customise

The How to install and How to use tabs list the steps. In short: unzip, npm ci, npm run dev, then copy the tool folder into your own app if you want to embed it.

Good to know

• A browser-only tool: there is no backend or database in the code.