robots.txt
A well-formed robots.txt: directives inside User-agent groups, known keys, proper key: value syntax. (site-wide file)
Broken variant
A rule before any group, a bare User-agent:, a missing colon, and an unknown directive.
Catalogued flags
| Flag | Kind | Severity |
|---|---|---|
robotsTxtDirectiveBeforeGroup | defect | high |
robotsTxtMissingColon | defect | medium |
robotsTxtEmptyUserAgent | defect | medium |
robotsTxtOversized | defect | medium |
robotsTxtUnknownDirective | informational — fails nothing | — |