HS系统从7.6.2之后支持设置robots.txt文件，建议设置内容：
User-agent: * Disallow: /install/ Disallow: /app/superadmin/ Disallow: /index.php?c=app&a=superadmin*如果您的站点没有开启伪静态功能或不支持伪静态功能，请保持robots.txt文件内容默认即可；
如果您的站点开启了伪静态且不希望百度去抓取您的站点的动态URL您可以将robots.txt文件内容设置为下：
User-agent: * Disallow: /install/ Disallow: /app/superadmin/ Disallow: /*?*
屏蔽无用蜘蛛池：
User-agent: MJ12bot Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: SemrushBot-BA Disallow: / User-agent: SemrushBot-SI Disallow: / User-agent: SemrushBot-SWA Disallow: / User-agent: SemrushBot-CT Disallow: / User-agent: SemrushBot-BM Disallow: / User-agent: SemrushBot-SEOAB Disallow: / user-agent: AhrefsBot Disallow: / User-agent: DotBot Disallow: / User-agent: Uptimebot Disallow: / User-agent: MegaIndex.ru Disallow: / User-agent: ZoominfoBot Disallow: / User-agent: Mail.Ru Disallow: / User-agent: BLEXBot Disallow: / User-agent: ExtLinksBot Disallow: / User-agent: aiHitBot Disallow: / User-agent: Researchscan Disallow: / User-agent: DnyzBot Disallow: / User-agent: spbot Disallow: / User-agent: YandexBot Disallow: /
相关文档：百度robots文件详解
