الكود:
User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Disallow: /contact.html
Disallow: /error-page.html
Disallow: /about.html
Disallow: /search/label/?updated-min=
Disallow: /search/label/?updated-max=
Allow: /
User-agent: *
Disallow: /*.html
Allow: /*.html$
Sitemap:http://www.sunnset.com/atom.xml?redirect=false&start-index=1&max-results=500
ملفات الخاصة بخريطة sitemap كتب:/atom.xml
/atom.xml?redirect=false&start-index=1&max-results=100
/feeds/posts/default
/rss.xml
/sitemap.xml