How To Make Custom Robots.txt Blogger Blogspot
Make custom Robot.txt is sugested and highly recomended in SEO Practice and importance in blog site especialy for blogger blogspot to get and improve Organic trafics and search result engine rank pages (SERP).
What is meant in each part above if we set up in robot.txt :
- User-agent: Mediapartner-Google , that user let all of Crawler machines to crawl gooogle Adsense as a partner to crawler to display, you must let it and not to change.
Disallow : Let Crawler machines to crawl anything in search engine for web pages.
- User Agent : * , let crawler machine not to crawl in search Engunes for our labels, this setting is default in custom robot.txt dasboard blogger setting for hindrance cintent duplicated.
- Allow : / is as same as homepages that let crawlers machine to crawl all of homepages content or index.html. We can set pages not to allow crawler machines to crawls entire of post or pages like old pages or specific time not to let crawl. we can di like this in Disallow : /post_url/03/2018 that meant pages in set on date not to let crawler machine to crawl on specicifics date posts e.g. Disallow: p/page_url this mean that web crawlers not to let page for indexing and crawler about page.
- Sitemap : https://example.blogspot.com/feeds/posts/default?orderby=UPDATED
You can set like that for crawling entire post for new published post but just limited for 25 publish post to crawl. If you want to more post above it you can set up like below to handle up to 500 post in your blog Sitemap: http://example.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500 and if you up to 1000 posts set up like
More 2000 post you can add like that but change 500 with 1000 and 1000 with 1500.
For alternative you can set Sitemap : https://example.blogspot.com/sitemap if you want to handle up to 3000 post but remove all sitemap above. If you have above 3000 pages add https://example.blogspot.com/stemap
How to make custom robot txt in blogger, For seting up robot.txt you can go to blogger dashboard > search prefrences > crawlers and indexing > custom robots and click yes and type your custom robots.txt and save.
To check your sitemap just write and go to
Note : change http or https it defend on your prefered on your webmaster configuration.
Thanks for reading.
In Blogger we can set Robot.txt file in Blogger Dashboard, It's a txt file to comand search engine crawler what pages in blog to crawl what pages to and no to. Each Blog site has default robot.txt you can see below as default :
User-agent: Mediapartners-GoogleDisallow:User-agent: *Disallow: /searchAllow: /Sitemap: https://yourwebsite.com/sitemap.xml
What is meant in each part above if we set up in robot.txt :
- User-agent: Mediapartner-Google , that user let all of Crawler machines to crawl gooogle Adsense as a partner to crawler to display, you must let it and not to change.
Disallow : Let Crawler machines to crawl anything in search engine for web pages.
- User Agent : * , let crawler machine not to crawl in search Engunes for our labels, this setting is default in custom robot.txt dasboard blogger setting for hindrance cintent duplicated.
- Allow : / is as same as homepages that let crawlers machine to crawl all of homepages content or index.html. We can set pages not to allow crawler machines to crawls entire of post or pages like old pages or specific time not to let crawl. we can di like this in Disallow : /post_url/03/2018 that meant pages in set on date not to let crawler machine to crawl on specicifics date posts e.g. Disallow: p/page_url this mean that web crawlers not to let page for indexing and crawler about page.
- Sitemap : https://example.blogspot.com/feeds/posts/default?orderby=UPDATED
You can set like that for crawling entire post for new published post but just limited for 25 publish post to crawl. If you want to more post above it you can set up like below to handle up to 500 post in your blog Sitemap: http://example.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500 and if you up to 1000 posts set up like
Sitemap: http://example.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500
Sitemap: http://example.blogspot.com/atom.xml?redirect=false&start-index=500&max-results=1000
More 2000 post you can add like that but change 500 with 1000 and 1000 with 1500.
For alternative you can set Sitemap : https://example.blogspot.com/sitemap if you want to handle up to 3000 post but remove all sitemap above. If you have above 3000 pages add https://example.blogspot.com/stemap
How to make custom robot txt in blogger, For seting up robot.txt you can go to blogger dashboard > search prefrences > crawlers and indexing > custom robots and click yes and type your custom robots.txt and save.
https://example.blogspot.com/robot.txtin your web browser so you can see like picture above.
Note : change http or https it defend on your prefered on your webmaster configuration.
Thanks for reading.
