Shopify changed the game for e-commerce with its multifaceted tools for managing and growing a business. As powerful and flexible the platform could be, companies in the past have had issues with parts of the system not being accessible to their end users. Now, Shopify allows you to customize your businesses with the use of the robots.txt file. 

What is the robots.txt file? The robots.txt file is used to inform search engine crawlers and web robots how you want your site to be indexed including which pages can and cannot be indexed. The robots.txt file will not appear anywhere in your theme or inside the Shopify admin area, so you might think it's not possible to customize it. But not only is it possible, it's fairly easy to do.

Here are the steps to customize your robots.txt file:

• Create a robots.txt file. Select Add a new template, selecting robots and finally create template. This generates the default robots.txt file that looks something like this

• For developers using Theme Kit, another way to do the first step is to create the file directly in your local copy of the theme. Create the new file in the templates directory and name it robots.txt.liquid. You then need to add the default configuration to the file: 

It's important to note that you won’t see different results with the default file set up because this is using the existing default rules that have always been applied to your Shopify store. The powerful part comes after you alter your robots.txt file.

• Customize your Shopify to stop search engines from indexing the “all” collection page, the search results page, product tag pages and blog tag pages. These are all extraneous content that users will not need to find via search engines. You can do this customization by adding these rules:

• The new rules are added below the existing file rules. The final file will look like this:

Now that you know the robots.txt trick, your options for customizing your Shopify store are endless. You might want to hide specific products or collections for certain reasons or add special rules required by particular crawlers. It all depends on the needs of your shop. Have fun personalizing your Shopify store using this robots.txt customization method and enjoy the benefits it brings to your growing business. For more tips and insights, check out our Shopify blog.