https://e-sutra.com/blog/the-importance-of-using-a-robots-txt-file-for-technical-seo/