User-agent: * Disallow: /administrator/ Disallow: /cli/ Disallow: /includes/ Disallow: /installation/ Disallow: /language/ Disallow: /libraries/ Disallow: /logs/ Disallow: /tmp/ # Allow crawling of specific file types Allow: /*.js* Allow: /*.css* Allow: /*.jpg* Allow: /*.gif* Allow: /*.png* # Crawl delay (optional) Crawl-delay: 10