Despite the fact that many consider this tool ineffective, technical SEO optimization is a necessary basis for promotion. There are examples when, despite the presence of a large amount of quality content, sites cannot achieve the desired indicators.
In this article, we will focus on the seven main stages of technical SEO in 2017. Some of them have always been relevant, others are quite new and are applied according to the latest changes in search engines.
1. Check indexing
Let’s start with the number of pages on your site that are indexed by search engines. You can check this by typing “site:domain.com” into the address bar of a search engine or using SEO services.
Site indexing
Ideally, this number should be roughly proportional to the total number of pages on your site, minus the ones you don’t want indexed. If there’s a big difference, you need to look at the pages that are being disallowed from indexing. Which brings us to the next point.
2. Make sure important resources are scanned.
To check if your site is crawlable, you can simply check your robots.txt. But often, this is as inaccurate as it is simple. Robots.txt is just one way to restrict page indexing, so you can use an SEO plugin (like Screaming Frog SEO Spider) to get a complete list of all blocked pages, regardless of whether a robots.txt directive, a NOINDEX meta tag, or an X-Robots-Tag was found.
The pages are closed in robots.txt
Remember that Google can now serve pages as modern browsers do. That’s why in 2017 it’s very important that not only your pages, but all types of resources (such as CSS and JavaScript) are crawled properly. If your CSS files are closed from indexing, Google will not be able to fully evaluate the site. Similarly, if your JS is not crawled, Google will not index any dynamically generated content on your site.
If your site is built using AJAX or relies on JavaScript, you need to specifically crawl with an application that can crawl and render JavaScript.
3. Optimize page crawling by search bots.
This is the number of pages on your site that search engines crawl during a given period of time. You can get information about how many pages are crawled per day in Google Search Console:
Scanning statistics
Unfortunately, Google Search Console won’t give you complete information about crawling individual pages. For more detailed information, you’ll need to view your server logs.
Once you know your crawl rate, you may want to increase it. Optimizers don’t know exactly how Google’s algorithm determines how many pages it needs to crawl, but there are two main theories:
The number of internal links on a page affects;
The number of backlinks from other sites affects.
You can try to increase the number of scanned pages using the following methods:
Get rid of duplicate pages. Every duplicate page that you can block should be blocked. From a crawling perspective, canonical URLs don’t really help: search engines will still crawl duplicate pages.
Prevent indexing of pages that are not relevant to SEO . Privacy policies, terms and conditions of expired promotions are good candidates for a Disallow rule in robots.txt. Additionally, you can set certain URL parameters in Google Search Console so that Google does not crawl the same pages with different parameters separately.
Fix broken links. Any time search engines view a link to a page with a 5XX or 4XX server response, the number of pages to crawl is reduced.
Keep your sitemap up to date , and make sure that an up-to-date Sitemap.xml is uploaded to Google Search Console.
4. Internal link audit
A logical site structure is a prerequisite for good user interaction and crawlability. Internal links also help pages rank more effectively.
Site structure
Here are some tips for auditing internal links:
Measure depth . Organize your site structure so that you can get to important pages no more than three clicks from the home page.
Broken links . They confuse visitors and hurt your page and site rankings. Most SEO crawlers will show broken links, but they can be difficult to find and fix on your site. In addition to HTML elements, don’t forget to look at tags, HTTP headers, and Sitemaps.
Redirect links. Even if a visitor eventually reaches the desired page, going through a series of redirects will negatively impact load time and crawlability. Look for chains of three or more redirects and update the redirect links on the page.
Orphan pages : These pages are not related to other pages on your site – and thus are difficult for both visitors and search engines to find.
5. Review your sitemap
You already know how important a Sitemap is. It tells search engines about your site structure and helps them find new content faster. There are a few criteria to check your Sitemap for:
Freshness : Your XML Sitemap should be updated every time new content is added to your site (preferably automatically).
Cleanliness . The sitemap should not contain garbage (4xx pages, non-canonical pages, redirected URLs and pages blocked from indexing) – otherwise, you risk that the sitemap will be ignored by search engines completely. Don’t forget to regularly check your sitemap for errors directly in Google Search Console.
Size . Google limits the number of URLs in a sitemap to 50,000. Ideally, you should keep it much shorter so that important pages are crawled more often. Many optimizers note that reducing the number of URLs in a Sitemap makes it more efficient to crawl it.