Eric Enge recently interviewed Matt Cutts (Google’s SEO mouthpiece & head of webspam), successfully getting answers and confirmation on how Google handles things such as crawling & indexing, redirects, poor site architecture, and duplicate content — to name a few. Below, we will recap key takeaways and provide commentary on how SEOtool identifies these issues and gives website owners specific details on SEO issues native to their site(s), its architecture and on-page optimization. This blog post is Part 1 of 2 in our analysis of the interview transcript.
Part 1: Crawling, Indexing, PageRank & SEOtool’s Application to Automated SEO Analysis
Topic: Duplicate Content’s Impact on Crawling & Indexing
Typically, duplicate content is not the largest factor on how many pages will be crawled, but it can be a factor. My overall advice is that it helps enormously if you can fix the site architecture upfront.
Imagine we crawl three pages from a site, and then we discover that the two other pages were duplicates of the third page. We’ll drop two out of the three pages and keep only one, and that’s why it looks like it has less good content……..the fact that you had duplicate content and we discarded those pages meant you missed an opportunity to have other pages with good, unique quality content show up in the index.
If you can reduce your duplicate content using site architecture, that’s preferable.
There is a strong, recurring theme straight from Google’s mouth — focus on your site architecture, both at a site-wide and page-level — this is the sure fire way to avoid crawling, indexing or duplicate content problems — and the place to strike gold in your competitive landscape! SEOtool was designed with that exact vision in mind — to remove the guess work and hunches of ongoing SEO site analyses through machine-driven, objective analysis at an architecture & on-page level. This interview 100% validates what SEOtool was built to do and IS DOING for current clients!
Topic: PageRank Distribution & Sculpting
Site architecture, how you make links and structure appear on a page in a way to get the most people to the products that you want them to see, is really a better way to approach it than trying to do individual sculpting of PageRank on links.
I feel like if you can get site architecture straight first, then you’ll have less to do, or no need to even think about PageRank Sculpting.……in my experience, PageRank Sculpting has not been the best use of peoples’ time.
Noticing any redundancy here? Site architecture is the primary theme of the interview! Google recommends you focus energy on perfecting your site architecture for PageRank distribution & consequently crawling potential (see below) — rather than sculpting PageRank via nofollow or other tactics. Again, SEOtool is THE cream of the crop from a SEO Site Architecture analysis platform!
Topic: Crawling Theories vs. Reality
There isn’t really such a thing as an indexation cap.
The number of pages that we crawl is roughly proportional to your PageRank.
There is the concept of host load. The host load is essentially the maximum number of simultaneous connections that a particular web server can handle……..Your site could be on a virtual host with a lot of other web sites on the same IP address. In theory, you can run into limits on how hard we will crawl your site…….that can then set some sort of upper bound on how many pages we are able to fetch from that host.
Following a SEOtool analysis, we’re able to identify not only PageRank distribution across the site, but also where the website inherently has crawling and structural issues that could hinder a search engine’s ability to fetch & index important, unique pages. The custom consulting we bundle into the SEOtool subscription can identify any structural issues preventing the optimal rate of crawling (or PR flow), host load issues that may exist for your site, including bad neighborhoods, cross linking issues and other factors at an IP or DNS level — but even more importantly, give you and your team specific solutions to resolve your greatest SEO deficiencies and ranking problems!
In Part 2, we’ll recap and connect the dots on duplicate content, redirects, site navigation and how SEOtool unearths these issues via actionable metrics.
It’s a gratifying, motivating and humbling feeling when independent & authority sources (like Cutts & Enge) validate what you’ve built and the vision you have for client-driven innovation. The interview between Eric Enge & Matt Cutts could not have been a stronger validation of SEOtool and the consultative approach we take with our clients!
If you would like a demo of SEOtool, please contact us to learn more.
