Server Response Time vs. Page Load Time for Google Crawl

Server Response Time vs. Page Load Time for Google Crawl

Recently, +John Mueller specifically said that if GoogleBots experience a high response time of “over than 2 seconds to fetch a single URL” from your site that Google will “severely limit the number of URLs it will crawl from your site,” as reported by +Barry Schwartz.

Then in a post on this topic from +Ana Hoffman he clarified that the “difference between server response time and page loading time” must be addressed and asked if anyone could write a post about that.

Here ‘tis.

And this is the same type of info I gather in Site Audits to help you get your site clean and lean.

Check the Waterfall

This part of the post concerns your overall page load time.

It’s super easy to see the response time of your hosting, and what affects it.

My favorite online tester is WebPage Test but it has been experiencing a lot of issues lately with its testing sites in the northeast of the U.S.

If you use this tester, be sure to select a more western location to get accurate results (as shown below). And choose Firefox as the browser because it caches less than Chrome and will give you worst case results, which will do you more good in knowing what to fix.

Webpage Test Locations

Once the test has finished, click the Details link in the navigation.

There you will see the Waterfall View.

The first line is your server response time. Here’s the one for BlogAid.

Server Response Time BlogAid

And here’s the legend for the color bands.

Waterfall Legend Response Time

As you can see, it took 116ms for all 4 of these events to happen when the homepage of BlogAid was requested. For easy conversion, that’s 0.116s.

If you want to get into the nitty gritty of each of those elements, then you can click the link in the left column of the waterfall and bring up the details. They will display in a pop up that looks like this.

Request Details WebPage Test

You can see exactly how much time each element took.

Full Load Time

Above the waterfall you’ll see all of the load numbers for the page.

Load Time Details WebPage Test

The first number is the full load time of the page. As you can see, it’s 2.518s. I’m in the process of revamping the site and won’t stop tweaking until that comes in under 2.0 seconds.

Rendering

In the 3rd column, you can see that the page has a Start Render time of 1.526s.

Server Request Time

This section deals with each request made from the server.

You can check the time it takes for each element of your page to render, and that’s the response time that Google is checking, if I understand John Mueller correctly.

Below the Waterfall you’ll see a Request Details list of each element requested from the server. This chart shows the initial connection times plus how long each element took to fully download, and so much more. It’s basically an informative index of every request, and it’s origin, that is required to produce the rendered page.

request-details-list-webpage-test

What Effects Server Response Time

The speed of your host server is the biggest factor in response time.

But it is certainly not the only factor.

The DNS Lookup is the first thing that happens when a visitor or a bot requests your site. (That’s the first blue band in the waterfall.) And using a CDN like CloudFlare or MaxCDN will impact that too, as they become your nameserver.

Number of servers delivering the content can impact load time too. Several items can come from outside your hosting account like Google fonts and counts on social share buttons, plus ads, if you run them.

Time to First Byte can be impacted by a whole slew of elements. FYI, there was an infamous article by CloudFlare a while back that suggested time to first byte was meaningless. They were deluged with ridicule over that. But, I still love them in spite of that one trip up.

Mainly because they not only speed up the site, but CloudFlare is the only free CDN that offers some modicum of bot protection too.

And that’s important.

If you could see the bots crawling your unprotected site, it would look like a roach invested apartment. They eat up system resources. And that negatively impacts every legit visitor and bot that wants access to your site too. It also affects load speed.

No Generic Help Here

There’s a reason why you don’t see posts from me with generic advice about how to make your site lean and clean. It’s because they don’t give you any specifics to do.

Here’s the list of what every generic post has to say:

  • Get on a fast host
  • Get a premium theme (yes, frameworks like Genesis actually speed things up)
  • Optimize your images
  • Limit database requests
  • Secure your site

Okay, that’s great. But can you make those changes? Do you even know where to start? Do you know how to measure the effectiveness of your changes?

Real Help

Ready for a real site audit yet?

You’ll get a detailed report (30-40 pages for the Gold audit) and a live chat (up to 2 hours) where we go over everything, in layman’s terms. You’ll fully understand exactly what the problems are and what needs to be changed, tweaked, or fixed. Then we can make a to do list that breaks it up into bite-size phases. You decide exactly what you want to do and what you want some help with.

Recently, my site audit client, Kelly Wilkness, took the page load speed of her site, My Soulful Home, from 11 seconds down to 3 seconds. It took her all of 10 minutes and she did it all herself. She just needed help identifying what needed to be done. That’s the power and clarity a site audit gives you.

Share on Google+Tweet about this on TwitterPin on PinterestShare on LinkedInShare on FacebookShare on TumblrShare on RedditBuffer this pageShare on StumbleUpon

WordPress SEO Beta 1.8 Plugin – First Look

WordPress SEO Beta 1.8 Test

Team Yoast has released beta versions of their popular WordPress SEO and Google Analytics plugins for the public to help test.

I put the WordPress SEO beta plugin through its paces to see if it meets the claim of 30-40% faster processing. Plus, I ran it through Google’s mobile testers and checked the microdata output on the Google Structured Data Tool as well. Get a quick look at my results. [Read more…]

Share on Google+Tweet about this on TwitterPin on PinterestShare on LinkedInShare on FacebookShare on TumblrShare on RedditBuffer this pageShare on StumbleUpon

The Real Reason Authorship Failed

Authorship Dies

Yesterday, Google announced they were killing authorship via a G+ post by +John Mueller who is a Webmaster Trends Analyst for Google. Right behind it, +Eric Enge and +Mark Traphagen, both of Stone Temple Consulting, and SEO gurus, released a post about it on Search Engine Land.

Both posts gave explanations for the demise of authorship, but you have to read between the lines to find the real reasons authorship failed, and what’s next for SEO, and why it matters.

Why Authorship was Started

Google instituted authorship markup in 2011 as a way to help identify the original author of published content. The idea was to use a G+ profile as a verified digital identity and link it to blog posts that author wrote, no matter whether it was on their site or elsewhere online.

The perk of giving the verified author a mug shot next to their post in SERPs was used to encourage adoption.

But the real reason Google started authorship was to help them identify spammy content from bogus authors or folks trying to game the system, and penalize it so that it did not rank higher than higher quality content that had more authority.

In other words, authorship was never about the author.
It was about Google’s ability to serve up better search results.

Why Authorship was Doomed

It stupefies me how Google consistently gets in their own way of making something work. And yet, they make it seem like we’re not cooperating with them. Here are a couple of examples of that.

Low Adoption Rates

This is one of the reasons that Mueller gave for why authorship was working.

The Enge/Traphagen article summarized Mueller’s thoughts with this:

“Even when sites attempted to participate, they often did it incorrectly. In addition, most non-tech-savvy site owners or authors felt the markup and linking were too complex, and so were unlikely to try to implement it.”

Okay, let’s have a look at that statement.

  • Google makes it difficult to implement authorship.
  • Then Google complains of low adoption rates.
  • Duh!

Tutorials Galore

Me and a few other folks quickly added tutorials for how to setup AuthorRank and profited handsomely from Google’s mastery of taking the simplest thing and making a convoluted mess out of it.

In fact, that’s how Mark Traphagen and I met. He was a rising star at that point, mainly over this topic.

He sternly corrected me for using the term AuthorRank. I kissed his ring and told him that he was the guru, and he had to use the correct term of authorship. I was a solo entrepreneur trying to help other folks like me, and I used any damn word I needed to so folks could find the tutorials. Come on, that’s good SEO on keyword for crying out loud! At least I’m white hat about it.

He understood, patted me on the head, and we’ve been on good terms since. And he has been the level voice of reason I’ve gone to for help reading between the lines on every message Matt Cutts put out about authorship.

Google’s Idea of Fixing the Broken System

Because they really wanted to give authorship a good experimental run, Google decided not to make it easier for more sites to integrate authorship.

Instead, Google decided that it could do a better job of just guessing who the real author was.

That “led to many well-publicized cases of mis-attribution, such as Truman Capote being shown as the author of a New York Times article 28 years after his death,” as cited by Enge/Traphagen.

Low Value to End Users

That’s the second reason Mueller gave for getting rid of author photos from Rich Snippets in search.

That’s what he said. This is what it means.

The Role of the User

End users – that’s what Google cares about.

Google’s aim has never been, and never will be, helping you promote your business.

Their goal is to deliver high quality to search to those who use Google for that purpose.

The Role of Cell Phones

And now, more end users are doing those searches on mobile devices, specifically cell phones.

Images take up precious screen space and bandwidth.

It’s more important for Google to show a title and a description than an image of an author or even an image of a video.

Lack of Clicks

Titles matter more in search.

Mueller is the analyst who watched the data for Google and compared click rates to SERPs with and without authorship snippets. According to him, images in Rich Snippets lost their value.

While Mueller didn’t explain what he meant by value, Enge/Traphagen surmised that

“Perhaps over time users had become used to seeing them and they lost their novelty.”

Is Authorship Truly Dead?

No.

Because search has become personalized, you are still likely to see the author image next to the post of someone in your G+ Circles or Gmail contacts in search.

We don’t know how much longer that will last.

Enge/Traphagen speculate:

“After three years the great Google Authorship experiment has come to an end … at least for now.”

At least for now. Tomorrow, who knows?

Google’s Always Moving Cheese

If you have authorship connected, leave it that way.

If you don’t have it connected, maybe you want to do it because Google could always change their minds, again.

Or, maybe you’ll leave well enough alone and go do the more traditional promotional routes of getting the content you create found.

Chasing Cheese Works

Every platform changes how it works and the perks or penalties they give.

If you’ve been in the online promotion game long, you’ll remember the rise and fall of Facebook business pages, the introduction of Pinterest and the subsequent whirlwind of adoption. And on and on.

Here’s what I know.

When I moved my business and content from a “me too” follower to “cutting edge” thought leader philosophy adopter, and I started chasing cheese earlier rather than later, I started making more money.

That has not changed.

Systems will come and go. Making the most of them while they are here is key.

When Not to Chase Cheese

There will always be perks and pitfalls on every platform.

Ways that work super for one business type will fall flat for another.

You have to capitalize on the cheese that make sense for you, your business, and your audience.

What Google is Pushing Today

The demise of authorship was slow. Before it completely went away, Google started giving perks sites with semantic markup, and they still are.

They’re also giving perks currently to sites that can capitalize on publishership.

And most recently, they announced SSL as a light-weight ranking factor to encourage adoption. This is the one where I drew the line. The other perks are not really all that hard to set up or chase. This one is, and comes with lots and lots of caveats and pitfalls when converting an existing site. (Building a site on SSL from the ground up is different.)

Get it or Die

For every type of cheese, there will be hype and controversy. Remember the famous words of Eric Schmidt in The New Digital Age, who said about authorship,

“Within search results, information tied to verified online profiles will be ranked higher than content without such verification, which will result in most users naturally clicking on the top (verified) results. The true cost of remaining anonymous, then, might be irrelevance.”

Yeah, I quoted him many times in an attempt to get more folks to jump on the authorship bandwagon and get some easy Google juice in the bargain. I certainly made more money from having that perk and encouraged others to do the same. I’ll still be setting up new sites with it – because I can, and Rich Snippets are still showing, at least a little.

And there will be hype over the next new thing too

And lots of folks, maybe even me, will make it sound like it’s the most important thing on the planet. I will if it makes me money and helps my clients make money with their site. Oh yes sir, you bet! That is THE bottom line, right?

So, those of us who tried to do what Google said may feel ripped off or betrayed or such.

Get over it!

Google does what’s good for Google. If you help Google do that, you’ll be rewarded for as long as that perk is available. Same with other platforms.

So, quit lamenting the death of authorship and get on board with whatever new thing makes sense for your business.

Share on Google+Tweet about this on TwitterPin on PinterestShare on LinkedInShare on FacebookShare on TumblrShare on RedditBuffer this pageShare on StumbleUpon

Google SSL Ranking Factor – the Why, What, and How

Google SSL Ranking

Yesterday, Google officially announced that sites with SSL (https) would get a bump in rankings. Multiple news sites ran it like it had to be done tomorrow. All manner of developer forums lit up with folks asking how to do it. And a few of us, said “not so fast!” The media circus on this will be going for at least another year. Here’s a follow up on yesterday’s post with more info and things to consider before you make this move to HTTPS.

A Cautionary Tale

Here’s my original post on this topic, advising that folks should take a moment to consider the bigger picture.

SSL as a Google Ranking Signal – What You Need to Know Now

In it mentioned several caveats without explanation. Most of them can be fixed, but require either research or more investment.

What I was mainly wanting to address in that post, and in a hurry, were the folks having a knee jerk reaction and leaping before looking by just getting an SSL certificate and changing all of their permalinks over from http to https and then having to fix all of the issues I mentioned after the fact.

A minute of planning is worth an hour of troubleshooting.

Today I want to go more in-depth about what’s really going on, and give you some resources to help you make a good decision for your site and avoid expensive pitfalls.

Is HTTPS More Secure?

Yes, but… It’s an encrypted connection. It makes trading info back and forth between a site and a visitor safe from middle man attacks.

Let’s have an example of that. You’ve heard of computer hacks where someone can record all of your keystrokes? That’s a middle man attack.

Sites that take transactions, like credit cards, have to use the highest level of SSL certificate to ensure that the information they are being given by the visitor remains private.

And that’s also why a lot of site owners use services like PayPal. Let them worry about the security. That’s also smart.

The Other Side of HTTPS

Security goes both ways. If a visitor wants to download a file from your site, if it’s over an HTTPS connection, there are no middle man hacks.

But, that does not guarantee the file is free from malicious code. It could still infect your computer, or other device you’re downloading it to.

So, is it worth having an SSL certificate for folks to get a PDF from you?

Hasn’t been so far. But that may change. More on that in a moment.

Does it make reading your blog post safer? Not really.

You’re still going to start hearing more about why your blog-only site needs SSL anyway. In fact, you’re going to hear 100s of opinions about this from all manner of “experts” for at least another year. And all of those opinions are not going to be in agreement. In fact, they will be from one end of the spectrum to the other.

Google’s Looking Out for Google

Here’s a quote that got overlooked in yesterday’s announcement from Google.

“We invest a lot in making sure that our services use industry-leading security, like strong HTTPS encryption by default. That means that people using Search, Gmail and Google Drive, for example, automatically have a secure connection to Google.”

So, it makes sense that Google wants you to just stay secure the whole time you’re connecting with them. That ensures there are no middle man attacks when you jump between their products.

Follow the Money

Think about it. Google makes money from the premium version of Google Apps. And those products are specifically geared to appeal to businesses with teams that want to get rid of the expense of buying Microsoft Office products.

To encourage folks to store everything in the cloud, including spreadsheets, emails, and other documents that may contain sensitive information, security has to be a top priority for Google.

Rise in Middle Man Attacks

We are in the midst of the highest sustained bot attack in history, and it’s still growing.

For the last several months, I’ve been encouraging folks to shut of XML-RPC completely on their site.

For the last several weeks I’ve been encouraging folks to shut down unsecured FTP accounts and ensure that they are retrieving their email via an encrypted connection.

Middle man attacks are on the rise.

Jumping from encrypted site to encrypted site as you surf the web and then grab email from an encrypted source and then go to an encrypted file storage, like Google Drive, helps keep you secure.

And that’s one of the reasons some folks have been screaming for years that all sites should be encrypted as https from the get go.

Google makes things like authorship, publishership, and now encryption a ranking factor to drive adoption.

Who Has to Have SSL Already

Any site that is taking transactions, like e-comm stores and payment gateways, are required to have an SSL certificate.

And some folks think those are the only types of sites that will ever need them. While true, that may be changing due to the middle man attacks and other factors.

The Caveats

If you made your site secure from the onset, good for you! You’re ahead of the game already. And now you get brownie points and a pat on the head from Google too.

If your site is not on https already, here’s more of what you need to consider.

Do it Because it’s Right For You

First, don’t make this change because of the ranking factor bump alone.

The original release from Google stated that it is a minor ranking factor, and that the might at some point give it more weight. If you read other reports from major news services, they make it sound like it’s a mandate from Google and factors big in ranking. Not so.

Permalinks

Changing permalinks is no small thing, whether it’s to https or just another permalink structure.

First, you have to change all the links on your site, including pages, posts, uploads, css, javascript, everything.

Until you can get all of that done, and maybe even after, you have to set up redirects. (There are a LOT of varying opinions on this.) That can be accomplished a lot of ways, perhaps one of the best is via the .htaccess file. And then change your URLs in Settings > General. (There are a LOT of varying opinions on this too.)

Then, refresh your XML and HTML sitemaps. Then submit the new sitemap to Google Webmaster Tools. Your old permalinks will still show up in SERPs until Google finishes indexing to show the new ones. And if someone clicks one of those links, they will be redirected to the new page.

A redirect means that there are more requests. Most performance graders don’t like too many redirects.

And redirects take more time to deliver the page. It may a super small time, but it’s there. Depends on your server and database, and a whole host of other factors.

After a while, Google will have re-indexed all of your pages and visitors will have updated their bookmarks and it will not be much of a factor. But that does take time.

(More on redirects from Yoast in a moment.)

SSL Speed

Google is encouraging folks to use the TLS (Transport Layer Security) way of delivering encrypted pages.

Here’s what they said in their announcement.

“In the coming weeks, we’ll publish detailed best practices (we’ll add a link to it from here) to make TLS adoption easier, and to avoid common mistakes.” (emphasis added)

For a long time, establishing a connection over an encrypted channel caused a speed issue. Only with advancements in tech, hardware, and faster ISP has that improved.

Even so, you need to help it out a bit.

SPDY  is an open-source protocol has been developed that offers better data compression, among other things, thus faster delivery.

To give you a relatable example, when .mp3 compression came along, music downloads stopped taking forever and sites like iTunes boomed. Same with the compression types used over the years by YouTube. Now it’s so good, and base ISP and wi-fi speeds are so fast, that they can deliver HD video quality.

The bottom line is that you have to make sure you have the proper compression available so that your new https pages don’t take a performance nose dive and lose ranking.

Costs

The prices for SSL certificates are all over the place.

And, there are different types of certificates.

One type is Shared, and this is the most common type that hosts offer for free. It’s “shared” because it is a blanket certificate, and not just for you. It is also the least secure of them.

This is the type of SSL certificate that everyone went scampering after when Facebook required content for gated pages to be encrypted.

More on costs in the next sections.

Integrating with Other Services

You may need to either get a static IP address for your site, which is an extra cost at some hosts. Or, check to see if your host supports Server Name Indication.

If you’re using a CDN like CloudFlare, the free version does not accept SSL encrypted sites. You have to get at least the $20/mo plan for that. And Amazon CloudFront CDN is even higher, to the tune of $600/mo for some setups (so I’m told, trying to verify this)

UPDATE: CloudFlare just announced they would make SSL available for free accounts beginning in mid October!
Read Google Now Factoring HTTPS Support Into Ranking; CloudFlare On Track to Make it Free and Easy

Code Breaking

Some existing code on your site may not play so well with SSL.

Joost de Valk (Yoast) has a nice post about moving his site to SSL back in April, shortly after Matt Cutts announced he would “love to make it part of the ranking algorithm.”

Here’s what he said about internal links and redirects:

“All of your internal links should start to use https, not just to pages, but for images, JavaScript, CSS, etc. This means going through your theme with a fine comb and cleaning all of those up. Of course you can have your web server redirect http to https (more on that below), but not having to do the redirect is a lot cheaper.”

Yoast mentions many other implementations and integrations that he made to make the site run fast and smooth with SSL. You’ll want to at least scan that post, even if you don’t fully understand all of the tech, just to see what it takes to make SSL work well on a converted site.

More Conversations

As mentioned already, you’re going to see a plethora of posts on this topic for another year as the conversation about online security escalates.

And, you’re going to see opinions from both sides, in the middle, and every place in between. It’s almost as divisive as talking religion at dinner. Folks hold these opinions with passion, whether they know what they’re talking about or not. And yes, even highly regarded experts have been known to be wrong on occasion.

Plus, there are lots of designers, developers, coders, security and web specialists who are drooling right now. All they see are the $$$$ of folks who will hire them to get their sites converted. They will NEVER tell you it’s not a good idea for your site. They will also never tell you the color bright orange isn’t a good background either. They are going to deliver what you pay them to deliver. Knowing what and why is up to you.

Do What’s Right For You

You’ll have to make up your own mind about what’s best for your site. At this time, I will not be converting BlogAid to SSL. That’s not to say that I won’t build a new client site from the ground up without it, though. That’s different.

I’ll keep you posted as I consider more data coming in about security, and Google’s motives, including how much this is going to line their pockets and those of their shareholders.

I can tell you now that I won’t be chasing the bump in rankings for this. It’s not a good enough reason for my site or my niche. I’ll let you know if that changes.

I’ll also be periodically updating this post as news is released, or security changes happen. If you see something out of date, please let me know. It’s changing rapidly.

Share on Google+Tweet about this on TwitterPin on PinterestShare on LinkedInShare on FacebookShare on TumblrShare on RedditBuffer this pageShare on StumbleUpon

SSL as a Google Ranking Signal – What You Need to Know Now

Google SSL and HTTPS Cheese

We all want secure sites. A big part of my business is helping folks do that. But, the new SSL ranking factor is NOT something you want to jump into lightly. In fact, you need to seriously consider the bigger picture before you make any changes this drastic to your site. Read on for what you need to know before you do anything.

UPDATE: Take a more in-depth look with my follow up post
Google SSL Ranking Factor – the Why, What, and How

[Read more…]

Share on Google+Tweet about this on TwitterPin on PinterestShare on LinkedInShare on FacebookShare on TumblrShare on RedditBuffer this pageShare on StumbleUpon

Noindex Hide Page from Google and Sitemaps

The WordPress SEO plugin has 5 settings that make it super easy to keep special site pages out of search. for example, special sales pages or membership pages. They can be hidden from both your HTML and XML sitemaps. See how to do it in this quick video tutorial.

Updated: Yoast has updated the plugin since I made this video. Settings are just a little different (less of them) but still close enough for jazz.


This is same kind of SEO info I teach in my 1-on-1 training classes.
And there are plenty more videos just like this
in the full BlogAid Video Tutorial Library.


Share on Google+Tweet about this on TwitterPin on PinterestShare on LinkedInShare on FacebookShare on TumblrShare on RedditBuffer this pageShare on StumbleUpon

Upgrade to Google Universal Analytics

Google Universal Analytics Logo

Google Analytics will soon be known as Universal Analytics. There are two steps to upgrade. Plus, there are several ways to connect analytics to your website. This is not something you want to put off, as Google will eventually deprecate the old analytics code. I’ll show you a quick way to check your analytics account and site code, as well as do updates to your site. [Read more…]

Share on Google+Tweet about this on TwitterPin on PinterestShare on LinkedInShare on FacebookShare on TumblrShare on RedditBuffer this pageShare on StumbleUpon