Part 1 - Launching a New Site
There is no point in reading any further if you don't have quality, original content. Sure, you can use shady SEO tricks, buy traffic, or re-post other people's content and maybe you'll make a few bucks doing it, but this series is about dealing with mainstream search engines - and they are constantly evolving to stay ahead of the cheats and shortcuts that you'll readily find on the internet. There are plenty of people who will sell you ways to increase your search engine rank, however, you will never be able to sustain it long term. If you truly want to build a brand in this crowded space, then you need to be on good terms with the two big players in search, and that means playing by their rules.
These days there are really only two mainstream search engines to be concerned with, Bing and Google. Targeting these guys also takes care of Yahoo and Ask.com as they are just mildly reformulated results from the other two respectively. Of course there are other mainstream search engines out there, but it is generally not worth the time to target them as decent rankings on Bing and Google will eventually land you there anyway. Plus, in terms of overall search traffic, you aren't looking at a lot of leads from their smaller counterparts.
Site Development
I recommend not "publishing" your site until you have the first iteration complete and ready for public consumption. While the definition of publishing your site varies depending on how you have it hosted, the idea is to prevent the search engines from noticing your site and associating the domain name with the unfinished content. The easiest way to do this is to use a robots.txt file to block the search engines from indexing your site while it is under development. Both Bing and Google have what are considered to be well-behaved spiders, meaning mainly that they will honor the contents of your robots.txt file and essentially ignore your site. To do this, create a file called robots.txt with any text editor and add the following two lines:
Save or copy the file to the root (home) directory of your site and you will be effectively invisible. Once you have completed your site and are ready for the world to see it, simply delete the robots.txt file or modify it to suit your needs (doing so is beyond the scope of this article) and you are ready to go live.
Further information: The Web Robots Pages
Going Live
The first thing I always suggest doing when launching a new site is signing up for the Webmaster Tools for both Bing and Google. This will give you insight into the way your site is being seen and indexed by the search engines that you will not be able to find anywhere else. Both are completely free and require only creating an account if you do not already have one, and then associating your site(s) with them. Statistics will take anywhere from a few days to a few weeks to show up once your site is indexed, but there are some things you should do in the interim.
At this point, you will want to create a sitemap so the search engines will understand the structure and layout of your site without having to guess at it. This is the quickest way to get the parts of your site indexed that you want to be seen. A sitemap is an XML file that represents your site in a simple, structured fashion. Here is a subset of our sitemap to serve as an example:
This file is saved as sitemap.xml and just as with the robots.txt file, placed in the root (home) directory of your site. It outlines the different pages on your site (<loc>), when they were last modified (<lastmod>), how often they typically change (<changefreq>) and their priority relative to one another (<priority>). Once the file is in place, you should submit it using the Webmaster Tools. It is important to note that other sitemaps are treated mostly as guidelines when a site is being crawled, and while you can't really control much of anything about your listing or rank with them, they are essential to an effective overall site strategy. Make sure to keep your sitemap up to date so the search engines are aware of changes to your site as they happen.
Once you are setup with Webmaster Tools and have submitted your sitemap, it is really just a waiting game until your site begins appearing in results. I have found Google to do this much more quickly than Bing, which makes sense considering that Google actually indexes the web in real-time. Microsoft simply does not have the horsepower to pull that off, so Bing lags as a result. If you keep an eye on the Webmaster Tools dashboards, you will be able to see exactly when your site has been indexed in the Sitemaps section. There are two bits of information there - the number of URLs submitted, and the number of URLs indexed. If these numbers are not the same (like if submitted is 7 and indexed is 6), you should look into the reason for the discrepancy. Usually the cause is duplicate or invalid entries in sitemap.xml, or it could be an unreachable URL. In any case, errors here will not directly affect your rankings, but the underlying reasons for them can so they are worth addressing.
Further information: sitemaps.org - Home
Now What?
So you've launched your site, signed up for Webmaster Tools, created and submitted a sitemap, and confirmed that your site has at least been indexed by Google - what happens now? You should not make any changes to your site (other than adding new content) for the first month or so while the search engines decide where to start you off, this will make it easier for you to see if you have any problems that need to be addressed once your site begins appearing in the listings.
With Google, you will usually see an increase in your site's rank shortly after it is indexed. More often than not, it will then settle back down a bit. Assuming you have followed my advice and not changed anything on your site yet, don't worry as this is perfectly normal. It is basically Google's way of figuring out where your site belongs to start out with - keep in mind there are billions of sites and yours has to fit relevantly in there somewhere.
Bing is a bit different and seemingly more haphazard, your site will generally start out lower in the rankings, and then very slowly crawl one way or other without ever truly settling (at least early on). Therefore, it is impossible to infer anything without using Webmaster Tools to see how you are doing. Even then, it is difficult to get any truly useful information out of Microsoft until your site has been live for a while. It is for this and other reasons, that I usually recommend people solely focus on Google at this point and moving forward for a bit.
Once your site has stabilized, you should again check the Webmaster Tools. Both engines offer suggestions as part of the package that can help you pinpoint any issues that might be affecting your search engine rankings, and most of the things that appear will need to be addressed. However, there is a right way and a wrong way to update a live website, which will be the subject of the next installment.
Visibility is the key to managing your site's search engine rankings, and there are many third-party tools out there that can aid in the process. Two that I use regularly and have extensive experience with are AWStats and Webalizer. Of course, being able to install these depends on where your site is being hosted, but if you have the option I recommend installing one or the other.
Further information: http://www.webalizer.org/, http://awstats.sourceforge.net/
About Me
I have been on the technology side of this industry for several years, as well as involved in mainstream tech (including search) since the mid-nineties. I currently run an adult search engine called Tent Pitcher.
There is no point in reading any further if you don't have quality, original content. Sure, you can use shady SEO tricks, buy traffic, or re-post other people's content and maybe you'll make a few bucks doing it, but this series is about dealing with mainstream search engines - and they are constantly evolving to stay ahead of the cheats and shortcuts that you'll readily find on the internet. There are plenty of people who will sell you ways to increase your search engine rank, however, you will never be able to sustain it long term. If you truly want to build a brand in this crowded space, then you need to be on good terms with the two big players in search, and that means playing by their rules.
These days there are really only two mainstream search engines to be concerned with, Bing and Google. Targeting these guys also takes care of Yahoo and Ask.com as they are just mildly reformulated results from the other two respectively. Of course there are other mainstream search engines out there, but it is generally not worth the time to target them as decent rankings on Bing and Google will eventually land you there anyway. Plus, in terms of overall search traffic, you aren't looking at a lot of leads from their smaller counterparts.
Site Development
I recommend not "publishing" your site until you have the first iteration complete and ready for public consumption. While the definition of publishing your site varies depending on how you have it hosted, the idea is to prevent the search engines from noticing your site and associating the domain name with the unfinished content. The easiest way to do this is to use a robots.txt file to block the search engines from indexing your site while it is under development. Both Bing and Google have what are considered to be well-behaved spiders, meaning mainly that they will honor the contents of your robots.txt file and essentially ignore your site. To do this, create a file called robots.txt with any text editor and add the following two lines:
Code:
User-agent: * Disallow: /
Further information: The Web Robots Pages
Going Live
The first thing I always suggest doing when launching a new site is signing up for the Webmaster Tools for both Bing and Google. This will give you insight into the way your site is being seen and indexed by the search engines that you will not be able to find anywhere else. Both are completely free and require only creating an account if you do not already have one, and then associating your site(s) with them. Statistics will take anywhere from a few days to a few weeks to show up once your site is indexed, but there are some things you should do in the interim.
At this point, you will want to create a sitemap so the search engines will understand the structure and layout of your site without having to guess at it. This is the quickest way to get the parts of your site indexed that you want to be seen. A sitemap is an XML file that represents your site in a simple, structured fashion. Here is a subset of our sitemap to serve as an example:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>[URL="https://www.tentpitcher.com/"]https://www.tentpitcher.com/[/URL]</loc>
<lastmod>2014-05-27</lastmod>
<changefreq>monthly</changefreq>
<priority>1.0</priority>
</url>
<url>
<loc>[URL="https://www.tentpitcher.com/blog"]https://www.tentpitcher.com/blog[/URL]</loc>
<lastmod>2014-06-01</lastmod>
<changefreq>daily</changefreq>
<priority>0.9</priority>
</url>
<url>
<loc>[URL="https://www.tentpitcher.com/xxxWords"]https://www.tentpitcher.com/xxxWords[/URL]</loc>
<lastmod>2014-06-01</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
</urlset>
Once you are setup with Webmaster Tools and have submitted your sitemap, it is really just a waiting game until your site begins appearing in results. I have found Google to do this much more quickly than Bing, which makes sense considering that Google actually indexes the web in real-time. Microsoft simply does not have the horsepower to pull that off, so Bing lags as a result. If you keep an eye on the Webmaster Tools dashboards, you will be able to see exactly when your site has been indexed in the Sitemaps section. There are two bits of information there - the number of URLs submitted, and the number of URLs indexed. If these numbers are not the same (like if submitted is 7 and indexed is 6), you should look into the reason for the discrepancy. Usually the cause is duplicate or invalid entries in sitemap.xml, or it could be an unreachable URL. In any case, errors here will not directly affect your rankings, but the underlying reasons for them can so they are worth addressing.
Further information: sitemaps.org - Home
Now What?
So you've launched your site, signed up for Webmaster Tools, created and submitted a sitemap, and confirmed that your site has at least been indexed by Google - what happens now? You should not make any changes to your site (other than adding new content) for the first month or so while the search engines decide where to start you off, this will make it easier for you to see if you have any problems that need to be addressed once your site begins appearing in the listings.
With Google, you will usually see an increase in your site's rank shortly after it is indexed. More often than not, it will then settle back down a bit. Assuming you have followed my advice and not changed anything on your site yet, don't worry as this is perfectly normal. It is basically Google's way of figuring out where your site belongs to start out with - keep in mind there are billions of sites and yours has to fit relevantly in there somewhere.
Bing is a bit different and seemingly more haphazard, your site will generally start out lower in the rankings, and then very slowly crawl one way or other without ever truly settling (at least early on). Therefore, it is impossible to infer anything without using Webmaster Tools to see how you are doing. Even then, it is difficult to get any truly useful information out of Microsoft until your site has been live for a while. It is for this and other reasons, that I usually recommend people solely focus on Google at this point and moving forward for a bit.
Once your site has stabilized, you should again check the Webmaster Tools. Both engines offer suggestions as part of the package that can help you pinpoint any issues that might be affecting your search engine rankings, and most of the things that appear will need to be addressed. However, there is a right way and a wrong way to update a live website, which will be the subject of the next installment.
Visibility is the key to managing your site's search engine rankings, and there are many third-party tools out there that can aid in the process. Two that I use regularly and have extensive experience with are AWStats and Webalizer. Of course, being able to install these depends on where your site is being hosted, but if you have the option I recommend installing one or the other.
Further information: http://www.webalizer.org/, http://awstats.sourceforge.net/
About Me
I have been on the technology side of this industry for several years, as well as involved in mainstream tech (including search) since the mid-nineties. I currently run an adult search engine called Tent Pitcher.
Comment