I think we need to take a few minutes to remind ourselves of some important concepts that relate to Keyword Research. It's easy to get caught up in the Keyword Research hype and overlook some of the core concepts and assumptions that all Keyword Research is based upon.

Let me say from the outset that this isn’t an exact science, and it certainly isn’t perfect. We're trying to pull out a bunch of information based on limited and often flawed data. In the same way that stockbrokers use existing data and tools to try to best guess what the market is going to do, we keyword researchers are doing a similar thing. Thank goodness Keyword Research is a lot easier than trying to best guess the stock market!

I will outline some important concepts and assumptions that you should have clearly in mind before you begin your Keyword Research. Even if you have been doing Keyword Research for years, it is worthwhile reviewing these concepts as a reminder of what foundation we're building our research upon.

As I explained in the “Keyword Research 101” article, the term Vertical Keyword Research describes the variations that can occur on a single keyword or keyword phrase. For example if you're creating a website about “turntables” you'll research phrases that include the word "turntable". A vertical keyword search for "turntable" may return phrases such as “turntables” (note the plural version), “turntable guide” or “turntable slip mats”. This is normally your starting point in Keyword Research from which you can expand your keyword list.

As I also explained in the “Keyword Research 101” article, the term Lateral Keyword Research is the process of finding the keywords or phrases that are similar to your seed (original) keyword, but do not necessarily contain that word or words exactly. So your lateral search for “turntable” may bring back words like “stylus”, “vinyl”, or "headphones". Often you'll need to do this kind of lateral thinking yourself, but many tools are smart enough to offer really useful suggestions.

We also have to remember that Lateral Keyword Research also relates to building your list with a variety of keyword combinations or misspellings. For example, some tools allow you to change the order of the words. A tool may add to your list “turntable slip mats” as well as “slip mats turntables”. Or it may offer some misspelling or alternative variations such as “turntable slipmats”. Or perhaps it will offer you international spellings such as “practice” versus “practise”.

The more creative both your mind and the Keyword Research tools are, the better your outcome. SEO people and Niche Miners may find words they hadn’t thought of that are popular but have little competition. In a similar way, PPC advertisers may generate a huge list of keywords that are overlooked and consequently inexpensive to advertise on. Lateral Keyword Research really is powerful in the right hands.

You may have heard the term “digging”. It’s a weird term to use, but it fairly accurately describes the process.  Digging is often automated or partly automated. It involves processing large lists of keywords to reveal the keywords that have the characteristics of terms that are going to make money! Often digging is done by the Keyword Research software itself and will often involve both Vertical and Lateral Keyword Research. It is kind of like having a swimming pool full of coins and having someone sort through all the nickels to find the quarters, or hopefully the banknotes! When digging is done right it can save you lots of time and make you lots of money.

Quite a few tools offer this digging functionality, and it certainly adds lots of value for users who know how to use it correctly. In my reviews I point out the tools that can do this kind of operation so be on the look out!

Keyword Popularity one of the easiest to understand yet most misunderstood concepts in Keyword Research. Simply it's the number of searches performed during a certain period - per day, per week, or per month. That's the easy part to understand.

What many people don’t understand is that this is only a relative indicator. If one tool says that there are 1246 searches a month for a particular keyword, that doesn’t mean this is an exact figure. You cannot then say with any sort of accuracy that during the past month throughout the whole Internet there were 1246 searches for that keyword.

What this figure means is that in the chunk or sample of data that was grabbed from this particular search engine during this month that keyword was found 1246 times. Sometimes this really is all of the searches that were done on this search engine, other times this is just a portion of all the search queries. The only time we could ever be sure that there were 1246 searches done a month would be if the sample data were taken from every single search engine on the Internet. No tool does this - not even close.

Let me clarify further. Wordtracker is one of the leading Keyword Research companies. It has a sample size of about 2 million queries a day while Google alone has about 200 million a day. When you then throw all the other search engines into the mix, you quickly realize that 2 million searches fall well short of the hundreds of millions of actual searches a day.

So does that mean that you can look at the search volume you find in these tools and multiply it by 200? Well, this is a very good question, and one that doesn’t get asked often enough. There is no simple answer, as it will depend on the tool. Many will “adjust” their results so you don’t have to do multiplication, while others will give you only the literal results, and some multiplication will have to take place. Whatever tool you get, it’s worthwhile asking this question.

Keyword Research tools use only samples of data. This means that we cannot assume that our list is 100% correct. It’s really just a best guess based on the chunk of data we have.

To demonstrate this point, take a look below at the differences between the five major keyword data sources for the top 10 terms related to “fly fishing”. If they were all accurate, you would expect them all to produce the same 10 top popularity results. However, as you can below, the tools all say different things. These results emphasize the fact that these tools are guides. They are not absolute truth and the results they bring you should be treated with some suspicion.

Word Tracker

fly fishing
fly fishing alaska
fly fishing equipment
fly fishing gear
alaska fly fishing
alaska fly fishing trip
fly fishing reels
fly fishing canada
alaska fly in fishing
fly fishing in alaska

Overture

fly fishing
fly in fishing
fly fishing trip
fly fishing gear
colorado fly fishing
fly fishing vacation
montana fly fishing
fly fishing in colorado
fly fishing equipment
fly fishing reel

Keyword Discovery

fly fishing
fly fishing equipment
fly fishing gear
michigan fly fishing
fly fishing reels
fly fishing rod
fly fishing alaska
fly fishing reel
fly fishing and tying journal
fly fishing tackle 

Keyword Intelligence

fly fishing
fly fishing knots
fly fishing flies
fly fishing tips
oregon fly fishing guides
fly fishing gear
fly fishing supplies
fly fishing rods
oregon fly fishing guide
fly fishing reels

Web CEO

fly fishing
fly fishing vacation
fly fishing equipment
fly in fishing ontario
fly in fishing
fly fishing instruction
montana fly fishing
fly fishing gear
pennsylvania fly fishing
new zealand fly fishing

The data pulled from Overture (Yahoo! Search Marketing) is also less than perfect for other reasons. In fact in many ways it is worse. The tool actually combines words to give a search result. For example, let’s say that one person searched for “bikini girl”, another person searched for “bikini girls” and yet another for “girls bikini”. These searches would all be combined by the Overture suggestion tool, which would tell you that the three people all searched for “bikini girls”, which is obviously not the case.

This is especially highlighted when you look at the meaning of the two searches “bikini girls” and “girls bikini”. The meanings are completely different, but according to the Overture (Yahoo! Search Marketing) suggestion tool they are the same query. However, in reality one is likely to be performed by a guy looking for some free eye candy, while the other search is likely to be made a young woman looking to spend some money to look good at the beach. I know which one I would rather be targeting as a business owner.

All this negative talk might be getting you a bit discouraged about Keyword Research. Don’t fret, it isn’t all bad. What it means is that we must use our heads a little more, and realize that Keyword Research provides us guidelines.

Firstly, you can get a rough indication of the search volume and fairly safely conclude that the actual search volume is going to be greater when you consider traffic across all search engines.
Secondly, and perhaps most importantly, you can get a relative indicator. By relative indicator I mean it allows you to be able to compare one keyword with other keywords in the same group of data. For example, you may find that “rugby union” is 30% lower than “rugby league”, or “cake recipes” is 100% higher than “cake recipe” (except if you were using the Overture suggestion tool due to the way it combines phrases).

Even though the actual volume isn't exactly correct, by being able to compare relatively you can still get some amazingly powerful information.

Another important concept in Keyword Research is keyword competition. This information can help you assess the difficulty of being able to do well in a particular market, from both a search engine ranking perspective, as well as a business competition perspective.

There are a two common methods that Keyword Research tools use to give an indicator of competition. The first one is called the R/S ratio, which shows the ratio of searches to web pages containing that keyword or keyword phrase. So basically the tool compares how many times something is searched for against how many pages are found in the search engine with that keyword or phrase included in the page.

The second common indicator is KEI, which stands for Keyword Effectiveness Index. This is very similar to R/S. However, it's weighted so the higher the keyword volume, the more tolerance it has to lots of competing pages. So keywords or phrases with lots of searches will have a higher KEI compared with ones that have fewer searches, even though their R/S ratio may be exactly the same.

Both of these measures certainly are of some use. However, they are both fundamentally flawed because they assume that the number of pages on the Internet for a given keyword is a reliable measure of competition. I don’t believe it is. In general, I choose to ignore these indicators until other factors are put into the equation.

In reality, it doesn’t matter how many competing pages you are up against. All that matters is how good are the websites that appear on the first (and maybe second) page that a search engine returns for a query. So how good are the top 10 results? Are they well optimized? Are they already popular, high traffic sites? Do lots of people like those sites and consequently have lots of links pointing to them? If you want to get real competition analysis, you should include factors such as Incoming Links, Keyword Density, PageRank, and Alexa Rank to name just a few. These provide a better way of assessing competition in real terms.

Unfortunately, only one or two tools on the market actually use these measures. Most tools stick to R/S and KEI. However, I'm hoping that as these tools evolve, they will begin to use these more advanced methods to indicate competition.

Many of the Keyword Research tools provide you with indicators given by PPC search engines on how hotly contested your keyword or keyword phrase is. For PPC advertisers this is useful, especially when the actual prices of the ads are shown as well. It allows advertisers to streamline their process for finding cheap yet relevant ads to buy on these PPC networks.

However, this data is also useful for people who are never going to spend a cent on PPC advertising. What the data does is indirectly give an indicator of how profitable a niche is. Unless people are remarkably stupid, it is fairly safe to assume that if these folks are willing to pay many dollars per click, then this area must either be converting well and/or be selling high profit items. You can also use this information to find untapped sub-niches and work those as well.

It's always important to remember while doing Keyword Research that your mind is still your best tool. Don’t be afraid to use it. Think laterally and think how your customers will think. When doing Keyword Research try to put yourself in your customer’s shoes. The more you can do this the more powerfully you can use the software tools you have at your disposal.

Also, consider other factors in the equation that a tool cannot tell you directly. For example, think about the conversion potential of the keyword area. If you’re niche mining, how easy is it to get your hands on a product to sell? Or is there a suitable affiliate product to promote? There's no point having lots of traffic going to a site that has no way to turn that traffic into dollars.

The best book I've seen which gives solid, useful examples of ways to use your own mind as a keyword research tool was published recently by Wordtracker. It's called Keyword Research Guide and it's free.

Wordtracker created a fictional company, Virginia Veg, and asked experts to provide real answers to the problems facing its CEO, Susan Webster. Although the ebook appears to be aimed at beginners, it is crammed with insights, tips, and techniques on keyword research that you can apply to your own website, whether you're using Wordtracker or some other tool.

Contributors include:

• Bryan Eisenberg - Convert more traffic using Wordtracker.
• B. L. Ochman - Understand your customers' real motivations.
• Kevin Lee - Combine thousands of phrases for an effective PPC campaign.
• Stephen Mahaney - Use Wordtracker to find the size of the market.
• Ken McGaffin - Discover the most important sites in your marketplace.
• John Alexander - Find inspiration in Wordtracker's Top 1,000 Words report.
• Neil Davidson - Learn how an ad agency uses keyword research to position a client.
• Robin Good - Use Wordtracker to identify niche opportunities.
• Nick Usborne - Incorporate keywords into your content and copy.

You can download Keyword Research Guide here.