Tuesday, March 12, 2013

Evil Experts-Exchange

Without doubt when I search for a computer related problem, Experts-Exchange pop up. Taunting me with the answer, but asking me to pay to view it.

But that's not evil right? They are providing a service for money. What is particularly evil about whatever website designer made this site is they have the answers display to the search engine bots, so that their pages are more relevant and come up higher in the results. Also the sample text Google provides will usually have almost the answer, trying to pull you in to pay.

There use to be a flaw in their plan. Whenever someone found their site, they would view the cached version by google, and were able to see the "hidden" answers. However, they have pulled the cached version, so that option is not available anymore. So what do we do?

Enter User Agent Switcher. When you visit a site, your browser tells the server which browser you are using to better help the server display it's contents. Well, experts-exchange decide that they are going to display their results to the Google Bot, and by using User Agent Switcher, you can pretend to be the Google Bot. After installing User Agent Switcher, go to tools -> default user agent -> Spiders -> Google Bot 2.1, and visit the site. Scroll down the the bottom the read the results for free. Just remember to switch it back to default when you are finished.