What do you do when a site disappears from Google and Yahoo overnight? $300 reward.
A few weeks ago we had a clients site disappear from Google and Yahoo overnight…when I say "disappear" I mean all the pages in Google were tossed out, and in Yahoo, all the pages except the homepage dropped out.
The site had had some pretty good rankings in both Google and in Yahoo.
The clients site is in what I’d consider a "greyer" area, and we were using some "grey" SEO methods. (FYI, the client knew the tactics we’d be using, and knew the "risks".)
So overnight the site is gone from Google and Yahoo. My first thought was maybe there’d been a big "clean up" with the engines, but no…the guy who’s #1 is buying 1/2 million way over the radar links…
So I check the robots.txt file…hey, if it got boinked from both Yahoo and Google on the same day, must be an issue with indexing the site….the robots text looked a bit "funny" but hadn’t been touched since august…we deleted it anyways….
I then thought, maybe its something in the .htaccess file…nope, that file didn’t exist…
So I thought, maybe it’s a dup content issue so I checked that…nope…that’s not the issue…
I pinged the site….seems fine…
I set up a google sitemap of the site and checked for errors….google didn’t report any errors…
We checked the logs…google and yahoo were still coming…in fact, google was still coming to the site over 50 times per day….but still no results with a site:thesite.com command.
Still convinced it must be a hosting issue that couldn’t be explained (we spoke with the hosting company on the phone as well a few times), we decided to move the site to another host….still nothing….another google sitemap…nothing…..then we removed all the pages, and just put up one page "site is under construction" to see if google would index this 1 page (homepage)….nothing….
I know I’m leaving out several other thing we tried…(but these are the main ones I’m recalling)…but still nothing.
So we were buying a few links that were actually related (really) and we’re in the process of removing all them (for I’d show the site to Google or Yahoo and ask them…but thier first response I know would be "hey, you’re buying links" so I can’t show it to the engines until we remove these paid links….if I do ever show it to them…(thinking maybe it’s some filter I tripped…but accross 2 engines the same day??)
I guess I’m my wits end on this one. I’m totally perplexed on what could cause a site to get bonked from Google and Yahoo overnight.
Anyone with ideas? What am I missing? (and I’ll tell you if I tried that yet). Anyone who finds the answer I’ll pay $300 to upon reindexing.
FYI, I’m not going to give the URL out (unless you’re a close friend I know).
***update – a close friend just mentioned DCMA to me…..ah…I bet that’s it! If that’s the issue, how do you get back? http://www.chillingeffects.org/
**** the above would explain it…but….the content is not coppied (all original), and I don’t see any reports at chillingeffects yet on this site…
15 Responses
Hi!
It’s probably not the case with you, but I had difficulties with Google and Yahoo with a particular site, a few years ago, after implementing Compact Policy headers (CPC) in the server headers.
I also saw a few oddities with web analytics tracking systems on a few sites I worked on, so I would check that out.
On the other hand, none of these problems delisted a site completely, but they made all the pages disappear, except for PDFs.
If you wish to give me more info on the particular site, please don’t hesitate to send me an email.
In any case, I wish you good luck and I’m looking forward to read how that issue will be solved.
Cordially,
Anne-Marie Castonguay
Senior SEO and Web Analytics consultant
VDL2 Communications inc.
700 Wellington, Suite1200
Montreal (Quebec), H3C 1S4
Phone: (514) 599-5712 X 227
Fax: (514) 599-5724
G should be showing a note at the bottom of the SERPs, saying that some results are missing with a link to the claim @ chillingeffects.org, if it was a DMCA.
have you checked whether it’s in an index when searching from another country?
have you checked it against all datacenters?
has there been any action like server move, ip change, insertion of adsense near to the time it happened?
has anybody heared about a central ban list used by google, yahoo and so on? in the past i have heard about a case where a site had been deleted from 3 engines simultaneously.
just dropping from google would not be astonishing, may be someone hacked the site, or abused a bug in the url console etc. but even setting a new robots.txt would most probably not force a deletion a the same day. have you checked the requests for robots.txt against both spiders? and the delivered size of the file?
last but not least: what’s about other engines like msn oder askjeeves? if it’s a dcma act i would expect to see other engines affected too.
holmes: “when you have eliminated the impossible, whatever remains, however improbable, must be the truth”
Anne – I called and left message…I’ll call again.
your_store – yea, there’s no DMCA message in the SERPS
Gerald – there’s 0 traffic from Google, so I’d image that it’s out of all datacenters in all countries. It’s still in MSN and in askjeeves…no hacks…we even moved the url to new host, new IP and put up just 1 page “under construction” and still no indexing…. prior to this there were no changes in IP’s or anything.
It may just be co-incidental that the site was dropped from both at the same time – or perhaps one of your competitors was aggressive on the spam reporting.
It may be that the category you were in happened to one Google have a very strong policy on – I have a niche DMOZ clone that was killed when it started ranking for competitive terms. I had to wonder if the Google “Church and State” hadn’t been blurred by an angry big spender.
DMCA is interesting – I had a fun experience with a claimed DMCA with Yahoo! earlier in the year, but the entire site was tanked by them – no index page, but fine in Google. Plus Google is pretty good at being clear in the page footer when a DMCA has led to a removal.
Possibly the best person I can think of to ask on the matter is DaveN – if there’s an actual technical problem, he’ll probably see it.
Hope that helps.
Brian,
I thought also that maybe the big G and Y had done a crack down…but the top 10 are .edu’s, .gov’s, or Major link buyers….I mean the guy at #1 has 1/2 million backlinks that are all major over the radar buys…our site was #18….wouldn’t think they’d go after #18 and leave the rest….btw, #4 is a “presell page” on a major newspaper…kinda like the wordpress fiasco thing… I can’t see them leaving this stuff, and dumping the guy at #18 who’s actually pretty clean compared to the rest (besides the .edu and .gov)
DaveN, if you’re out there, I’d love you to look at this. Maybe even Jake too 😉
If it was spam reported at the same time, that could have done it, Jim. It’s not hard to imagine – a competitor looks, emails Yahoo! & Google (maybe they have a personal connection to one or both) and ZAP!, the site’s gone.
I’d try a few emails through the Goog webmasters form with “canonicalpages” as the subject line – tell them you moved servers and want to know what happened 🙂
rand…yea, that’s the plan after we clean up all the backlinks…if I sent it in now I’m sure the response would be “hey, you’re buying links”….who me?? I still find it hard to image that they’d zap me when the others in the top 10 are so much worse and weren’t touched.
Like I said – watch out for a competitor with a beeline to just the right spam specialists. And Jim – buying links shouldn’t hurt you according to the powers that be… It should just prevent those pages from passing rank… Anything else would be… well frankly… a weapon. 🙂
Don’t sweat the link buys Jim. Matt enjoys wagging the finger for them but I believe their value is just being discounted/ignored. Unless you are buyings schwacks of links, I’d accept the finger wagging and send in a reinclusion request. IF/When you get a response, at least they should clarify whether it is technical or because of rogue SEO practices.
I find the responses are usually to vague to be useful, but at least it should narrow the field.
Have you looked into subdomain issues? We had a site get it’s arse kicked because of wildcard subdomain resolutions.
http://www.site.com
site.com
garbage.site.com
randomtxt.site.com
Would all dish up the content and leave the subs intact, resulting in several hundred thousand extra pages being indexed – then eventually being banned from Yahoo! and Google (though not on the same day). All wildcard subs now redirect to www and all is well.
Maybe a competitor saw this whole and put up some links for Goog and slurp to ‘find’…
How about a removal request being submitted to Google and Yahoo by an unknown party – i.e. competitor?
I am concerned with thing being mistaken as spam thing. Yesterday I tried a tool called pingoat.com that pings services. After checking the boxes of what to ping I hit enter and it said “you have been blacklisted” and it put me on a list of splogs they are reporting on http://splogspot.com/recent wtf?
Here is where it relates, there are so many ways that people can report things that you could end up being thrown in the trash by someone who manually edits if they determine you are not good. The above is a tool that automatically adds you to a splog list then asks you to register and join their forum if you want to get out, that is pure evil.
All this inaccurate tattle tailing has to end…
And I have sites that are several months old that do not show in Yahoo or Google for anything. They are good honest sites.
I’m with Rand on this one. Maybe you can identify the competitor by looking at other keywords. It’s not always a guy from the most desired SERP.
Is it an oldie Jim, ie. a 302 / 301 issue… JS baiting the spider possibly???
Comments are closed.