Google Removed My Blog from the Index

Background

Usually, when there’s an SEO need (it’s a bit odd to say “an SEO need,” actually), I’ll register my domain in Google Search Console as well. Besides submitting a Sitemap, it also helps Google quickly find your website and start indexing it, while letting you fix issues found by Google’s search engine.

Bing has a similar feature called Bing Webmaster Tools.

Starting in November 2024, I noticed that my blog’s visibility began showing a clear downward trend from around May 2024.

During this period, there were no code changes on my blog, and there were no error messages in Search Console either, so I temporarily ruled out structural issues.

However, I really couldn’t figure out what content would make Google remove all of my articles from the index.

Investigating the Root Cause

As of around May 2024, the number of indexed pages on my site was still over 700. By mid-November 2024, that number had sharply dropped to just over 100 pages, and by the end of the year, there were only a little over 20 pages left.

This whole sequence was very confusing to me. Aside from nothing looking wrong in the backend, even after asking Google to crawl again, the only explanation I got was “Crawled - currently not indexed.” According to Google’s official documentation, this means:

Google has crawled the page but has not yet indexed it. It may or may not be indexed in the future; you don’t need to resubmit this URL for crawling.

That was completely baffling, so I looked at other results too. For example, this article mentions that for small websites, having pages that aren’t indexed is normal, and that you should prioritize content quality and SEO optimization.

But after searching for a long time, I still couldn’t find a case similar to mine — where visibility was fine at first, then suddenly dropped and the pages were removed from the index

At this point, I don’t even know whether I should complain about Google or not. Low-quality AI content farms occupy the top spots in search results, while these more useful personal blogs are somehow blocked for no reason.

Not to mention that when you click into some content-farm articles, you’re greeted by all kinds of landing-page ads. A few of my earlier posts that were relatively well-read include:

• On HTML Form Applications
• On Third-Party Cookies
• Lessons from Working as a Software Engineer in Fukuoka
• My Experience Using Zeabur’s One-Click Deployment Service

These are the kinds of shares I thought were pretty useful, and before they were removed from the index, people could usually find them right away by entering keywords. But overnight, Google pulled them all. It also made me clearly realize that I can’t rely only on Google SEO traffic to get people to see my site.

Google SEO Trends

Because of the drop in visibility and the removal from the index, I started looking into what else I should pay attention to besides the SEO basics developers usually already know about, such as SSR, meta tags, ld-json, and sitemap.

I unexpectedly came across this awoo article, 【After HCU and AI Overviews: Where Should E-commerce Site SEO Go?】, which explains things in great detail and gives a very thorough account of AI’s development and how SEO has responded and evolved. I recommend everyone take a look.

If you want to learn SEO thoroughly, you don’t actually need to spend a lot of money taking SEO courses. If you’re familiar with web development and have control over the site (so you can directly modify the HTML), most SEO knowledge can be learned from Google’s official guide.

In simple terms, although Google keeps saying it values helpful content, the results it actually shows are not like that at all. Big brands can easily get search rankings, while real expert content gets pushed out.

Google hopes to continuously update its algorithms to filter out useful content, but the results haven’t been as expected, and have even directly buried many small websites. The sites mentioned in the article have all experienced dramatic drops in visibility.

If you look for clues in the awoo article, the issue most relevant to my site might be topical drift. Although my site is mostly focused on technical topics (software development), it also contains a lot of personal commentary, opinions, and life updates. That may be the reason visibility was reduced, but coming back to my original question — removing the entire index because of this makes no sense at all.

My Attempt — Rewriting and Moving to a New Domain

After December 2024, I started rewriting the entire blog. Besides improving the design, I also redesigned the article paths and ld-json, slightly optimized the SEO-related parts, and moved everything to a new domain.

I hesitated for a long time before deciding to move to a new domain, because kalan.dev holds a lot of memories for me. I bought it as soon as Google launched the .dev top-level domain, and it had been with me for more than five years.

After moving to dev.kalanlife.com and setting up 301 redirects, Google successfully indexed my articles after just two days. But two days later after that, Google removed all of my articles from the index again, with the same reason: “Crawled - currently not indexed.”

Whether this is truly ineffective or simply too soon to tell remains to be verified. But at this point, I decided to move the domain back once again to the familiar blog.kalan.dev, while keeping the new design and architecture, to see whether things improve this time.

Putting My Fate in My Own Hands

I’ve already tried emailing Google and asking the community for help. I probably won’t see results anytime soon, so I’ll keep running the blog using different methods. Here are a few possible strategies:

• When I publish new posts, I’ll also try to drive traffic and promotion through major social platforms as much as possible. Sometimes Threads actually ranks better (though I don’t really like it)
• Build up an email newsletter and community
• Tell everyone not to use Google, and instead use Duckduckgo and Bing, since they can both find my site

I’d recommend Duckduckgo more. Its interface is cleaner and doesn’t cram in a bunch of random news like Bing does, and it doesn’t track your behavior.

I’ll still keep speaking through my own website and writing articles as much as I can, and I sincerely ask for everyone’s support! If you like my articles, please also consider subscribing via RSS!