Discovered-Currently not Indexed or Crawled What Should We Do?



Ajmal
Whenever I publish a new blog post or page, it is not being indexed by Google and is stuck in "Discovered – currently not indexed" or "Crawled – currently not indexed", I have a good quality of content as well, Not Artificial Intelligence (AI) or cheap content.
Anyone who can help me with this? Anything which I should test or try? Thank you!
10 👍🏽1 💟11
32 💬🗨

📰👈

discovered currently not indexed or crawled what should we do
🔗🏹

Matty
While this would be hard to diagnose and answer without looking at your website and Google Search Console (GSC), here are a few general suggestions to look at:
1) First thing I'd do is a 'site:' search operator check in Google to make sure the URLs aren't indexed.
Sometimes GSC is a little slow to update and may display URLs which have been resolved.
2) How long after you publish a post are you checking to see if it's indexed? If it's a small site it may take a little time to get indexed.
Don't expect a page to be indexed in 5 minutes.
3) Are the pages/posts being noindexed? Check the source code to ensure you have an index tag on your pages/posts and that they're listed in your XML sitemap
4) Check to ensure you have a canonical tag on each page and that it's correct
5) Internal linking. Are you linking to these pages? If your website allows it, add a 'recent posts' widget on your homepage so you automatically link to your new content from your homepage – that'll give it more visibility from the start. Or you can just add a contextual link somewhere on the homepage.
Make sure they aren't orphan pages.
6) Resubmit your sitemap to Google Search Console (GSC). Can GSC read and check your sitemap or do they show errors?
7) Do you have duplicate content? Is this content too similar to another page that's already indexed?
8)Is your content public? If your robots.txt file isn't setup correctly there are times when crawlers can find private content (a rare case though)
Also just to add – what type of URLs are these?
Are they regular page URLs?
Are they Rich Site Summary (RSS) feed URLs?
Are they paginated URLs?
The two latter ones I wouldn't even worry about.
That's my 2c off the top of my head.
Edit: we can also go deeper into crawl budget, but this post is getting long 😅
💟👍🏽2

Ajmal ✍️ » Matty
Thank you for such a good comment and here is my response to your questions:
1) No, It is not indexed.
2) It is 8 months old, and posts from the last 2 to 3 months are not indexed.
3) They are listed in sitemap and while i check those with GSC test live URL, it is fine!
4) I have no idea about canonical tag 🙁
5) I have blog posts section on homepage, and some of them even are not orphan pages
6) Google Search Console (GSC) can read and show errors, working fine!
7) I don't think so there is very similar content on the website.
8 ) Yes, it is public and i almost get 200 traffic per day from other indexed pages of my website.
9) Regular and paginated, both of them!
Do you have few moments to have a look at my website? Maybe you find something there, could be the issue!

Peter
All good points from Matty – also worth noting – none of the search engines promise to index anything or everything, and, sometimes Google Search Console (GSC) will give you false/positives – take a chunk of text (15 words or so) wrap it in quotes and Google it (an exact match search) – it's not uncommon for GSC to state something isn't indexed when it is

Ajmal ✍️ » Peter
I did that search with quotes and I got this (No results found for "TEXT TEXT".)
Peter » Ajmal
That would clarify you don't have a false positive situation – refer to the previous suggestions and/or wait
Ajmal ✍️ » Peter
Okay, Thank You! and i am waiting for last 2 months or so 🙁
Peter » Ajmal
If you've been waiting that long – and you don't have any of the issues mentioned in the first reply to the thread – it would imply perhaps that the content itself is just not that good compared to other sites on the same topic
Ajmal ✍️ » Peter
Right. Maybe! I will test it out and add some more content on the same page and optimize further. Thank you!

📰👈

Faberig
I've got the same problem with two posts since a few weeks. Tried to add more internal links, but it didn't work. I think the problem is this: Google crawls it but doesn't index, because it thinks the post is low quality. In my case the two posts have less words than most posts on my site and cover a similar search intent. Therefore I'll try to consolidate the two posts. Consolidating posts is what John Mueller recommends in this case, according to this article: https://www.SEO-suedwest.de/4314-google-erklaert-warum-gefundene-seiten-oft-nicht-indexiert-werden.html (German, you need to translate)
Google erklärt, warum gefundene Seiten oft nicht indexiert werden – SEO Südwest
SEO-SUEDWEST.DE
Google erklärt, warum gefundene Seiten oft nicht indexiert werden – SEO Südwest

Ajmal ✍️ » Faberig
Right. But in my case those are 4k words each and some are 800 words. Mix of both and on different topics. Right now what i am thinking is that lets give it a single backlink from another site and lets see what happens. And on second post, I will pass that with surfer and optimize it and lets see what happens there!

Jay
Try putting an outbound link to Wikipedia or some other very authoritative site in all posts. I did a little bit of testing with my own website and posts that include Wikipedia links are being indexed same day (for me anyways). I also had an issue before I started linking out to Wikipedia, a post was in "Discovered" for 4 weeks. Finally I copied and pasted the entire post, gave it a new URL and included an outbound Wiki link and it got indexed same day.

Ajmal ✍️ » Jay
Sounds interesting. Sure, This is something i will test right away!
Thank you 😊

Chowdhury
Did you request indexing manually from Google Search Console (GSC)? I'd recommend giving it a shot, especially for 'Discovered-current not indexed' pages. Those pages are not yet crawled by Google. So I assume there are no on-page factors associated with the issue. Crawl budget limit or something like that might be a reason. So, I'd suggest you request indexing them manually from GSC. That will place those URLs in the queue to crawl. Googlebot could see your preference and try to crawl those URLs first even though you have a very limited crawl budget. And one more thing, don't submit all affected URLs to index. That might end up confusing the Google bot. Instead, request indexing only the 'Discovered- currently not indexed' pages first. And then wait and see if this works. That strategy worked for me as I was facing the same issue. 'Crawled-but not indexed' pages are different. They are crawled by Google. But for some reason, Google decided not to index them. Most of the time they are low-value pages or pages that should not be indexed by Google, such as comments feed. So, I don't recommend you dilute things. Request indexing only the 'Discovered- currently not indexed' pages first. Hope this helps.

Ajmal ✍️ » Chowdhury
Thank You
Chowdhury » Ajmal
You're welcome.

Kim
I have the same issue. But I don't do anything with it. I let it index naturally. I have about 160 post. But didn't start to post actively before March this year. But the site is about 1.5 year old. When I post, sometimes they get indexed within 10 hrs. But I have also had best of post that I published in april that did not index before last week. But I regularly post from every week to every 2nd week. About 10 – 15 times a month. Since I started the website I have never used GSC to index the post. I'm curious, let me know how it goes •.
Another thing that I came to think about is that it can have something to do with if you are just in a niche (will get faster indexed) rather than an entire industry (get slower indexed).

Ajmal ✍️ » Kim
Well, the niche is technology!
Kim » Ajmal
1st, is it a niche in the technology industry? Or is it the entire broad industry? 🙂 2nd: That's the name of game man, indexed or not. Ranking or not. Conversion or not. Penalty… and the list is longer (as you know) than my 3 arms. Its all about solving problems. Whether you lose interest in it, you are just one of many millions who try 😉 Sorry to say it…It's TOUGH LOVE MAN!! If you want I can take a look…? 3rd: If you feel any better… I would say you are not the only one… like other have said above here, including me. And SE roundtable has also discussed this issue just a few days ago. https://www.YouTube.com/watch?v=IYXhV09ZdLE That's why Iam saying, I would have just kept grinding.
Google Ranking Update, Pages Dropping In & Out, Big Publishers Benefit & Author URLs and Rankings
YouTube.COM
Google Ranking Update, Pages Dropping In & Out, Big Publishers Benefit & Author URLs and Rankings
Ajmal ✍️ » Kim
Haha right. Will do what you and others are recommending. Thanks.
So you are saying to wait and let it index naturally. Somehow you are right but at the moment this site is not making good money and i really need to add content and get them indexed so that i could make money with it. If it will be like this, posts not getting indexed then i will loose interest in it.

Cohen
This happened to me on my site, nearly 100 pages and I noticed after a few months that 20~ or so were unindexed. Nothing worked to fix it except waiting. My site was new and maybe just needed more signals? I wrote almost all the content and I assure you that it's as good if not better than most of the drivel my competition is putting out. I know it sucks, but that's how it be I suppose.

Ajmal ✍️ » Cohen
Looks like i have to write more and more and wait for Google to index it and even trying some suggestions from other members! Lets see!

📰👈



Discuss Discovered Currently not Indexed by Google SE

What is the Quickest Way to Get a Page Indexed?

Published some Articles on an Expired Domain, but they’re not Indexed yet

Is Crawled Currently Not Indexed Will Never or Hard To Show because It Only been Archive?

Does Google Usually Crawl Text Content of an URL first and delay Indexing it?

Discussion about to Get Solutions for Excluded URLs in GSC



Leave a Reply

Your email address will not be published. Required fields are marked *