Google Search Console 5xx Error on Some Posts

[KevinOH]

Hi Guys,

I moved my websites to a new host in mid Aug. Everything was good but about a week ago, I noticed that one website had significantly lost traffic.

I checked the website on GSC and found 500 errors warning. About 3/4 of the total content had 500 errors. These URLs had over the course of the month gone and been deindexed (I'm guessing because the G bots couldn't get to them)

Anyway, the server if fine (I'm using uptime robot and there are no reported downtimes). However, when I try submitting the deindexed URLs, I'm getting a message that the bots can't access the server because of a 5xx error.

Now, some pages of the site are indexed and when I submit them on GSC, they are put to the indexing queue. However, the posts that got dropped from the index are not getting crawled because of the 5xx error. Can anyone know what could be the problem?

My hosting service says there's no error with their servers.

Here is what I've tried so far, without success, to remedy the problem.

i) turned on ModSec on the hosting account
ii) Turned off MLS antimalware plugin
iii) Removed WP rocket caching plugin

Note: the robots.txt file is fine.

I've run out of ideas. Anyone with a suggestion?

 

[Ryuzaki]

Are you using Wordpress?

This is a long shot but assuming you're using Wordpress, try going to your Settings and re-save the permalinks. You can move them to default, save, and then back to what you had it is and save again. When moving sites around I run into this often enough. Start here, should take 15 seconds.

You also might have had file permissions change based on the PHP handler and MPM package the new host is using.

Check your .htaccess file too. If you've never tweaked it, then you could download a fresh install of your CMS and re-upload it.

You might need to increase your memory limit: define('WP_MEMORY_LIMIT', '64M); in your config.php, though I'd assume your host would have recognized this was an issue.

Turning on WP_DEBUG might turn up something (again if you're on WP). You can turn it on, set displaying of errors off, and turn on the debug.log file so you can read the errors there.

Could be a plugin, if all else fails. I'd turn them all off and try to request a crawl and see if it goes 500.

 

[KevinOH]

Thanks for your reply.Your're right, I'm using WordPress.

Here's what I've done from your recommendations:

i) Reset the permalinks
ii) Deleted the .htaccess file contents and put the default WP configuration
iii) Increased WP memory limit
iv) File permissions (all good here. 644)
v) WP Debug (no errors)

None of the actions has solved the problems.

I noticed something different when checking the http headers of the posts that are indexed and those that throw out the 500 error. Check the images below:

500 Error Posts

Indexed Posts

I'm interested in this line that's on the post that output the 500 error: Expires => Wed, 11 Jan 1984 05:00:00 GMT

Could this be causing the error? Not sure how to interpret it

 

[KevinOH]

Solved.

Just wanted to update this thread that I've solved the problem. Actually, the 500 error was being caused by the website theme. I'm not sure how the theme was coded but I reverted to WP's default theme and the URLs stopped throwing the 500 error. Moreover, they also got crawled by Google bots.

I've just changed the theme to a different one. I'm all good now.