Easy Way to Debug WP_HTTP Errors in WordPress

WP_HTTP is WordPress functionality used for sending and getting information from the internet. It is used to communicate with itself or other websites. Most common use is starting cron jobs, getting WordPress updates, installing plugins from repositories, checking site health, and sending pingbacks.

WordPress plugins use WP_HTTP to communicate with external services. It can be used by plugins for creating backups on google drive, creating encrypted PayPal buttons for ecommerce sites, sending emails using external providers via API, loading search console data to WordPress, get Google analytics data, send command to purge cache from CDN, send image for compression on external servers etc.

There are so many functionalities that rely on WP_HTTP.

By effectively debugging WP_HTTP requests, you can ensure your WordPress site stays connected, up-to-date, and delivers a smooth user experience without sacrificing performance.

WordPress does not have a built in feature to identify or debug WP_HTTP requests. Thankfully plugins can be used to address this shortcoming and provide helpful tools for debugging.

How to debug WP_HTTP requests?

There is a free WordPress plugin called HTTP Requests Manager. Which is programmed to debug WP_HTTP requests.

It is a lightweight plugin that records WP_HTTP requests.

Using this plugin you will get answers to the following concerns:

• How heavily WP_HTTP requests are used?
• Are there any connection errors?
• Which plugins send remote requests?
• Are remote requests slowing down your website?

Plugin records all WP_HTTP requests happening on any page. Which is not possible with another popular plugin like Query Monitor.

Requests inside frontend, admin area, cron jobs, ajax calls, login page, xmlrpc and rest_api will be recorded.

To start debugging WordPress website follow these steps:

1. Install and activate “HTTP Request Manager” from “Plugins” → “Add new plugin” page.
2. Navigate to “Tools” → “HTTP Requests Manager” page and check that it works and records some WP_HTTP requests.
3. Wait a couple days while the plugin collects more requests.
4. Check again “HTTP Requests Manager” page to see recorded data.
5. For easy debugging group requests by: domain, plugin, response status, page.
6. Click on request to view more info about it. You can view sent and received data in a popup. Backtrace of functions initiating request, checkpoint showing memory and time usage, additional requests on the same page also can be viewed there.

Additional tips:

• If you have a particular plugin that you want to debug then use that plugin in a way to trigger some WP_HTTP requests.
• Some requests may happen once monthly so you may need to wait more to collect diverse requests.
• By default the plugin will use “Smart block” mode. If you do not want to block any important requests at this stage and do not want any optimization then switch to “Only Log HTTP Requests” operation mode.

Now let’s look at how to debug WP_HTTP for errors and for performance optimization.

Debug WP_HTTP errors

To debug WP_HTTP request errors:

1. From the plugin page click on the select box and choose the “Group by response” option.
2. Click on the “error” group to expand it.

“Error” group will be populated only if the given report page has errors. If there are no errors then navigate other pages using pagination links at the top and bottom.

From here you can see requests with “Status” 404 are errors. Meaning there is no such file on the remote server.

You will also see red boxes with error messages. In the screenshot we saw that there was an SSL error while getting content from our own server.

Pay attention to Request URLs in order to find failed important requests.

In addition to the error group you can expand the “blocked” (colored with orange bar) group.

Pay attention to request URLs to make sure that “Smart block” optimization mode has not prevented any important API communication. For example connection to payment servers, backup servers, mail APIs etc.

Debug WP_HTTP for performance optimization

For debugging performance issues related to WP_HTTP you can group requests by domain or page type and look for unusual activity.

Here is “Group by request domain” shows that too many PayPal API calls are made via WP_HTTP requests. That is a warning which needs to be investigated and fixed. I explain this issue in more detail in the case study below.

And this is normal WP_HTTP requests without any issues. Most requests are made to the WordPress API which is generally checking updates. You can expand each group by clicking on it.

With all these functionality now lets see how debugging helped us fix WP_HTTP errors and optimize WordPress performance.

Real-World WP_HTTP Debugging Case Studies

I have multiple WordPress websites and all of them have the HTTP Request Manager plugin installed for debugging and optimization of WP_HTTP requests.

I use it to fix WP_HTTP related performance issues as well as monitor and make sure that each WordPress website works without problem.

Here I want to list most WP_HTTP related problems with solutions that I encounter on WordPress websites. Also I want to share a solution here so that others can use it for fixing WP_HTTP related issues.

Case Study 1: Too many PayPal API calls. 94% of all WP_HTTP requests.

Problem:

The HTTP Requests Manager report showed that 94% of all recent requests were made to PayPal API. Viewing details we can see that request done on frontend Pricing page via s2Member plugin. 2 remote requests were sent to PayPal servers to get encrypted button code. WordPress Caching was present but ineffective.

Solution:

• Step1: Remove custom URL variables — We need to use a single Pricing page URL without any additional custom variables. Checking the s2Member plugin setting we found that disabling MOP vars will remove custom variables from the Pricing page. Now the Pricing page can be cached and served for all requests from cache. This will reduce PayPal API usage to normal. Maybe it will reach up to 5% of all WP_HTTP requests. Which is around 20x improvement.
• Step 2: Use static HTML button code — With the above approach there is one caveat. While we removed custom MOP vars from our existing website. Web spiders for many years have indexed thousands of /pricing/ page URLs with MOP vars. They will continue to check those not optimized URLs for several months or years. To fix this we can optimize further and use a static version of PayPal buttons instead of s2member shortcode. This way we have a static version of encrypted PayPal buttons for selling software membership.

Read the full case study and tutorial here.

Result:

After optimization we do not have any PayPal API calls. Payment page uses encrypted static PayPal buttons. Payment page is functional and fast.

Case Study 2: WP_HTTP timeout error prevents sending WordPress backups to Google Drive

Problem:

Connection timeout to Updraft server getting token then storing website backup to Google Drive. Backup is not uploaded to Google Drive.

Updraft plugin takes backup of your WordPress website and stores on local machines as well as in remote servers. I prefer to store backups on Google drive. Plugin automatically takes weekly/monthly backups. Then send it to google drive. While one of those backup operations took place I noticed a connection error (reported in HTTP Requests Manager plugin) to the Updraft server for getting a token for Google Drive via WP_HTTP.

I use the HTTP Requests Manager plugin to monitor WP_HTTP usage for debugging and optimization. Connection to the Updraft server was due to request timeout. Which is reduced to 2 seconds by the same “HTTP Requests Manager” plugin in “Smart block” optimization mode.

Solution:

Ideal solution would be using HTTP Request Manager that has custom rules feature. There you can define a new rule to allow all requests to auth.updraftplus.com domain for getting a token without timing out.

Explicitly allowed rules will not be blocked or limited by short timeouts. Your important remote requests to the API will work as expected.

Result:

Sending WordPress backups to Google Drive by Updraft plugin was restored.

Case Study 3: SSL certificate problem

Problem:

WordPress site health feature will check if your website works correctly over HTTPS.

Solution:

Do not rely solely on Site Health reports. Check if the site is loaded over https using your web browser.

In my case the site is behind Cloudflare CDN servers. So SSL certificate is provided by Cloudflare. WordPress Site Health is checking itself using local DNS without going through Cloudflare. For this reason there is only a self signed SSL certificate when accessed from the same server.

You can safely ignore this warning as we can see that SSL certificate is working when accessed via browser.

Case Study 4: Too many enclosure checks via WP_HTTP

Problem:

When we publish a post WordPress checks all URLs inside the post (links, image sources etc.) for detecting the existence of video and audio files. When found it adds them to the database. Some plugins or themes may use it to format related media files inside content.

But generally 99% of time all those URLs are to web pages and image files. Checking file type of each link to webpage and image via WP_HTTP is a waste of server resources.

Post having 20 internal links and 30 images will make 50 WP_HTTP requests each time you click save button on that post. Those 50 requests can take more than 60 seconds to complete.

Additionally when you check 20 internal pages your own WordPress website has to generate those pages. Which will keep your server busy while generating those pages. 20 pages when it takes 2 seconds to generate will take 40 seconds of your server CPU for providing data to the useless feature “enclosure checks”.

Solution:

Disable enclosure checks with “Smart block” operation mode in HTTP Requests Manager plugin for WordPress.

You can see performance gain from the screenshot above.

Result:

Debugging use of enclosure checks helped us to add functionality of preventing enclosure checks to HTTP Request Manager plugin. This way all WordPress users can optimize their website with “Smart block” mode.

Learn more about optimizing enclosure checks.

Case Study 5: Too many Pingbacks sent via WP_HTTP

Problem:

When post saved WordPress sends pingback to all URLs mentioned in post content. This leads to too many useless WP_HTTP request usage. Once useful, old features are not used any more by blogs.

Solution:

• Suggested definitive solution: Disable pingbacks completely from WordPress settings
• Alternative limited solution: Use “Smart block” mode in HTTP Requests Manager to prevent self-pingbacks.

View detailed instructions to disable all pingbacks.

Result:

Faster WordPress website, no pingback spam and preserving server resources for more important tasks.

Case Study 6: Explicitly block unwanted requests

View on YouTube.

Check how to block WP_HTTP request video.

Problem:

Some plugins and themes may send/request additional information communicating to their own website via WP_HTTP. You do not want to get news, additional promoted features, news, ads etc to be loaded and shown inside your WordPress admin area.

Solution:

If you are not using any features relying on remote communication then it is safe to block those requests.

In the above screenshot we can see that Neve theme tries to connect to their API. But I am not using any features of the theme that rely on API. For this reason I will explicitly block those connections.

While debugging you will notice that API connection is checked as part of WordPress Site Health check. So Neve theme wants to make sure that connection will work when needed.

Explicitly block any plugin, theme, domain communication using “Custom Rules” inside HTTP Requests Manager.

Important to make sure that you are blocking only not important functionality provided by plugin or theme.

Result:

WordPress performance optimized without needless loading of ads, offers, news, unused features into your admin area.

My process for debugging WP_HTTP requests

My process is very simple. I do not explicitly look for WP_HTTP errors or optimization.

I just once in a while (week, month) look into report generated by HTTP Requests Manager plugin.

Most of the time everything is ok without much difference.

But when I see some anomalies then I pay attention and  learn more about problem by clicking on related WP_HTTP request. If it is plugin related issue then look into plugin behavior and settings. Try to understand why this error or anomaly happens.

I double check to make sure if this is really important error. When error is not important do not do anything.

Then look for solution within plugin or WordPress settings. Mostly changing one checkbox or setting will solve the WP_HTTP problem. Like in case study related to encrypted PayPal button or disabling pingbacks.

When there is no solution in settings then I apply “Custom rule” ether to allow or block requests in question.

FAQ

How do I identify if my issue is related to WP_HTTP requests?

Sometimes WordPress can show some error messages related to WP_HTTP requests. But they are too general and you will not know for sure if it is related to WP_HTTP. Additionally WordPress can show errors only on visible pages. You will not get any error on cron or ajax pages.

– Can you understand the above error message related to WP_HTTP?

– No.

WordPress does not have a built-in feature to monitor WP_HTTP requests. You need to use plugins for viewing and debugging WP_HTTP errors.

Use HTTP Requests Manager plugin to log all WP_HTTP requests for debugging. It will record and group requests that had some sort of error in them.

You can even debug requests performed inside cron jobs which are not visible in the web browser.

Check this error message reported by the plugin.

When you find a logged WP_HTTP error you can click on it and see the related plugin and page from which request was made. Additionally you will see an error message with a short description.

All this information will help you to understand and fix errors as fast as possible.

How can I capture and analyze raw HTTP request and response data for WP_HTTP calls?

HTTP Requests Manager plugin records WP_HTTP requests. Click on the request that you want to view data. On the opened window you will see a field containing request and response data.

Too big responses will be trimmed in order to keep the log viewing interface fast and responsive.

Check the screenshot above that displays raw WP_HTTP request and response data.

What are some common symptoms of WP_HTTP request problems?

Common symptoms are:

• Request timeout.
• File not found on remote server.
• SSL certificate problem.

Do I need to enable debug mode in the WordPress config file for debugging WP_HTTP?

No. With the HTTP request Manager plugin you can debug without enabling debug mode. It is designed to work on a live website without using WP_DEBUG constant in the config file.

It does not throw error messages to website visitors or site admin. Instead it logs all requests and then admin can view them in the “Tools” → “HTTP Requests Manager” page. From there you can look for requests with errors. Click on them to view more information and then use it for debugging.

Can I view WP_HTTP errors in WordPress debug mode without any plugin?

No. WordPress debug mode with WP_DEBUG constant does not reveal any additional information related to WP_HTTP request errors.

Check this screenshot with enabled WP_DEBUG constant.

• There is not much indication that this is a WP_HTTP problem.
• Not any detail about WP_HTTP request or response.
• When WP_HTTP error occurs in cron job or ajax call then you will not see any “Warning” message. you Will not notice that there were some WP_HTTP errors.

For these reasons you cannot debug WP_HTTP requests without a plugin only by defining WP_DEBUG constant.

Conclusion

Check every error and make sure that your website functionality is not broken. Some errors can be ignored while others can break your website.

I have explained how to record and debug WP_HTTP requests. Find errors, discover more details, pinpoint the related plugin and page then fix those errors.

Additionally you can monitor logs and auto fix WP_HTTP related performance issues.

I have discovered and fixed all related issues on my websites thanks to the HTTP requests manager plugin. I documented some of those fixes in case studies that are shared in this article.

Use HTTP Request Manager to debug and optimize WP_HTTP requests on your WordPress website.

Click here to learn more & download HTTP Request Manager plugin.