Recently while checking WP_HTTP requests noticed that one plugin started to make too many external requests. Found that error was due to database charset mismatch. Converted database charset from old utf8 to utf8mb4 and related WP_HTTP requests returned to normal.
Watch tutorial on YouTube.
Table of Contents
Details of excess WP_HTTP requests
Using HTTP Requests Manager Plugin for optimizing and monitoring WP_HTTP requests in WordPress.
After one of regular plugin updates noticed excess use of WP_HTTP requests.
Here is the quantity of requests:
- 2 external requests per page
- 100 requests in 15 minutes
- More than 1000 requests per day
Because of these excess external requests 1 second time was constantly added to every page generation time on my website. This is not acceptable for any website. Too many requests were made to WordPress repository which was not acceptable for them as well.
Initial thoughts
Problem started after updating Rank Math plugin to version 1.0.242. Initially I thought that there was some problem with Rank Math plugin and they will fix it in couple days.
I waited a week. Nobody was reporting similar issue on plugin support forum. They didn’t release any fixes or updates.
I wanted to write a support request and explain problem to them. Prepared report but then didn’t find a way to reproduce this error on other websites.
Then I started to investigate problem and look for a solution.
Identifying real problem
Using HTTP Requests Manger plugin, I can see where in code those external requests were made.
Looking through source code of related plugin in editor noticed that WP_HTTP response stored in database as transient.
Checked database but couldn’t find related transient record there.
Noticed that some tables were using old utf8 charset. wp_options database table was also using old utf8 charset.
Utf8 charset can store 1-3 byte UTF8 characters while utf8mb4 can store 1-4 byte utf8 characters. Everywhere utf8 can be 1-4 bytes. But in MySQL you need explicitly define it as utf8mb4 in order support 4 byte characters.
When a string containing 4-byte UTF8 characters sent to regular MySQL utf8 field it will give an error and no data will be written. In most cases, error is not visible to end user or site administrator because errors are hidden on a live (production) website.
To solve problem, we need to convert all database tables from utf8 to utf8mb4 charset format.
Solution
Convert all tables with utf8 collation to utf8mb4_unicode_ci collation using following SQL command.
ALTER TABLE `wp_options` COLLATE='utf8mb4_unicode_ci', CONVERT TO CHARSET utf8mb4 COLLATE 'utf8mb4_unicode_ci';
Convert other utf8 tables as well by using related table name instead of ‘wp_options’.
Additionally change database default collation as well so that new tables will use utf8mb4_unicode_ci.
Result
After fixing database collation transients saved without problem and external requests returned to regular interval. In this case 1 request every 12 hours.
This problem showed two important lessons related to WordPress performance:
- Database errors may remain unnoticed and seem like a plugin error.
- Regularly monitor WordPress performance using debugging plugins. This will help to detect and fix performance related issues.
What is next:
- Use HTTP Requests Manager plugin for monitoring external WP_HTTP requests in WordPress.
- View more WP_HTTP debugging case studies.
- Optimize WordPress speed for better PageSpeed score.