Upload New Media Intermittent Failure

We have suddenly started to experience an intermittent (inconsistent) error when uploading media:
“Post-processing of the image failed likely because the server is busy or does not have enough resources. Uploading a smaller image may help. Suggested maximum size is 2500 pixels.”

and when it’s a non-image (PDF, ZIP):
“Unexpected response from the server. The file may have been uploaded successfully. Check in the Media Library or reload the page.”

The corresponding error in the apache log is:
[Fri Jul 31 04:42:04.308166 2020] [proxy_fcgi:error] [pid XXXX:tid XXXXXXXXXXXXXXX] (70008)Partial results are valid but processing is incomplete: [client XXX.XXX.XXX.XXX:0] AH01075: Error dispatching request to : (reading input brigade), referer: https://XXXXXXX.XXX/wp-admin/media-new.php

  • When the upload begins, the progress bar reaches 100%. Approximately 30-40 seconds later, the error appears.
  • This error occurs approximately 30% of the time.
  • The error can be replicated across different browsers and logins.
  • File size or file type do not matter; the error appears in tiny and large files alike.
  • If a file fails to upload, and we try again with the same file, the file will upload at the same rate of success (approximately 30%)

The following steps have been taken to attempt to fix this:

  • Increase PHP memory_limit
  • Disabling reqtimeout module (Apache)
  • Increasing max execution time in PHP
  • Updating PHP/Apache (currently 7.4.8 FPM and 2.4.43, respectively)
  • Increasing server resources (and all other server settings accordingly)
  • Restarting Apache/PHP
  • Restarting the server
  • Swearing loudly (very cathartic)
  • Telling the client to be patient (not so cathartic)
  • Disable/reinstall/reenable ImageMagick (thought it could be a silent fail with thumbnail creation, given the fact it fails with both image and PDFs)
  • Disabling plugins
  • Googling until tabs thought they were one with the tab bar

Additional potentially useful info

  • The uploads directory is NOT divided by date. Don’t know why it was setup this way, but switching it to date-based directories is not feasible due to some hacky scripts the last guy dumped me with
  • There are 271,309 files in the uploads directory – HOWEVER we attempted trying a completely empty uploads folder, and it still produced the same error.
  • This is a brand new problem. It wasn’t happening 2 days ago, and this install is about 5 years old.

Source link