Performance drop upgrading from MySQL 5.7.33 to 8.0.31 – why did it stop using the Index Condition Pushdown Optimization?

I have a table like this (details elided for readability):

CREATE TABLE UserData (
  id bigint NOT NULL AUTO_INCREMENT,
  userId bigint NOT NULL DEFAULT '0', ...
  c6 int NOT NULL DEFAULT '0', ...
  hidden int NOT NULL DEFAULT '0', ...
  c22 int NOT NULL DEFAULT '0', ...
  PRIMARY KEY (id), ...
  KEY userId_hidden_c6_c22_idx (userId,hidden,c6,c22), ...
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 

and was happily doing queries on it like this in MySQL 5.7:

mysql> select * from UserData use index (userId_hidden_c6_c22_idx) where (userId = 123 AND hidden = 0) order by id DESC limit 10 offset 0;
+----
| ...
+----
10 rows in set (0.03 sec)

However, in MySQL 8.0 these queries started doing this:

mysql> select * from UserData use index (userId_hidden_c6_c22_idx) where (userId = 123 AND hidden = 0) order by id DESC limit 10 offset 0;
+----
| ...
+----
10 rows in set (1.56 sec)

Explain shows the following, 5.7:

mysql> explain select * from UserData use index (userId_hidden_c6_c22_idx) where (userId = 123 AND hidden = 0) order by id DESC limit 10 offset 0;
+----+-------------+----------+------------+------+---------------+---------------+---------+-------------+-------+----------+---------------------------------------+
| id | select_type | table    | partitions | type | possible_keys | key           | key_len | ref         | rows  | filtered | Extra                                 |
+----+-------------+----------+------------+------+---------------+---------------+---------+-------------+-------+----------+---------------------------------------+
|  1 | SIMPLE      | UserData | NULL       | ref  | userId_hidden_c6_c22_idx | userId_hidden_c6_c22_idx | 12      | const,const | 78062 |   100.00 | Using index condition; Using filesort |
+----+-------------+----------+------------+------+---------------+---------------+---------+-------------+-------+----------+---------------------------------------+

8.0:

mysql> explain select * from UserData use index (userId_hidden_c6_c22_idx) where (userId = 123 AND hidden = 0) order by id DESC limit 10 offset 0;
+----+-------------+----------+------------+------+---------------+---------------+---------+-------------+-------+----------+----------------+
| id | select_type | table    | partitions | type | possible_keys | key           | key_len | ref         | rows  | filtered | Extra          |
+----+-------------+----------+------------+------+---------------+---------------+---------+-------------+-------+----------+----------------+
|  1 | SIMPLE      | UserData | NULL       | ref  | userId_hidden_c6_c22_idx | userId_hidden_c6_c22_idx | 12      | const,const | 79298 |   100.00 | Using filesort |
+----+-------------+----------+------------+------+---------------+---------------+---------+-------------+-------+----------+----------------+

The main difference seems to be that 5.7 is Using index condition; Using filesort and 8.0 is only Using filesort.

Why did 8.0 stop using the index condition, and how can I get it to start using it?

EDIT: Why did performance drop 10-100x with MySQL 8.0? It looks like it’s because it stopped using the Index Condition Pushdown Optimization – how can I get it to start using it?

The table has ~150M rows in it, and that user has ~75k records, so I guess it could be a change in the size-based heuristics or whatever goes into the MySQL decision making?