You are here

Home

The handler_read_* status variables

Because I do a lot of Performance Tuning gigs I get often in contact with these status variables. In the beginning I had a problem to understand them and now I have a problem to memorize the relation of the name and the meaning. Therefore I wrote this little summary:

Prepare the example

To show you the effect I have worked out a little example:

CREATE TABLE test (
    id    INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY
  , data  VARCHAR(32)
  , ts    TIMESTAMP
  , INDEX (data)
);

INSERT INTO test
VALUES (NULL, 'abc', NOW()), (NULL, 'abc', NOW()), (NULL, 'abd', NOW())
     , (NULL, 'acd', NOW()), (NULL, 'def', NOW()), (NULL, 'pqr', NOW())
     , (NULL, 'stu', NOW()), (NULL, 'vwx', NOW()), (NULL, 'yza', NOW())
     , (NULL, 'def', NOW())
;

SELECT * FROM test;
+----+------+---------------------+
| id | data | ts                  |
+----+------+---------------------+
|  1 | abc  | 2008-01-18 16:28:40 |
|  2 | abc  | 2008-01-18 16:28:40 |
|  3 | abd  | 2008-01-18 16:28:40 |
|  4 | acd  | 2008-01-18 16:28:40 |
|  5 | def  | 2008-01-18 16:28:40 |
|  6 | pqr  | 2008-01-18 16:28:40 |
|  7 | stu  | 2008-01-18 16:28:40 |
|  8 | vwx  | 2008-01-18 16:28:40 |
|  9 | yza  | 2008-01-18 16:28:40 |
| 10 | def  | 2008-01-18 16:28:40 |
+----+------+---------------------+

To see the effect of a query do the following steps:

  1. FLUSH STATUS;
  2. Execute the query
  3. SHOW SESSION STATUS LIKE 'handler_read%';
  4. Do an EXPLAIN of the query

Handler_read_first

The number of times the first entry was read from an index. If this value is high, it suggests that the server is doing a lot of full index scans.

+-------------+          +---+---+
| Table       |          | In|ex |
|             |          |   |   |
|             |          |   |   |
|             |          |   |   |
|             |          |   |   |
|             |          |   v   |
|             |          |       |
|             |          |       |
+-------------+          +-------+

SELECT data FROM test;
10 rows in set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 1     |
| Handler_read_key      | 0     |
| Handler_read_next     | 10    |
+-----------------------+-------+

EXPLAIN SELECT data FROM test;
+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+
| id | select_type | table | type  | possible_keys | key  | key_len | ref  | rows | Extra       |
+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+
|  1 | SIMPLE      | test  | index | NULL          | data | 35      | NULL |   10 | Using index |
+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+

So what we can basically say is, that we had 1 full index scan and it did 10+1 index fetches.

Let us do some more examples

SELECT data FROM test WHERE data BETWEEN 'A' AND 'O';
6 rows in set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 0     |
| Handler_read_key      | 1     |
| Handler_read_next     | 6     |
+-----------------------+-------+

+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+
| id | select_type | table | type  | possible_keys | key  | key_len | ref  | rows | Extra                    |
+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+
|  1 | SIMPLE      | test  | range | data          | data | 35      | NULL |    5 | Using where; Using index |
+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+

Here it seems the query is not starting with Handler_read_first though it could theoretically. Instead of we get a Handler_read_key. What we can also see is the "wrong" estimation of the optimizer in the execution plan.

Whit this example the query really could start from the beginning...

SELECT data FROM test WHERE data < 'O';
6 rows in set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 0     |
| Handler_read_key      | 1     |
| Handler_read_next     | 6     |
+-----------------------+-------+

+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+
| id | select_type | table | type  | possible_keys | key  | key_len | ref  | rows | Extra                    |
+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+
|  1 | SIMPLE      | test  | range | data          | data | 35      | NULL |    5 | Using where; Using index |
+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+

But it does not!

The same for this query:

SELECT data FROM test WHERE data LIKE 'a%';
4 rows in set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 0     |
| Handler_read_key      | 1     |
| Handler_read_next     | 4     |
+-----------------------+-------+

+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+
| id | select_type | table | type  | possible_keys | key  | key_len | ref  | rows | Extra                    |
+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+
|  1 | SIMPLE      | test  | range | data          | data | 35      | NULL |    4 | Using where; Using index |
+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+

And this query does something completely different:

SELECT data FROM test WHERE data IN ('abc', 'abd', 'acd');
4 rows in set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 0     |
| Handler_read_key      | 3     |
| Handler_read_next     | 4     |
+-----------------------+-------+

+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+
| id | select_type | table | type  | possible_keys | key  | key_len | ref  | rows | Extra                    |
+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+
|  1 | SIMPLE      | test  | range | data          | data | 35      | NULL |    4 | Using where; Using index |
+----+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+

I was not able to get any Handler_read_first count other than by a real full index scan. So I would say that a Handler_read_first is equivalent to Number of full index scans.

A full index scan is better than a full table scan but still not good because they burn a lot of CPU cycles. But sometimes you cannot avoid it...

Handler_read_key

The number of requests to read a row based on a key. If this value is high, it is a good indication that your tables are properly indexed for your queries.

See also the examples in the previous chapter.

+-------------+          +-------+
| Table       |          | Index |
|             | <------  |       | <--+
|             |          |       |
|             |          |       |
|             |          |       |
|             |          |       |
|             |          |       |
|             |          |       |
+-------------+          +-------+

SELECT data FROM test where data = 'abc';
2 rows in set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 0     |
| Handler_read_key      | 1     |
| Handler_read_next     | 2     |
+-----------------------+-------+

+----+-------------+-------+------+---------------+------+---------+-------+------+--------------------------+
| id | select_type | table | type | possible_keys | key  | key_len | ref   | rows | Extra                    |
+----+-------------+-------+------+---------------+------+---------+-------+------+--------------------------+
|  1 | SIMPLE      | test  | ref  | data          | data | 35      | const |    2 | Using where; Using index |
+----+-------------+-------+------+---------------+------+---------+-------+------+--------------------------+

What makes me wondering in this example (an also in the previous) is, that based on the query there is IMHO no reason to access the table (row)...

SELECT * FROM test where data = 'pqr';
1 row in set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 0     |
| Handler_read_key      | 1     |
| Handler_read_next     | 1     |
+-----------------------+-------+

+----+-------------+-------+------+---------------+------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key  | key_len | ref   | rows | Extra       |
+----+-------------+-------+------+---------------+------+---------+-------+------+-------------+
|  1 | SIMPLE      | test  | ref  | data          | data | 35      | const |    1 | Using where |
+----+-------------+-------+------+---------------+------+---------+-------+------+-------------+

In this example it makes clearly sense...!

The number of requests to read the next row in key order. This value is incremented if you are querying an index column with a range constraint or if you are doing an index scan.

See also the examples in the previous chapters.

+-------------+          +-------+
| Table       |          | Index |
|             |          |       |
|             | <------  |   +   |
|             | <------  |   |   |
|             | <------  |   v   |
|             |          |       |
|             |          |       |
|             |          |       |
+-------------+          +-------+

Handler_read_prev

The number of requests to read the previous row in key order. This read method is mainly used to optimize ORDER BY ... DESC.

+-------------+          +-------+
| Table       |          | Index |
|             |          |       |
|             | <------  |   ^   |
|             | <------  |   |   |
|             | <------  |   +   |
|             |          |       |
|             |          |       |
|             |          |       |
+-------------+          +-------+

SELECT data FROM test ORDER BY data DESC;
10 rows in set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 0     |
| Handler_read_key      | 0     |
| Handler_read_next     | 0     |
| Handler_read_prev     | 10    |
+-----------------------+-------+

+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+
| id | select_type | table | type  | possible_keys | key  | key_len | ref  | rows | Extra       |
+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+
|  1 | SIMPLE      | test  | index | NULL          | data | 35      | NULL |   10 | Using index |
+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+

There is no such status like Handler_read_last implemented like it could be according to the HANDLER functions [ 1, 3 ].

SELECT * FROM test where data between 'A' and 'B' ORDER BY data DESC;
4 rows in set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 0     |
| Handler_read_key      | 1     |
| Handler_read_next     | 0     |
| Handler_read_prev     | 4     |
+-----------------------+-------+

+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+
| id | select_type | table | type  | possible_keys | key  | key_len | ref  | rows | Extra       |
+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+
|  1 | SIMPLE      | test  | range | data          | data | 35      | NULL |    4 | Using where |
+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+

Handler_read_rnd

The number of requests to read a row based on a fixed position. This value is high if you are doing a lot of queries that require sorting of the result. You probably have a lot of queries that require MySQL to scan entire tables or you have joins that don't use keys properly.

This status comes into account if the old filesort mechanism is used [ 2 ].

To make this work we have to modify slightly our table:

ALTER TABLE test ADD COLUMN file_sort text;

UPDATE test SET file_sort = 'abcdefghijklmnopqrstuvwxyz' WHERE id = 1;
UPDATE test SET file_sort = 'bcdefghijklmnopqrstuvwxyza' WHERE id = 2;
UPDATE test SET file_sort = 'cdefghijklmnopqrstuvwxyzab' WHERE id = 3;
UPDATE test SET file_sort = 'defghijklmnopqrstuvwxyzabc' WHERE id = 4;
UPDATE test SET file_sort = 'efghijklmnopqrstuvwxyzabcd' WHERE id = 5;
UPDATE test SET file_sort = 'fghijklmnopqrstuvwxyzabcde' WHERE id = 6;
UPDATE test SET file_sort = 'ghijklmnopqrstuvwxyzabcdef' WHERE id = 7;
UPDATE test SET file_sort = 'hijklmnopqrstuvwxyzabcdefg' WHERE id = 8;
UPDATE test SET file_sort = 'ijklmnopqrstuvwxyzabcdefgh' WHERE id = 9;
UPDATE test SET file_sort = 'jklmnopqrstuvwxyzabcdefghi' WHERE id = 10;


SELECT * FROM test ORDER BY file_sort asc;
10 rows in set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_rnd      | 10    |
| Handler_read_rnd_next | 11    |
+-----------------------+-------+

+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra          |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
|  1 | SIMPLE      | test  | ALL  | NULL          | NULL | NULL    | NULL |   10 | Using filesort |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+

This is really a performance killer and should be avoided whenever possible!

Handler_read_rnd_next

The number of requests to read the next row in the data file. This value is high if you are doing a lot of table scans. Generally this suggests that your tables are not properly indexed or that your queries are not written to take advantage of the indexes you have.

+------+------+          +-------+
| Table|      |          | Index |
|      |      |          |       |
|      |      |          |       |
|      |      |          |       |
|      |      |          |       |
|      v      |          |       |
|             |          |       |
|             |          |       |
+-------------+          +-------+

SELECT * FROM test;
10 rows in set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_rnd_next | 11    |
+-----------------------+-------+

+----+-------------+-------+------+---------------+------+---------+------+------+-------+
| id | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+-------+
|  1 | SIMPLE      | test  | ALL  | NULL          | NULL | NULL    | NULL |   10 |       |
+----+-------------+-------+------+---------------+------+---------+------+------+-------+

Obviously also filtering does not have a impact on the work which is performed:

SELECT * FROM test WHERE ts = '2008-01-18 17:33:39';
Empty set

+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_rnd_next | 11    |
+-----------------------+-------+

+----+-------------+-------+------+---------------+------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra       |
+----+-------------+-------+------+---------------+------+---------+------+------+-------------+
|  1 | SIMPLE      | test  | ALL  | NULL          | NULL | NULL    | NULL |   10 | Using where |
+----+-------------+-------+------+---------------+------+---------+------+------+-------------+

Literature

  1. [ 1 ] HANDLER Statement
  2. [ 2 ] Filesort
  3. [ 3 ] MariaDB HANDLER Commands

Open items, more to investigate

  • What about Falcon, InnoDB, MySQL Cluster and and other Storage Engines?
  • filesort and read_rnd_buffer_size
  • Why are all values + 1?
  • What about joins?

Why are all values + 1?

Roel Van de Paar gave me the following hint:
Values are +1 because of 'end of something' - for instance when you're reading from a data file, the server will try one more time to read the next record and this is what is being logged.

tags: 
mysql
handler
handler interface
status variables
status

Comments

We got some feedback from Paul Campbell about Handler_read_first:

Thought you might like to know that you can get do this without a full index scan (in 5.7 at least) with a query for SELECT MIN(pkey), only the first key is read in this instance. SELECT MAX(pkey) will do the same for Handler_read_last.

e.g.

mysql> EXPLAIN select min(a) from t2;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+------------------------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                        |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+------------------------------+
|  1 | SIMPLE      | NULL  | NULL       | NULL | NULL          | NULL | NULL    | NULL | NULL |     NULL | Select tables optimized away |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+------------------------------+

mysql> select min(a) from t2;
+--------+
| min(a) |
+--------+
|      3 |
+--------+

mysql> show status like 'handler%';
+----------------------------+-------+
| Variable_name              | Value |
+----------------------------+-------+
| Handler_commit             | 1     |
| Handler_delete             | 0     |
| Handler_discover           | 0     |
| Handler_external_lock      | 2     |
| Handler_mrr_init           | 0     |
| Handler_prepare            | 0     |
| Handler_read_first         | 1     |
| Handler_read_key           | 1     |
| Handler_read_last          | 0     |
| Handler_read_next          | 0     |
| Handler_read_prev          | 0     |
| Handler_read_rnd           | 0     |
| Handler_read_rnd_next      | 0     |
| Handler_rollback           | 0     |
| Handler_savepoint          | 0     |
| Handler_savepoint_rollback | 0     |
| Handler_update             | 0     |
| Handler_write              | 0     |
+----------------------------+-------+

In this instance there were 10 rows in the table, so we don't scan the entire index only the first node and then we stop.

Shinguzcomment Mon, 2021-02-08 09:35