Fixed by caching last binary log number used in last_used_log_number
Other things:
- Moved locking of LOCK_log form new_file_impl() to new_file(). This fixed
a bug where LOCK_log could have been unlocked even if 'need_lock' was
not set. Removed not anymore used argument need_lock.
- Made generate_new_name() virtual to simplify the code between
other logs and binary log.
Reviewed by Andrei Elkin
now we can afford it. Fix -Werror errors. Note:
* old gcc is bad at detecting uninit variables, disable it.
* time_t is int or long, cast it for printf's
This patch contains a fix for the MDEV-17262/17243 issues and
new mtr test.
These issues (MDEV-17262/17243) have two reasons:
1) After an intermediate commit, a transaction loses its status
of "transaction that registered in the MySQL for 2pc coordinator"
(in the InnoDB) due to the fact that since version 10.2 the
write_row() function (which located in the ha_innodb.cc) does
not call trx_register_for_2pc(m_prebuilt->trx) during the processing
of split transactions. It is necessary to restore this call inside
the write_row() when an intermediate commit was made (for a split
transaction).
Similarly, we need to set the flag of the started transaction
(m_prebuilt->sql_stat_start) after intermediate commit.
The table->file->extra(HA_EXTRA_FAKE_START_STMT) called from the
wsrep_load_data_split() function (which located in sql_load.cc)
will also do this, but it will be too late. As a result, the call
to the wsrep_append_keys() function from the InnoDB engine may be
lost or function may be called with invalid transaction identifier.
2) If a transaction with the LOAD DATA statement is divided into
logical mini-transactions (of the 10K rows) and binlog is rotated,
then in rare cases due to the wsrep handler re-registration at the
boundary of the split, the last portion of data may be lost. Since
splitting of the LOAD DATA into mini-transactions is technical,
I believe that we should not allow these mini-transactions to fall
into separate binlogs. Therefore, it is necessary to prohibit the
rotation of binlog in the middle of processing LOAD DATA statement.
https://jira.mariadb.org/browse/MDEV-17262 and
https://jira.mariadb.org/browse/MDEV-17243
* MDEV-16509 Improve wsrep commit performance with binlog disabled
Release commit order critical section early after trx_commit_low() if
binlog is not transaction coordinator. In order to avoid two phase commit,
binlog_hton is not registered for THD during IO_CACHE population.
Implemented a test which verifies that the transactions release
commit order early.
This optimization will change behavior during recovery as the commit
is not two phase when binlog is off. Fixed and recorded wsrep-recover-v25
and wsrep-recover to match the behavior.
* MDEV-18730 Ordering for wsrep binlog group commit
Previously out of order execution was allowed for wsrep commits.
Established proper ordering by populating wait_for_commit
for every wsrep THD and making group commit leader to wait for
prior commits before proceeding to trx_group_commit_leader().
* MDEV-18730 Added a test case to verify correct commit ordering
* MDEV-16509, MDEV-18730 Review fixes
Use WSREP_EMULATE_BINLOG() macro to decide if the binlog_hton
should be registered. Whitespace/syntax fixes and cleanups.
* MDEV-16509 Require binlog for galera_var_innodb_disallow_writes test
If the commit to InnoDB is done in one phase, the native InnoDB behavior
is that the transaction is committed in memory before it is persisted to
disk. This means that the innodb_disallow_writes=ON may not prevent
transaction to become visible to other readers before commit is completely
over. On the other hand, if the commit is two phase (as it is with binlog),
the transaction will be blocked in prepare phase.
Fixed the test to use binlog, which enforces two phase commit, which
in turn makes commit to block before the changes become visible to
other connections. This guarantees that the test produces expected
result.
With wsrep_gtid_mode=ON, the appropriate commit hooks were not
called in all cases for applied streaming transactions.
As a fix, removed all special handling of commit order critical
section from Wsrep_high_priority_service and Wsrep_storage_service.
Now commit order critical section is always entered in ha_commit_trans().
Check for wsrep_run_commit_hook is now done in handler.cc, log.cc.
This makes it explicit that the transaction is an active wsrep
transaction which must go through commit hooks.
fill_status.
Also, remove LOCK_status around calc_sum_of_all_status()
Also, rename LOCK_show_status into LOCK_all_status_vars.
This reflects the variable the lock protects.
PROBLEM
-------
Memory sanitizer reports uninitialized comparisons
in log_in_use(), because strings are compared with
memcmp() instead of strncmp.
FIX
---
Use strncmp() to compare strings
extra/mariabackup/fil_cur.cc:361:42: warning: format specifies type 'unsigned long' but the argument has type 'ib_int64_t' (aka 'long long') [-Wformat]
extra/mariabackup/fil_cur.cc:376:9: warning: format specifies type 'unsigned long' but the argument has type 'ib_int64_t' (aka 'long long') [-Wformat]
sql/handler.cc:6196:45: warning: format specifies type 'unsigned long' but the argument has type 'wsrep_trx_id_t' (aka 'unsigned long long') [-Wformat]
sql/log.cc:1681:16: warning: format specifies type 'unsigned long' but the argument has type 'size_t' (aka 'unsigned int') [-Wformat]
sql/log.cc:1687:16: warning: format specifies type 'unsigned long' but the argument has type 'size_t' (aka 'unsigned int') [-Wformat]
sql/wsrep_sst.cc:1388:86: warning: format specifies type 'long' but the argument has type 'wsrep_seqno_t' (aka 'long long') [-Wformat]
sql/wsrep_sst.cc:232:86: warning: format specifies type 'long' but the argument has type 'wsrep_seqno_t' (aka 'long long') [-Wformat]
storage/connect/filamdbf.cpp:450:47: warning: format specifies type 'short' but the argument has type 'int' [-Wformat]
storage/connect/filamdbf.cpp:970:47: warning: format specifies type 'short' but the argument has type 'int' [-Wformat]
storage/connect/inihandl.cpp:197:16: warning: address of array 'key->name' will always evaluate to 'true' [-Wpointer-bool-conversion]
storage/innobase/btr/btr0scrub.cc:151:17: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/innobase/buf/buf0buf.cc:5085:8: warning: nonnull parameter 'bpage' will evaluate to 'true' on first encounter [-Wpointer-bool-conversion]
storage/innobase/fil/fil0crypt.cc:2454:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/innobase/handler/ha_innodb.cc:18685:7: warning: format specifies type 'unsigned long' but the argument has type 'wsrep_trx_id_t' (aka 'unsigned long long') [-Wformat]
storage/innobase/row/row0mysql.cc:3319:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/innobase/row/row0mysql.cc:3327:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/maria/ma_norec.c:35:10: warning: implicit conversion from 'int' to 'my_bool' (aka 'char') changes value from 131 to -125 [-Wconstant-conversion]
storage/maria/ma_norec.c:42:10: warning: implicit conversion from 'int' to 'my_bool' (aka 'char') changes value from 131 to -125 [-Wconstant-conversion]
storage/maria/ma_test2.c:1009:12: warning: format specifies type 'unsigned long' but the argument has type 'size_t' (aka 'unsigned int') [-Wformat]
storage/maria/ma_test2.c:1010:12: warning: format specifies type 'unsigned long' but the argument has type 'size_t' (aka 'unsigned int') [-Wformat]
storage/mroonga/ha_mroonga.cpp:9189:44: warning: use of logical '&&' with constant operand [-Wconstant-logical-operand]
storage/mroonga/vendor/groonga/lib/expr.c:4987:22: warning: comparison of constant -1 with expression of type 'grn_operator' is always false [-Wtautological-constant-out-of-range-compare]
storage/xtradb/btr/btr0scrub.cc:151:17: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/xtradb/buf/buf0buf.cc:5047:8: warning: nonnull parameter 'bpage' will evaluate to 'true' on first encounter [-Wpointer-bool-conversion]
storage/xtradb/fil/fil0crypt.cc:2454:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/xtradb/row/row0mysql.cc:3324:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/xtradb/row/row0mysql.cc:3332:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
unittest/sql/mf_iocache-t.cc:120:35: warning: format specifies type 'unsigned long' but the argument has type 'int' [-Wformat]
unittest/sql/mf_iocache-t.cc:96:35: note: expanded from macro 'INFO_TAIL'
- Changed ERROR to WARNING for MyISAM/Aria message
that are warnings in the check utilities.
This affects for example "client is using or
hasn't closed the table properly".
- Print "Table is fixed" if check succeded in
fixing the table.
Changing data types for:
- seconds from longlong to ulonglong
- microseconds from long to ulong
in:
- parameters of calc_time_diff()
- parameters of calc_time_from_sec()
- Members of Sec6_add
This will help to reuse the code easier:
all other functions use ulonglong+long
for seconds/microsecond, e.g.:
- number_to_time()
- number_to_datetime()
- number_to_datetime_with_warn()
- Field_temporal_with_date::store_decimal()
- my_decimal2seconds()
- Item::get_seconds()
For the purpose of reporting an error to error log, shutdown thread was
attempting to access current_thd->variables.lc_messages->errmsgs->errmsgs.
Whereas current_thd was NULL.
We should log errors according to global lc_messages setting instead of
session setting.
Fixed by deleting the sequence if we where not able to initialize it
I also noticed that we didn't always set the error message when
check_killed(), which could lead to aborted queries without error
beeing properly set. Fixed by default setting error message if
check_error() noticed that killed had been called.
This allowed me to remove a lot of calls to thd->send_kill_message().
Modern compilers (such as GCC 8) emit warnings that the
'register' keyword is deprecated and not valid C++17.
Let us remove most use of the 'register' keyword.
Code in 'extra/' is not touched.
MyRocks internally will print non-critical messages to
sql_print_verbose_info() which will do what InnoDB does in similar cases:
check if (global_system_variables.log_warnings > 2).
For galera compatibility, the main thing is to ensure the FD 1, 2 are
not opened with O_CLOEXEC otherwise galera sst errors don't appear in
the error.log
Files without O_CLOEXEC from the test below:
0 -> /dev/pts/9
1 -> /tmp/error.log (intended)
2 -> /tmp/error.log (intended)
5 -> /tmp/datadir
6 -> /tmp/datadir/aria_log.00000001
(Innodb temp files)
8 -> /tmp/ibIIrhFL (deleted)
9 -> /tmp/ibfx1vai (deleted)
10 -> /tmp/ibAQUKFO (deleted)
11 -> /tmp/ibWBQSHR (deleted)
15 -> /tmp/ibEXEcfo (deleted)
20 -> /tmp/datadir/mysql/host.MYD
22 -> /tmp/datadir/mysql/user.MYD
... (rest of MYD files)
Test for this and the previous commit.
sql/mysqld --skip-networking --datadir=/tmp/datadir --log-bin=/tmp/datadir/mysqlbin --socket /tmp/s.sock --lc-messages-dir=${PWD}/sql/share --verbose --log-error=/tmp/error.log --general-log-file=/tmp/general.log --general-log=1 --slow-query-log-file=/tmp/slow.log --slow-query-log=1
180302 10:56:41 [Note] sql/mysqld (mysqld 5.5.60-MariaDB-wsrep) starting as process 26056 ...
$ cd /proc/26056
$ ls -la --sort=none fd
total 0
dr-x------. 2 dan dan 0 Mar 2 10:57 .
dr-xr-xr-x. 9 dan dan 0 Mar 2 10:56 ..
lrwx------. 1 dan dan 64 Mar 2 10:57 0 -> /dev/pts/9
l-wx------. 1 dan dan 64 Mar 2 10:57 1 -> /tmp/error.log
l-wx------. 1 dan dan 64 Mar 2 10:57 2 -> /tmp/error.log
lrwx------. 1 dan dan 64 Mar 2 10:57 3 -> /tmp/datadir/mysqlbin.index
lrwx------. 1 dan dan 64 Mar 2 10:57 4 -> /tmp/datadir/aria_log_control
lr-x------. 1 dan dan 64 Mar 2 10:57 5 -> /tmp/datadir
lrwx------. 1 dan dan 64 Mar 2 10:57 6 -> /tmp/datadir/aria_log.00000001
lrwx------. 1 dan dan 64 Mar 2 10:57 7 -> /tmp/datadir/ibdata1
lrwx------. 1 dan dan 64 Mar 2 10:57 8 -> /tmp/ibIIrhFL (deleted)
lrwx------. 1 dan dan 64 Mar 2 10:57 9 -> /tmp/ibfx1vai (deleted)
lrwx------. 1 dan dan 64 Mar 2 10:57 10 -> /tmp/ibAQUKFO (deleted)
lrwx------. 1 dan dan 64 Mar 2 10:57 11 -> /tmp/ibWBQSHR (deleted)
lrwx------. 1 dan dan 64 Mar 2 10:57 12 -> /tmp/datadir/ib_logfile0
lrwx------. 1 dan dan 64 Mar 2 10:57 13 -> /tmp/datadir/ib_logfile1
l-wx------. 1 dan dan 64 Mar 2 10:57 14 -> /tmp/slow.log
lrwx------. 1 dan dan 64 Mar 2 10:57 15 -> /tmp/ibEXEcfo (deleted)
l-wx------. 1 dan dan 64 Mar 2 10:57 16 -> /tmp/general.log
lrwx------. 1 dan dan 64 Mar 2 10:57 17 -> socket:[1897356]
lrwx------. 1 dan dan 64 Mar 2 10:57 18 -> socket:[45335]
l-wx------. 1 dan dan 64 Mar 2 10:57 19 -> /tmp/datadir/mysqlbin.000004
lrwx------. 1 dan dan 64 Mar 2 10:57 20 -> /tmp/datadir/mysql/host.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 21 -> /tmp/datadir/mysql/host.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 22 -> /tmp/datadir/mysql/user.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 23 -> /tmp/datadir/mysql/user.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 24 -> /tmp/datadir/mysql/db.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 25 -> /tmp/datadir/mysql/db.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 26 -> /tmp/datadir/mysql/proxies_priv.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 27 -> /tmp/datadir/mysql/proxies_priv.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 28 -> /tmp/datadir/mysql/tables_priv.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 29 -> /tmp/datadir/mysql/tables_priv.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 30 -> /tmp/datadir/mysql/columns_priv.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 31 -> /tmp/datadir/mysql/columns_priv.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 32 -> /tmp/datadir/mysql/procs_priv.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 33 -> /tmp/datadir/mysql/procs_priv.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 34 -> /tmp/datadir/mysql/servers.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 35 -> /tmp/datadir/mysql/servers.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 36 -> /tmp/datadir/mysql/event.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 37 -> /tmp/datadir/mysql/event.MYI
O_CLOEXEC files are those with flags 02000000
/usr/include/bits/fcntl-linux.h:# define __O_CLOEXEC 02000000
/usr/include/bits/fcntl-linux.h:# define O_CLOEXEC __O_CLOEXEC /* Set close_on_exec. */
$ find fdinfo/ -type f -ls -exec cat {} \; | more
1924720 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/0
pos: 0
flags: 0100002
mnt_id: 25
1924721 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/1
pos: 9954
flags: 0102001
mnt_id: 82
1924722 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/2
pos: 10951
flags: 0102001
mnt_id: 82
1924723 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/3
pos: 116
flags: 02100002
mnt_id: 82
1924724 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/4
pos: 52
flags: 0100002
mnt_id: 82
lock: 1: POSIX ADVISORY WRITE 26056 00:2c:1866365 0 EOF
1924725 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/5
pos: 0
flags: 0100000
mnt_id: 82
1924726 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/6
pos: 16384
flags: 0100002
mnt_id: 82
1924727 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/7
pos: 0
flags: 02100002
mnt_id: 82
lock: 1: POSIX ADVISORY WRITE 26056 00:2c:1866491 0 EOF
1924728 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/8
pos: 0
flags: 0100002
mnt_id: 82
1924729 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/9
pos: 0
flags: 0100002
mnt_id: 82
1924730 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/10
pos: 0
flags: 0100002
mnt_id: 82
1924731 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/11
pos: 0
flags: 0100002
mnt_id: 82
1924732 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/12
pos: 0
flags: 02100002
mnt_id: 82
lock: 1: POSIX ADVISORY WRITE 26056 00:2c:1866492 0 EOF
1924733 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/13
pos: 0
flags: 02100002
mnt_id: 82
lock: 1: POSIX ADVISORY WRITE 26056 00:2c:1866493 0 EOF
1924734 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/14
pos: 763
flags: 02102001
mnt_id: 82
1924735 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/15
pos: 0
flags: 0100002
mnt_id: 82
1924736 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/16
pos: 473
flags: 02102001
mnt_id: 82
1924737 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/17
pos: 0
flags: 02000002
mnt_id: 9
1924738 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/18
pos: 0
flags: 02
mnt_id: 9
1924739 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/19
pos: 245
flags: 02100001
mnt_id: 82
1924740 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/20
pos: 0
flags: 0100002
mnt_id: 82
1924741 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/21
pos: 503
flags: 0500002
mnt_id: 82
1924742 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/22
pos: 324
flags: 0100002
mnt_id: 82
1924743 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/23
pos: 642
flags: 0500002
mnt_id: 82
1924744 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/24
pos: 880
flags: 0100002
mnt_id: 82
1924745 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/25
pos: 581
flags: 0500002
mnt_id: 82
1924746 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/26
pos: 1386
flags: 0100002
mnt_id: 82
1924747 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/27
pos: 498
flags: 0500002
mnt_id: 82
1924748 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/28
pos: 0
flags: 0100002
mnt_id: 82
1924749 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/29
pos: 513
flags: 0500002
mnt_id: 82
1924750 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/30
pos: 0
flags: 0100002
mnt_id: 82
1924751 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/31
pos: 494
flags: 0500002
mnt_id: 82
1924752 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/32
pos: 0
flags: 0100002
mnt_id: 82
1924753 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/33
pos: 535
flags: 0500002
mnt_id: 82
1924754 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/34
pos: 0
flags: 0100002
mnt_id: 82
1924755 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/35
pos: 396
flags: 0500002
mnt_id: 82
1924756 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/36
pos: 0
flags: 0100002
mnt_id: 82
1924757 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/37
pos: 517
flags: 0500002
mnt_id: 82
