Commit graph

179544 commits

Author SHA1 Message Date
Daniel Black
c73b4b1698 wsrep_sst_mysqldump: enforce a minimum version only 2018-01-17 20:16:59 +11:00
Sergei Golubchik
b80fa4000d bug: ha_heap was unilaterally increasing reclength
MEMORY engine needs the record length to be at least sizeof(void*),
because it stores a pointer there (linking deleted records into a list).
So when the reclength is less than sizeof(void*), it's set to sizeof(void*).
That is done inside heap_create(), and the upper layer doesn't know
that the engine writes beyond share->reclength.

While it's usually safe (in-memory record size is rounded up to
sizeof(double), so even if share->reclength is too small,
share->rec_buff_len is not), it could cause problems in the code that
copies records and expects them to fix in share->reclength,
e.g. in partitioning.
2018-01-16 23:31:29 +01:00
Sergei Golubchik
444587d8a3 BIT field woes
* get_rec_bits() was always reading two bytes, even if the
  bit field contained only of one byte
* In various places the code used field->pack_length() bytes
  starting from field->ptr, while it should be field->pack_length_in_rec()
* Field_bit::key_cmp and Field_bit::cmp_max passed field_length as
  an argument to memcmp(), but field_length is the number of bits!
2018-01-16 23:29:48 +01:00
Sergei Golubchik
5e7593add4 add support for ASAN instrumentation 2018-01-16 22:57:28 +01:00
Sergei Golubchik
6634f460a9 fix compilation with ASAN
if the property is not found, set it to the empty string,
otherwise it'll show as libmysql_link_flags-NOTFOUND on the linker
command line, and the linker won't like it.

Also, don't specify LINK_FLAG_NO_UNDEFINED twice, MERGE_LIBRARIES
already put it into LINK_FLAGS.
2018-01-16 22:29:20 +01:00
Marko Mäkelä
f44017384a MDEV-14968 On upgrade, InnoDB reports "started; log sequence number 0"
srv_prepare_to_delete_redo_log_files(): Initialize srv_start_lsn.
2018-01-16 20:02:38 +02:00
Alexander Barkov
81378b3947 Moving a change_list related methods from THD to Item_change_list
1. Moving the following methods from THD to Item_change_list:
   nocheck_register_item_tree_change()
   check_and_register_item_tree_change()
   rollback_item_tree_changes()
   as they work only with the "change_list" member and don't
   require anything else from THD.
2. Deriving THD from Item_change_list

This change will help to fix "MDEV-14603 signal 11 with short stacktrace" easier.
2018-01-16 16:18:50 +04:00
Marko Mäkelä
be85c2dc88 Mariabackup --prepare: Do not access transactions or data dictionary
innobase_start_or_create_for_mysql(): Only start the data dictionary
and transaction subsystems in normal server startup and during
mariabackup --export.
2018-01-16 13:57:30 +02:00
Marko Mäkelä
33ecf8345d Follow-up fix to MDEV-14441: Fix a potential race condition
btr_cur_update_in_place(): Read block->index only once,
so that it cannot change to NULL after the first read.
When block->index != NULL, it must be equal to index.
2018-01-16 13:55:45 +02:00
Marko Mäkelä
f5e158183c Follow-up fix to MDEV-14441: Correct a misplaced condition
btr_cur_update_in_place(): The call rw_lock_x_lock(ahi_latch) must
of course be inside the if (ahi_latch) condition. This is a mistake
that I made when backporting the fix-under-development from 10.3.
2018-01-16 07:50:15 +02:00
Daniel Black
c4b7074e72 wsrep_sst_xtrabackup*: use mysqld defaults arguments
It was assumed in
https://bugs.launchpad.net/codership-mysql/+bug/1378355 that xtrabackup
didn't have the groups-defaults-suffix because it wasn't listed in the
help output.

Also --defaults-group was always just an additional group despite the
misleading description implying it was a replacement.

So actually we can use the default* mysqld arguements to xtrabackup as
the standard mysqld configuration parser was used.
2018-01-16 14:44:39 +11:00
Igor Babaev
6267be460a Fixed mdev-14911: zero_date is considered as NULL, depending on
optimizer_switch

For DATE and DATETIME columns defined as NOT NULL,
"date_notnull IS NULL" has to be modified to:
"date_notnull IS NULL OR date_notnull == 0"
if date_notnull is from an inner table of outer join);
"date_notnull == 0" - otherwise.

This must hold for such columns of mergeable views and derived
tables as well. So far the code did the above re-writing only
for columns of base tables and temporary tables.
2018-01-15 11:19:14 -08:00
Sergei Petrunia
0292cd0a27 Better explanation why rpl_row_triggers is disabled. 2018-01-15 21:08:00 +03:00
Marko Mäkelä
3fdd390791 MDEV-14441 InnoDB hangs when setting innodb_adaptive_hash_index=OFF during UPDATE
This race condition is a regression caused by MDEV-12121.

btr_cur_update_in_place(): Determine block->index!=NULL only once
in order to determine whether an adaptive hash index bucket needs
to be exclusively locked and unlocked.

If we evaluated block->index multiple times, and the adaptive hash
index was disabled before we locked the adaptive hash index, then
we would never release the adaptive hash index bucket latch, which
would eventually lead to InnoDB hanging.
2018-01-15 19:02:38 +02:00
Sergei Golubchik
d8001106c9 MDEV-14469 build with cmake -DMYSQL_MAINTAINER_MODE=ON fails: 'readdir_r' is deprecated
1. test readdir_r() availability under -Werror
2. don't protect readdir() with mutexes, it's not needed for the way
   we use readdir()
2018-01-15 15:57:04 +01:00
Sergei Golubchik
d31ee64da6 compiler warning 2018-01-15 15:57:04 +01:00
Sergei Golubchik
d7b84f9413 compiler warning
gcc 6 issues a warning about a suspicious construct

  while(0);
  {
     some code
  }
2018-01-15 15:57:03 +01:00
Marko Mäkelä
9c6fc7b644 Fix -Wsign-compare introduced by Compilation speed (#546) 2018-01-15 16:38:16 +02:00
Marko Mäkelä
27b6b2625e Merge 10.1 into 10.2 2018-01-15 16:22:35 +02:00
Marko Mäkelä
4794e5b091 Fix a test that always failed on --embedded 2018-01-15 16:19:46 +02:00
Sergei Petrunia
85aea5a12b Update .result for rocksdb.rpl_row_triggers (not the whole test works yet) 2018-01-15 16:50:18 +03:00
Marko Mäkelä
850702da6b MDEV-13626: Merge InnoDB test cases from MySQL 5.7 (part 6)
innodb.truncate_inject: Replacement for innodb_zip.wl6501_error_1

Note: unlike MySQL, in some cases TRUNCATE does not return
an error in MariaDB. This should be fixed in the scope of
MDEV-13564 or similar.
2018-01-15 15:40:28 +02:00
Marko Mäkelä
ec062c6181 MDEV-12121 follow-up: Unbreak the WITH_INNODB_AHI=OFF build 2018-01-15 15:40:28 +02:00
Sergey Vojtovich
7e3c1e02b7 MDEV-14796 - debian: insecure root password is only if plugin is empty
Enumerate plugins that use password field.
2018-01-15 16:21:45 +04:00
Daniel Black
1879b2b8df debian: insecure root password is only if plugin is empty 2018-01-15 16:19:53 +04:00
Alexander Barkov
88a9b23396 MDEV-14609 XA Transction unable to ROLLBACK TO SAVEPOINT
The function trans_rollback_to_savepoint(), unlike trans_savepoint(),
did not allow xa_state=XA_ACTIVE, so an attempt to do ROLLBCK TO SAVEPOINT
inside an XA transaction incorrectly returned an error
"...command cannot be executed ... in the ACTIVE state...".

Partially merging a MySQL patch:
  7fb5c47390311d9b1b5367f97cb8fedd4102dd05
  This is WL#7193 (Decouple THD and st_transactions)...

The currently merged part includes these changes:
- Introducing st_xid_state::check_has_uncommitted_xa()
- Reusing it in both trans_rollback_to_savepoint() and trans_savepoint(),
  so now both allow XA_ACTIVE.
2018-01-15 13:50:28 +04:00
Eugene Kosov
72136ae75c Compilation speed (#546)
Speed up compilation

Standard C++ headers contribute a lot to compilation time. Avoid algorithm
and sstream in frequently used headers.
2018-01-14 20:50:45 +04:00
Daniel Black
3f27fa3797 mtr: minor (and incomplete) fixes for suite galera_3node 2018-01-14 23:31:14 +11:00
Daniel Black
a2a038152e wsrep_sst_xtrabackup*: use wsrep_sst_common parsed vars 2018-01-14 23:31:14 +11:00
Daniel Black
51ea696c8e wsrep_sst_common: keep WSREP_SST_OPT_HOST_UNESCAPED for IPv4
change wsrep_sst_mysqldump.sh to use WSREP_SST_OPT_HOST_UNESCAPED
directly.
2018-01-14 23:31:14 +11:00
Daniel Black
722df90534 wsrep_sst_xtrabackup*: read all sections of config not nust mysqld
--mysqld parsed to parse_cnf aka my_print_defaults will parse all
sections which the server read and not just mysqld.
2018-01-14 23:31:13 +11:00
Daniel Black
cc8abb21e3 wsrep_sst_xtrabackup*: du -s removed lessens output
and only displays the summary. As this is the only bit
that is used there will be less du,kernel,awk CPU usage.
2018-01-14 23:31:13 +11:00
Daniel Black
e78e308e81 wsrep_sst_common: parse --address and split WSREP_SST_OPT_PATH
Use the string functionality of sh to split the address:

IPv6 test:
. scripts/wsrep_sst_common.sh --address [ipv6]:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2
$ set | grep WSREP_SST
WSREP_SST_OPT_ADDR='[ipv6]:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2'
WSREP_SST_OPT_AUTH=
WSREP_SST_OPT_BINLOG=
WSREP_SST_OPT_BYPASS=0
WSREP_SST_OPT_CONF='  '
WSREP_SST_OPT_DATA=
WSREP_SST_OPT_DEFAULT=
WSREP_SST_OPT_EXTRA_DEFAULT=
WSREP_SST_OPT_HOST='[ipv6]'
WSREP_SST_OPT_HOST_UNESCAPED=ipv6
WSREP_SST_OPT_LSN=lsn_version
WSREP_SST_OPT_MODULE=module
WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
WSREP_SST_OPT_PORT=port
WSREP_SST_OPT_PSWD=
WSREP_SST_OPT_SST_VER=sst_ver
WSREP_SST_OPT_SUFFIX_DEFAULT=
WSREP_SST_OPT_USER=

IPv4 test:
. scripts/wsrep_sst_common.sh --address ipv4:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2
$ set | grep WSREP_SST
WSREP_SST_OPT_ADDR=ipv4:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2
WSREP_SST_OPT_AUTH=
WSREP_SST_OPT_BINLOG=
WSREP_SST_OPT_BYPASS=0
WSREP_SST_OPT_CONF='  '
WSREP_SST_OPT_DATA=
WSREP_SST_OPT_DEFAULT=
WSREP_SST_OPT_EXTRA_DEFAULT=
WSREP_SST_OPT_HOST=ipv4
WSREP_SST_OPT_LSN=lsn_version
WSREP_SST_OPT_MODULE=module
WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
WSREP_SST_OPT_PORT=port
WSREP_SST_OPT_PSWD=
WSREP_SST_OPT_SST_VER=sst_ver
WSREP_SST_OPT_SUFFIX_DEFAULT=
WSREP_SST_OPT_USER=
2018-01-14 23:31:13 +11:00
Daniel Black
95e5fe6732 wsrep_sst_common: parse_cnf - use awk rather than grep/cut/tail excessiveness
Test cases:

f($var) = awk -v var="${var}"  \
     'BEGIN { OFS=FS="="}
     { gsub(/_/,"-",$1); if ( $1=="--"var ) lastval=substr($0,length($1)+2) }
     END { print lastval }'

Missing input is blank:

$ echo '--var_aa=something' | f(var-b-not-ther)
(blank as expected)

All RHS of = is unmunged:

$ echo '--var_aa=password==_-$' | f(var-aa)
password==_-$

Mixed - and _ in var name:
$ echo '--var_aa-bb_cc=1' | f(var-aa-bb-cc)
1

No value returns blank line:
$ echo '--var_aa-bb_cc' | f(var-aa-bb-cc)
(blank line as expected)

Multiples return the last:
$  echo -e "--bb=cc\n--bb=dd" | f(bb)
dd
2018-01-14 23:31:01 +11:00
Sergei Golubchik
943c62a5d4 Backport 4bb49d84a9, correct handling on defaults[-extra]-file is SST scripts
(from 10.1 to 10.0-galera)

This conflicted signficantly with 7d550c76be
which added --defaults-group-suffix support.

Took the approach of 4bb49d84a9 and adapted the
--defaults-group-suffix handling to be consistent.

The following changes as follows:

SST scripts now use $MY_PRINT_DEFAULTS rather than the lowercase for
consistency and this include all required --default arguements.

Backport/merge by Daniel Black <daniel@linux.vnet.ibm.com>
2018-01-14 22:48:02 +11:00
Oleksandr Byelkin
5fe1d7d076 MDEV-14526: MariaDB keeps crashing under load when query_cache_type is changed
The problem was in such scenario:
T1 - starts registering query and locked QC
T2 - starts disabling QC and wait for UNLOCK
T1 - unlock QC
T2 - disable QC and destroy signals without waiting for query unlock
T1 a) - not yet unlocked query in qc and crash on attempt to unlock because
        QC signals are destroyed
   b) if above was done before destruction, it execute end_of results first
      time at exit on after try_lock which see QC disables and return TRUE.
      But it do not reset query_cache_tls->first_query_block which lead to
      second call of end_of_result when diagnostic arena has already
      inappropriate status (not is_eof()).

Fix is:
  1) wait for all queries unlocked before destroying them by locking and
     unlocking
  2) remove query_cache_tls->first_query_block if QC disabled
2018-01-14 10:49:56 +01:00
Marko Mäkelä
fc65577873 MDEV-14887 On a 32-bit system, MariaDB 10.2 mishandles data file sizes exceeding 4GiB
This is a regression that was introduced in MySQL 5.7.6 in
19855664de

fil_node_open_file(): Use proper 64-bit arithmetics for truncating
size_bytes to a multiple of a file extent size.
2018-01-13 18:15:04 +02:00
Monty
09ef28abd7 Fixed BUILD scripts
- Removed extra set -x -v used for debugging
- Fixed that that gcc version tests works for gcc 7
2018-01-13 16:38:43 +02:00
Sergey Vojtovich
b75d767689 Fixed mysql_install_db --no-defaults
Regression after 5ea2801.
2018-01-13 13:05:09 +04:00
Sergei Petrunia
1eea7966f3 Merge branch 'bb-10.2-mariarocks' into 10.2 2018-01-13 01:27:35 +03:00
Sergei Petrunia
4cafd8e66f rocksdb.information_schema testcase is not stable 2018-01-13 01:26:06 +03:00
Igor Babaev
abc123391f Fixed mdev-6706 Wrong result (missing rows)
with joins, SQ, ORDER BY, semijoin=on

A bug in get_sort_by_table() could mislead the function
setup_semijoin_dups_elimination(). As a result the optimizer
could produce invalid execution plans for queries with ORDER BY
and subquery predicates that could be converted to semi-joins.
2018-01-12 13:14:27 -08:00
Oleksandr Byelkin
6293e3bbcf MDEV-14743: Server crashes in Item_func_match::init_search
Remove non prepared (and so belonging to removed clauses FT functions) from the list.

in later version it will be fixed by building the list during preparation.
2018-01-12 20:00:45 +01:00
Sergei Petrunia
2da1917912 Attempt to eliminate race conditions in rocksdb.information_schema 2018-01-12 16:04:29 +00:00
Varun Gupta
028e2ddc54 Added a missing result file to the rocksdb_sys_vars result suite 2018-01-12 19:16:36 +05:30
Sergei Petrunia
c481fc9ca7 Change MyRocks maturity from Alpha to Beta 2018-01-12 15:58:34 +03:00
Sergei Petrunia
d32f5be307 MDEV-14372: Fix and enable rocksdb.information_schema test
- Make Rdb_binlog_manager::unpack_value to not have a stack overrun
  when it is reading invalid data (which it currently does as we in
  MariaDB do not store binlog coordinates under BINLOG_INFO_INDEX_NUMBER,
  see comments in MDEV-14892 for details).
- We may need to store these coordinates in the future, so instead of
  removing the call of this function, let's make it work properly for
  all possible inputs.
2018-01-12 15:58:34 +03:00
Daniel Black
5ea28015d5 mysql_install_db: Use --defaults-group-suffix if specified
Signed-off-by: Daniel Black <daniel@linux.vnet.ibm.com>
2018-01-12 12:09:06 +04:00
Oleksandr Byelkin
a5285a8fb7 Fixed misleading voariable names. 2018-01-11 17:24:29 +01:00
Oleksandr Byelkin
abb9e703d2 MDEV-14690: Assertion `page_link == &fake_link' failed in pagecache_write_part
Fix the call to correspond protocoll of pagecache call.
Fix of misleading variables names.
2018-01-11 17:24:29 +01:00