Commit graph

176454 commits

Author SHA1 Message Date
Sergei Golubchik
943c62a5d4 Backport 4bb49d84a9, correct handling on defaults[-extra]-file is SST scripts
(from 10.1 to 10.0-galera)

This conflicted signficantly with 7d550c76be
which added --defaults-group-suffix support.

Took the approach of 4bb49d84a9 and adapted the
--defaults-group-suffix handling to be consistent.

The following changes as follows:

SST scripts now use $MY_PRINT_DEFAULTS rather than the lowercase for
consistency and this include all required --default arguements.

Backport/merge by Daniel Black <daniel@linux.vnet.ibm.com>
2018-01-14 22:48:02 +11:00
Oleksandr Byelkin
5fe1d7d076 MDEV-14526: MariaDB keeps crashing under load when query_cache_type is changed
The problem was in such scenario:
T1 - starts registering query and locked QC
T2 - starts disabling QC and wait for UNLOCK
T1 - unlock QC
T2 - disable QC and destroy signals without waiting for query unlock
T1 a) - not yet unlocked query in qc and crash on attempt to unlock because
        QC signals are destroyed
   b) if above was done before destruction, it execute end_of results first
      time at exit on after try_lock which see QC disables and return TRUE.
      But it do not reset query_cache_tls->first_query_block which lead to
      second call of end_of_result when diagnostic arena has already
      inappropriate status (not is_eof()).

Fix is:
  1) wait for all queries unlocked before destroying them by locking and
     unlocking
  2) remove query_cache_tls->first_query_block if QC disabled
2018-01-14 10:49:56 +01:00
Sergey Vojtovich
b75d767689 Fixed mysql_install_db --no-defaults
Regression after 5ea2801.
2018-01-13 13:05:09 +04:00
Igor Babaev
abc123391f Fixed mdev-6706 Wrong result (missing rows)
with joins, SQ, ORDER BY, semijoin=on

A bug in get_sort_by_table() could mislead the function
setup_semijoin_dups_elimination(). As a result the optimizer
could produce invalid execution plans for queries with ORDER BY
and subquery predicates that could be converted to semi-joins.
2018-01-12 13:14:27 -08:00
Oleksandr Byelkin
6293e3bbcf MDEV-14743: Server crashes in Item_func_match::init_search
Remove non prepared (and so belonging to removed clauses FT functions) from the list.

in later version it will be fixed by building the list during preparation.
2018-01-12 20:00:45 +01:00
Daniel Black
5ea28015d5 mysql_install_db: Use --defaults-group-suffix if specified
Signed-off-by: Daniel Black <daniel@linux.vnet.ibm.com>
2018-01-12 12:09:06 +04:00
Oleksandr Byelkin
a5285a8fb7 Fixed misleading voariable names. 2018-01-11 17:24:29 +01:00
Oleksandr Byelkin
abb9e703d2 MDEV-14690: Assertion `page_link == &fake_link' failed in pagecache_write_part
Fix the call to correspond protocoll of pagecache call.
Fix of misleading variables names.
2018-01-11 17:24:29 +01:00
Monty
1f18bd630a MDEV-8200 aria bug with insert select and lock tables
This bug happens when locking the same Aria "transactional" table
(page format) more then once with LOCK TABLES and inserting into one
of them with INSERT ... SELECT when the table is empty.

Fixed by ensuring we don't use fast bulk insert if table is opened
twice with LOCK TABLES (as this changes table->s->state)

Code changes:
- Added use_count to MARIA_USED_TABLES to be able to check if
  table is opened twice for a statement/lock table
- Don't clear history or reset info->start_state if we
  don't have versioning. One reason for the bug was
  was that info->start_state was set to point to different
  states for the two tables.  If there is no versioning
  info->start_state should always point to info->s->state.common.

Other things:
- Fixed also some typos that was noticed while scanning the code
- More DBUG_PRINT
2018-01-11 16:39:29 +02:00
Marko Mäkelä
578ffcc5ef Skip mariabackup.huge_lsn if encryption is not available 2018-01-11 10:56:13 +02:00
Marko Mäkelä
c15b3d2d41 Merge 10.0 into 10.1 2018-01-11 10:44:05 +02:00
Marko Mäkelä
4c1479545d Merge 5.5 into 10.0 2018-01-11 10:16:52 +02:00
Marko Mäkelä
bdcd7f79e4 MDEV-14916 InnoDB reports warning for "Purge reached the head of the history list"
The warning was originally added in
commit c67663054a
(MySQL 4.1.12, 5.0.3) to trace claimed undo log corruption that
was analyzed in https://lists.mysql.com/mysql/176250
on November 9, 2004.

Originally, the limit was 20,000 undo log headers or transactions,
but in commit 9d6d1902e0
in MySQL 5.5.11 it was increased to 2,000,000.

The message can be triggered when the progress of purge is prevented
by a long-running transaction (or just an idle transaction whose
read view was started a long time ago), by running many transactions
that UPDATE or DELETE some records, then starting another transaction
with a read view, and finally by executing more than 2,000,000
transactions that UPDATE or DELETE records in InnoDB tables. Finally,
when the oldest long-running transaction is completed, purge would
run up to the next-oldest transaction, and there would still be more
than 2,000,000 transactions to purge.

Because the message can be triggered when the database is obviously
not corrupted, it should be removed. Heavy users of InnoDB should be
monitoring the "History list length" in SHOW ENGINE INNODB STATUS;
there is no need to spam the error log.
2018-01-11 09:55:10 +02:00
Oleksandr Byelkin
9c9cf556a1 MDEV-13933: Wrong results in COUNT() query with EXISTS and exists_to_in
Roll back to most general duplicate removing strategi in case of different stratagies for one position.
2018-01-10 16:58:04 +01:00
Marko Mäkelä
a9c55c0059 MDEV-13814 Extra logging when innodb_log_archive=ON
Backport the fix from 10.0.33 to 5.5, in case someone compiles XtraDB
with -DUNIV_LOG_ARCHIVE
2018-01-10 10:21:52 +02:00
Marko Mäkelä
a408e881cf MDEV-14174 crash on start with innodb-track-changed-pages
The XtraDB option innodb_track_changed_pages causes
the function log_group_read_log_seg() to be invoked
even when recv_sys==NULL, leading to the SIGSEGV.

This regression was caused by
MDEV-11027 InnoDB log recovery is too noisy
2018-01-10 09:27:01 +02:00
Jan Lindström
07aa985979 MDEV-14776: InnoDB Monitor output generated by specific error is flooding error logs
innodb/buf_LRU_get_free_block
	Add debug instrumentation to produce error message about
	no free pages. Print error message only once and do not
	enable innodb monitor.

xtradb/buf_LRU_get_free_block
	Add debug instrumentation to produce error message about
	no free pages. Print error message only once and do not
	enable innodb monitor. Remove code that does not seem to
	be used.

innodb-lru-force-no-free-page.test
	New test case to force produce desired error message.
2018-01-09 12:48:31 +02:00
Marko Mäkelä
18ccbf014a Make the MDEV-14874 test case more robust 2018-01-08 17:09:21 +02:00
Marko Mäkelä
29b6e809a9 Merge 10.0 into 10.1 2018-01-08 14:51:20 +02:00
Marko Mäkelä
c903ba2f1e MDEV-13205 InnoDB: Failing assertion: !dict_index_is_online_ddl(index) upon ALTER TABLE
dict_foreign_find_index(): Ignore incompletely created indexes.
After a failed ADD UNIQUE INDEX, an incompletely created index
could be left behind until the next ALTER TABLE statement.
2018-01-08 14:26:55 +02:00
Marko Mäkelä
16d308e21d MDEV-14874 innodb_encrypt_log corrupts the log when the LSN crosses 32-bit boundary
This bug affects both writing and reading encrypted redo log in
MariaDB 10.1, starting from version 10.1.3 which added support for
innodb_encrypt_log. That is, InnoDB crash recovery and Mariabackup
will sometimes fail when innodb_encrypt_log is used.

MariaDB 10.2 or Mariabackup 10.2 or later versions are not affected.

log_block_get_start_lsn(): Remove. This function would cause trouble if
a log segment that is being read is crossing a 32-bit boundary of the LSN,
because this function does not allow the most significant 32 bits of the
LSN to change.

log_blocks_crypt(), log_encrypt_before_write(), log_decrypt_after_read():
Add the parameter "lsn" for the start LSN of the block.

log_blocks_encrypt(): Remove (unused function).
2018-01-08 09:44:40 +02:00
Vladislav Vaintroub
3a22d6c136 Fix conf_to_src build.
2cd3169113 broke conf_to_src,
because strings library is now dependend on mysys (my_alloc etc are used
now directly in string lib)

Fix by adding appropriate dependency.

Also exclude conf_to_src from VS IDE builds. EXCLUDE_FROM_ALL
is not enough for that.
2018-01-05 18:22:57 +00:00
Aleksey Midenkov
d9e0c06b5d Tests: detect table count for some encryption tests
debug_key_management
encrypt_and_grep
innodb_encryption

If real table count is different from what is expected by the test, it
just hangs on waiting to fulfill hardcoded number. And then exits with
**failed** after 10 minutes of wait: quite unfriendly and hard to
figure out what's going on.
2018-01-05 10:31:11 +02:00
Marko Mäkelä
21470de148 Merge 10.0 into 10.1 2018-01-04 20:42:29 +02:00
Marko Mäkelä
4496fd71f4 Fix a truncation warning introduced in MDEV-12323 2018-01-04 20:39:00 +02:00
Marko Mäkelä
1e89c86dd7 Merge 10.0 into 10.1 2018-01-03 20:41:34 +02:00
Marko Mäkelä
8ac1982fcc Merge 5.5 into 10.0 2018-01-03 20:40:41 +02:00
Marko Mäkelä
84c9c8b2e9 Silence some -Wimplicit-fallthrough by proper spelling 2018-01-03 15:01:17 +02:00
Marko Mäkelä
016caa3d20 Merge 10.0 into 10.1 2018-01-02 21:57:22 +02:00
Marko Mäkelä
51e4650ed0 Merge 5.5 into 10.0 2018-01-02 21:52:46 +02:00
Marko Mäkelä
20fab71b14 Follow-up to MDEV-14799: Remove bogus debug assertions
trx_undo_rec_get_partial_row(): When the PRIMARY KEY includes a
column prefix of an externally stored column, the already parsed
part of the undo log record may contain a reference to
an off-page column. This is the case in the bug58912 test in
innodb.innodb.
2018-01-02 21:41:39 +02:00
Marko Mäkelä
d384ead0f0 MDEV-14799 After UPDATE of indexed columns, old values will not be purged from secondary indexes
This is a regression caused by MDEV-14051 'Undo log record is too big.'

Purge in the secondary index is wrongly skipped in
row_purge_upd_exist_or_extern() because node->row only does not contain all
indexed columns.

trx_undo_rec_get_partial_row(): Add the parameter for node->update
so that the updated columns will be copied from the initial part
of the undo log record.
2018-01-02 19:11:10 +02:00
Ian Gilfillan
eef2bc5a5c Update mysqladmin man page 2017-12-28 19:44:42 +04:00
Sergei Golubchik
7e4c185c77 MDEV-14272 Mariadb crashes with signal 11 when using federatedx engine and galera
cherry-pick e6ce97a592
2017-12-28 02:13:25 +01:00
Sergei Golubchik
1300627a5d MDEV-14309 MTR tests require perl-Env which is not always in the default installation
* don't use Env module in tests, use $ENV{xxx} instead
* collateral changes:
** $file in the error message was unset
** $file in the other error message was unset too :)
** source file arguments are conventionally upper-cased
** abort the test (die) on error, don't just echo/exit
2017-12-27 23:17:28 +01:00
Vicențiu Ciorbaru
d1c2cd30b7 Merge remote-tracking branch '10.0' into 10.1 2017-12-27 17:50:39 +02:00
Oleksandr Byelkin
462808f3b6 MDEV-10657: incorrect result returned with binary protocol (prepared statements)
If translation table present when we materialize the derived table then
change it to point to the materialized table.

Added debug info to see really what happens with what derived.
2017-12-27 16:01:37 +01:00
Sergei Golubchik
5377242fff MDEV-14026 ALTER TABLE ... DELAY_KEY_WRITE=1 creates table copy for partitioned MyISAM table with DATA DIRECTORY/INDEX DIRECTORY options
set data_file_name and index_file_name in HA_CREATE_INFO
before calling check_if_incompatible_data()
2017-12-25 15:18:21 +01:00
Sergei Golubchik
6d8b1bd620 cleanup: ha_partition::update_create_info 2017-12-25 15:18:21 +01:00
Sergei Golubchik
c881d82c93 cleanup: ha_myisam::data_file_name and index_file_name
don't allocate them on THD::mem_root on every init(HA_STATUS_CONST) call,
do it once in open() (because they don't change) on TABLE::mem_root
(so they stay valid until the table is closed)
2017-12-25 15:18:21 +01:00
Sachin Setiya
2fe6186124 MDEV-10715 Galera: Replicate MariaDB GTID to other nodes in the cluster
Problem:- Gtid are not transferred in Galera Cluster.

Solution:- We need to transfer gtid in the case on either when cluster is
slave/master in async replication. In normal Gtid replication gtid are generated on
recieving node itself and it is always on sync with other nodes. Because galera keeps
node in sync , So all nodes get same no of event groups. So the issue arises when
say galera is slave in async replication.
A
|    (Async replication)
D <-> E <-> F  {Galera replication}
So what should happen is that all node should apply the master gtid but this does
node happen, becuase node E, F does not recieve gtid from D in write set , So what E(or F)
does is that it applies wsrep_gtid_domain_id, D server-id , E gtid next seq no. This
generated gtid does not always work when say A has different domain id.

So In this commit, on galera node when we see that this event is recieved from master
we simply write Gtid_Log_Event in write_set and send it to other nodes.
2017-12-25 13:57:42 +05:30
Daniel Bartholomew
db3bdca7c2 bump the VERSION 2017-12-22 16:45:20 -05:00
Sergey Vojtovich
4b8cd4536a MDEV-13626 Merge InnoDB test cases from MySQL 5.7
Coverage for temporary tables modifications in read-only transactions.
Introduced in 5.7 by 325cdf426
2017-12-22 14:03:25 +04:00
Vicențiu Ciorbaru
24efee9100 Follow up to MDEV-12366: FLUSH privileges can break hierarchy of roles
A suggestion to make role propagation simpler from serg@mariadb.org.

Instead of gathering the leaf roles in an array, which for very wide
graphs could potentially mean a big part of the whole roles schema, keep
the previous logic. When finally merging a role, set its counter
to something positive.

This will effectively mean that a role has been merged, thus a random pass
through roles hash that touches a previously merged role won't cause the problem
described in MDEV-12366 any more, as propagate_role_grants_action will stop
attempting to merge from that role.
2017-12-21 18:10:00 +02:00
Marko Mäkelä
461cf3e5a3 Make a test more robust
Sometimes, the test would fail with a result difference for
the READ UNCOMMITTED read, because the incremental backup
would finish before redo log was written for all the rows
that were inserted in the second batch.

To fix that, cause a redo log write by creating another
transaction. The transaction rollback (which internally does commit)
will be flushed to the redo log, and before that, all the preceding
changes will be flushed to the redo log as well.
2017-12-21 17:40:01 +02:00
Marko Mäkelä
9d7c0882fa Merge 10.0 into 10.1 2017-12-21 17:25:51 +02:00
Marko Mäkelä
0202e47274 MDEV-12827 Assertion failure when reporting duplicate key error in online table rebuild
row_log_table_apply_insert_low(), row_log_table_apply_update():
When reporting the error_key_num, only count the clustered index
if it corresponds to a key in the SQL layer.

The assertion failure was probably introduced by the (incomplete)
MySQL 5.6.28 bug fix
Bug #21364096 THE BOGUS DUPLICATE KEY ERROR IN ONLINE DDL
WITH INCORRECT KEY NAME
which we are improving.

Side note: the fix was incorrectly merged to MySQL 5.7.10;
incorrect key names will continue to be reported in MySQL 5.7.
2017-12-21 17:19:13 +02:00
Marko Mäkelä
1cf28964f5 MDEV-6247 post-fix: Re-enable some debug assertions
These assertions were disabled in MariaDB 10.1.1 in
commit df4dd593f2
with a bogus comment referring to the function wsrep_fake_trx_id()
that was introduced in the very same commit.
2017-12-21 10:19:49 +02:00
Elena Stepanova
5f896b3604 Updated list of unstable tests for 10.1.30 release 2017-12-21 03:31:39 +02:00
Sachin Setiya
2a4faa8ab6 MDEV-13478 Full SST sync fails because of the error in the cleaning part
Problem:
 The command was:
    find $paths -mindepth 1 -regex $cpat -prune -o -exec rm -rf {} \+
 Which was supposed to work as
    * skipping $paths directories themselves (-mindepth 1)
    * see if the dir/file name matches $cpat (-regex)
    * if yes - don't dive into the directory, skip it (-prune)
    * otherwise (-o)
    * remove it and everything inside (-exec)
 Now -exec ... \+ works like this:
    every new found path is appended to the end of the command line.
    when accumulated command line length reaches `getconf ARG_MAX` (~2Gb)
    it's executed, and find continues, appending to a new command line.

 What happens here, find appends some directory to the command line,
 then dives into it, and starts appending files from that directory.
 At some point command line overflows, rm -rf gets executed and removes
 the whole directory. Now find tries to continue scanning the directory
 that was already removed.

Fix: don't dive into directories that will be recursively removed
anyway, use -prune for them. Basically, we should be pruning both paths
that have matched $cpat and paths that have not matched it. This is
achived by pruning unconditionally, before the regex is tested:
    find $paths -mindepth 1 -prune -regex $cpat -o -exec rm -rf {} \+

Patch Credit:- Serg
2017-12-20 18:51:30 +01:00