Commit graph

177 commits

Author SHA1 Message Date
Jan Lindström
e6a50e41da MDEV-20051: Add new mode to wsrep_OSU_method in which Galera checks storage engine of the effected table
Introduced a new wsrep_strict_ddl configuration variable in which
Galera checks storage engine of the effected table. If table is not
InnoDB (only storage engine currently fully supporting Galera
replication) DDL-statement will return error code:

ER_GALERA_REPLICATION_NOT_SUPPORTED
       eng "DDL-statement is forbidden as table storage engine does not support Galera replication"

However, when wsrep_replicate_myisam=ON we allow DDL-statements to
MyISAM tables. If effected table is allowed storage engine Galera
will run normal TOI.

This new setting should be for now set globally on all
nodes in a cluster. When this setting is set following DDL-clauses
accessing tables not supporting Galera replication are refused:

* CREATE TABLE (e.g. CREATE TABLE t1(a int) engine=Aria
* ALTER TABLE
* TRUNCATE TABLE
* CREATE VIEW
* CREATE TRIGGER
* CREATE INDEX
* DROP INDEX
* RENAME TABLE
* DROP TABLE

Statements on PROCEDURE, EVENT, FUNCTION are allowed as effected
tables are known only at execution. Furthermore, USER, ROLE, SERVER,
DATABASE statements are also allowed as they do not really have
effected table.
2020-02-11 15:17:50 +02:00
mkaruza
d0c8316bf5
Incorrect behaviour of WSREP_SYNC_WAIT_UPTO_GTID (#1442)
Function `signal_waiters` assigned `m_committed_seqno` variable outside of
mutex lock which caused incorrect behavior of WSREP_SYNC_WAIT_UPTO_GTID.
Fixed by moving assignment inside lock. Added handling of OOM and now
error is reported.
Remove hard-coded seqno value and read seqno directly from current node state.
2020-02-05 10:02:33 +02:00
mkaruza
41bc736871 Galera GTID support
Support for galera GTID consistency thru cluster. All nodes in cluster
should have same GTID for replicated events which are originating from cluster.
Cluster originating commands need to contain sequential WSREP GTID seqno
Ignore manual setting of gtid_seq_no=X.

In master-slave scenario where master is non galera node replicated GTID is
replicated and is preserved in all nodes.

To have this - domain_id, server_id and seqnos should be same on all nodes.
Node which bootstraps the cluster, to achieve this, sends domain_id and
server_id to other nodes and this combination is used to write GTID for events
that are replicated inside cluster.

Cluster nodes that are executing non replicated events are going to have different
GTID than replicated ones, difference will be visible in domain part of gtid.

With wsrep_gtid_domain_id you can set domain_id for WSREP cluster.

Functions WSREP_LAST_WRITTEN_GTID, WSREP_LAST_SEEN_GTID and
WSREP_SYNC_WAIT_UPTO_GTID now works with "native" GTID format.

Fixed galera tests to reflect this chances.

Add variable to manually update WSREP GTID seqno in cluster

Add variable to manipulate and change WSREP GTID seqno. Next command
originating from cluster and on same thread will have set seqno and
cluster should change their internal counter to it's value.
Behavior is same as using @@gtid_seq_no for non WSREP transaction.
2020-01-29 15:06:06 +02:00
Marko Mäkelä
a983b24407 Merge 10.4 into 10.5 2020-01-28 14:17:09 +02:00
Jan Lindström
8a931e4d16 MDEV-17571 : Make systemd timeout behavior more compatible with long Galera SSTs
This is 10.4 version.

Idea is to create monitor thread for both donor and joiner that will
periodically if needed extend systemd timeout while SST is being
processed. In 10.4 actual SST is executed by running SST script
and exchanging messages on pipe using blocking fgets. This fix
starts monitoring thread before SST script is started and
we stop monitoring thread when SST has been completed.
2020-01-22 16:55:59 +02:00
Marko Mäkelä
780d2bb8a7 Merge 10.4 into 10.5 2019-09-06 14:25:20 +03:00
Sergei Golubchik
e21408b799 Decrease stack space usage of mysql_execute_command()
- WSREP_DEBUG called WSREP_LOG which allocated a 1K variable on stack
  for each macro usage (at least gcc can't reuse the stack space).
  Move this concatenation to happen compile-time, not run-time.
2019-08-23 22:06:30 +02:00
Jan Lindström
7b4de10477 MDEV-20378: Galera uses uninitialized memory
Problem was that wsrep thread argument was deleted on wrong
place. Furthermore, scan method incorrectly used unsafe c_ptr().
Finally, fixed wsrep thread initialization to correctly set
up thread_id and pass correct argument to functions and
fix signess problem causing compiler errors.
2019-08-20 10:32:04 +03:00
Marko Mäkelä
1d15a28e52 Merge 10.3 into 10.4 2019-08-14 18:06:51 +03:00
Marko Mäkelä
65d48b4a7b Merge 10.2 to 10.3 2019-08-13 19:28:51 +03:00
Jan Lindström
5edc4ea4d9 MDEV-20324: Galera threads are not registered to performance schema
Galera threads were not registered to performance schema and
used pthread_create when mysql_thread_create should have been
used.

Added test case to verify current galera performance schema
instrumentation does work.
2019-08-13 12:52:01 +03:00
Marko Mäkelä
e9c1701e11 Merge 10.3 into 10.4 2019-07-25 18:42:06 +03:00
Eugene Kosov
0f83c8878d Merge 10.2 into 10.3 2019-07-16 18:39:21 +03:00
Jan Lindström
ec49976e38 MDEV-19746: Galera test failures because of wsrep_slave_threads identification
Problem was that tests select INFORMATION_SCHEMA.PROCESSLIST processes
from user system user and empty state. Thus, there is not clear
state for slave threads.

Changes:
- Added new status variables that store current amount of applier threads
(wsrep_applier_thread_count) and rollbacker threads
(wsrep_rollbacker_thread_count). This will make clear how many slave threads
of certain type there is.
- Added THD state "wsrep applier idle" when applier slave thread is
waiting for work. This makes finding slave/applier threads easier.
- Added force-restart option for mtr to always restart servers between tests
to avoid race on start of the test
- Added wait_condition_with_debug to wait until the passed statement returns
true, or the operation times out. If operation times out, the additional error
statement will be executed

Changes to be committed:
	new file:   mysql-test/include/force_restart.inc
	new file:   mysql-test/include/wait_condition_with_debug.inc
	modified:   mysql-test/mysql-test-run.pl
	modified:   mysql-test/suite/galera/disabled.def
	modified:   mysql-test/suite/galera/r/MW-336.result
	modified:   mysql-test/suite/galera/r/galera_kill_applier.result
	modified:   mysql-test/suite/galera/r/galera_var_slave_threads.result
	new file:   mysql-test/suite/galera/t/MW-336.cnf
	modified:   mysql-test/suite/galera/t/MW-336.test
	modified:   mysql-test/suite/galera/t/galera_kill_applier.test
	modified:   mysql-test/suite/galera/t/galera_parallel_autoinc_largetrx.test
	modified:   mysql-test/suite/galera/t/galera_parallel_autoinc_manytrx.test
	modified:   mysql-test/suite/galera/t/galera_var_slave_threads.test
	modified:   mysql-test/suite/wsrep/disabled.def
	modified:   mysql-test/suite/wsrep/r/variables.result
	modified:   mysql-test/suite/wsrep/t/variables.test
	modified:   sql/mysqld.cc
	modified:   sql/wsrep_mysqld.cc
	modified:   sql/wsrep_mysqld.h
	modified:   sql/wsrep_thd.cc
	modified:   sql/wsrep_var.cc
2019-07-15 10:17:07 +03:00
Oleksandr Byelkin
c07325f932 Merge branch '10.3' into 10.4 2019-05-19 20:55:37 +02:00
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Vicențiu Ciorbaru
cb248f8806 Merge branch '5.5' into 10.1 2019-05-11 22:19:05 +03:00
Marko Mäkelä
5f31f8cc87 Fix cmake -DWITH_WSREP=OFF
Provide a dummy definition of WSREP_NNULL.
This was broken in commit b896f60a73.
2019-04-03 10:56:55 +03:00
Teemu Ollakka
1ef50a34ec 10.4 wsrep group commit fixes (#1224)
* MDEV-16509 Improve wsrep commit performance with binlog disabled

Release commit order critical section early after trx_commit_low() if
binlog is not transaction coordinator. In order to avoid two phase commit,
binlog_hton is not registered for THD during IO_CACHE population.

Implemented a test which verifies that the transactions release
commit order early.

This optimization will change behavior during recovery as the commit
is not two phase when binlog is off. Fixed and recorded wsrep-recover-v25
and wsrep-recover to match the behavior.

* MDEV-18730 Ordering for wsrep binlog group commit

Previously out of order execution was allowed for wsrep commits.
Established proper ordering by populating wait_for_commit
for every wsrep THD and making group commit leader to wait for
prior commits before proceeding to trx_group_commit_leader().

* MDEV-18730 Added a test case to verify correct commit ordering

* MDEV-16509, MDEV-18730 Review fixes

Use WSREP_EMULATE_BINLOG() macro to decide if the binlog_hton
should be registered. Whitespace/syntax fixes and cleanups.

* MDEV-16509 Require binlog for galera_var_innodb_disallow_writes test

If the commit to InnoDB is done in one phase, the native InnoDB behavior
is that the transaction is committed in memory before it is persisted to
disk. This means that the innodb_disallow_writes=ON may not prevent
transaction to become visible to other readers before commit is completely
over. On the other hand, if the commit is two phase (as it is with binlog),
the transaction will be blocked in prepare phase.

Fixed the test to use binlog, which enforces two phase commit, which
in turn makes commit to block before the changes become visible to
other connections. This guarantees that the test produces expected
result.
2019-03-15 07:09:13 +02:00
Sergey Vojtovich
88d89ee0ba Less abort_loop references
Removed redundant initialisation in unireg_init(): already done by
mysql_init_variables().

Slave threads already check THD::killed, which eliminates the need to
check abort_loop.

Removed unused wsrep_kill_mysql().
2019-03-09 20:22:24 +04:00
seppo
785092ee23 LOCK_thread_count and COND_thread_count removed from wsrep modules (#1197)
Refactored wsrep patch to not use LOCK_thread_count and COND_thread_count anymore.
This has partially been replaced by using old LOCK_wsrep_slave_threads mutex.
For slave thread count change waiting, new COND_wsrep_slave_threads signal has been added

Added LOCK_wsrep_cluster_config mutex to control that cluster address change cannot happen in parallel

Protected wsrep_slave_threads variable changes with LOCK_cluster_config mutex
This is for avoiding concurrent slave thread count and cluster joining operations to happen

Fixes according to Teemu's review
2019-02-26 13:39:05 -05:00
Oleksandr Byelkin
93ac7ae70f Merge branch '10.3' into 10.4 2019-02-21 14:40:52 +01:00
Jan Lindström
c568e25379
Merge pull request #1185 from codership/10.4-wsrep_schema_cleanup
Cleanup wsrep_schema and remove all references to wsrep_thd_pool
2019-02-14 14:30:13 +02:00
Daniele Sciascia
047754a728 Cleanup wsrep_schema and remove all references to wsrep_thd_pool
* Removed all references related to wsrep_thd_pool (which was removed)

* Removed unused declarations in wsrep_schema.h

* The following would result invalid reads in
  Wsrep_schema::replay_transaction():
  ```
  frag_table->field[4]->val_str(&buf);

  Wsrep_schema_impl::end_index_scan(frag_table);
  Wsrep_schema_impl::finish_stmt(thd);
  ret= wsrep_apply_events(thd, rli, buf.c_ptr_safe(), buf.length());
  ```

  because `buf` was accessed after closing the table. The fix is to
  perform storage reads using a different THD.

* In Wsrep_schema::recover_sr_transactions(), cluster_table was opened
  for write, however it is only read here. And frag_table was opened
  for read, wereas write is potentially needed.
  Also, avoid copy caused by String::c_ptr() to zero terminate the c
  string, use c_ptr_quick instead.
2019-02-14 09:55:14 +01:00
mkaruza
3e64e7f24c WSREP debug log levels support
Global variable wsrep_debug now can be used to filter wsrep-lib messages based on debug level provided.
Type of wsrep_debug is now set to be unsigned int, so tests and configuration files changed accordingly.
2019-02-13 18:47:27 +01:00
Teemu Ollakka
f06a0b5338 Implement wsrep_load_data_splitting with streaming replication
If wsrep_load_data_splitting is configured, change streaming replication
parameters internally to match the original behavior, i.e. replicate
on every 10000 rows. After load data is over, restore original
streaming replication settings.

Removed redundant wsrep_tc_log_commit().
2019-02-12 10:37:05 +02:00
Oleksandr Byelkin
65c5ef9b49 dirty merge 2019-02-07 13:59:31 +01:00
Marko Mäkelä
081fd8bfa2 Merge 10.1 into 10.2 2019-02-02 11:40:02 +02:00
Oleksandr Byelkin
560799ebd8 Merge branch '10.0-galera' into 10.1 2019-01-31 09:34:34 +01:00
Brave Galera Crew
36a2a185fe Galera4 2019-01-23 15:30:00 +04:00
Marko Mäkelä
1afed20774 Merge 10.2 into 10.3
The test galera_sst_mariabackup_table_options was disabled,
because the server refuses to start up due to wrong parameters.
2018-11-26 14:05:15 +02:00
Jan Lindström
244cc35e7b MDEV-17801: Galera test failure on galera_var_reject_queries
Problem was that controlling connection i.e. connection that
executed the query SET GLOBAL wsrep_reject_queries = ALL_KILL;
was also killed but server would try to send result from that
query to controlling connection resulting a assertion
mysqld: /home/jan/mysql/10.2-sst/include/mysql/psi/mysql_socket.h:738: inline_mysql_socket_send: Assertion `mysql_socket.fd != -1' failed.
as socket was closed when controlling connection was closed.

wsrep_close_client_connections()
	Do not close controlling connection and instead of
	wsrep_close_thread() we do now soft kill by THD::awake

wsrep_reject_queries_update()
	Call wsrep_close_client_connections using current thd.
2018-11-26 08:11:56 +02:00
Jan Lindström
dde0a83fff MDEV-17801: Galera test failure on galera_var_reject_queries
Problem was that controlling connection i.e. connection that
executed the query SET GLOBAL wsrep_reject_queries = ALL_KILL;
was also killed but server would try to send result from that
query to controlling connection resulting a assertion
mysqld: /home/jan/mysql/10.2-sst/include/mysql/psi/mysql_socket.h:738: inline_mysql_socket_send: Assertion `mysql_socket.fd != -1' failed.
as socket was closed when controlling connection was closed.

wsrep_close_client_connections()
	Do not close controlling connection and instead of
	wsrep_close_thread() we do now soft kill by THD::awake

wsrep_reject_queries_update()
	Call wsrep_close_client_connections using current thd.
2018-11-22 16:36:16 +02:00
Jan Lindström
b0fe082b36 Merge remote-tracking branch 'origin/5.5-galera' into 10.0-galera 2018-10-30 13:22:52 +02:00
Vasil Dimov
27dcef3900 Add a new config variable wsrep_certification_rules
This is used for controlling whether to use a new/optimized
certification rules or the old/classic ones that could cause more
certification failures - when foreign keys are used and two INSERTs are
done concurrently to the child table from different nodes.

(cherry picked from commit 815d73e6af8daace6262ab63ca6c043ffc4204b3)
2018-10-10 13:14:36 +03:00
Sergei Golubchik
57e0da50bb Merge branch '10.2' into 10.3 2018-09-28 16:37:06 +02:00
Oleksandr Byelkin
28f08d3753 Merge branch '10.1' into 10.2 2018-09-14 08:47:22 +02:00
Sergei Golubchik
db947b7599 Merge branch '10.0-galera' into 10.1 2018-09-07 15:25:27 +02:00
Marko Mäkelä
2f4c391958 Merge 10.2 into 10.3 2018-09-06 22:35:45 +03:00
Sergei Golubchik
a6246cab16 fix failures of innodb_plugin tests in --embedded
Post-fix for 7e8ed15b95

Also, apply the same innodb fix to xtradb.
2018-09-04 09:19:50 +02:00
Marko Mäkelä
206528f722 Merge 10.1 into 10.2 2018-08-31 15:10:02 +03:00
Marko Mäkelä
3b5d3cd68e Revert MDEV-9519 due to regressions
This reverts commit 75dfd4acb9.
2018-08-31 12:36:31 +03:00
Marko Mäkelä
7830fb7f45 Merge 10.2 into 10.3 2018-08-28 12:22:56 +03:00
Jan Lindström
b87b8c1344
Merge pull request #828 from tempesta-tech/sysprg/10.1-MDEV-10754
MDEV-10754 wsrep_sst_rsync does not support innodb_data_home_dir
2018-08-21 15:58:09 +03:00
Marko Mäkelä
9258097fa3 Merge 10.1 into 10.2 2018-08-21 15:20:34 +03:00
Julius Goryavsky
75dfd4acb9 This is patch for the https://jira.mariadb.org/browse/MDEV-9519 issue:
If we have a 2+ node cluster which is replicating from an async master
and the binlog_format is set to STATEMENT and multi-row inserts are executed
on a table with an auto_increment column such that values are automatically
generated by MySQL, then the server node generates wrong auto_increment
values, which are different from what was generated on the async master.

The causes and fixes:

1. We need to improve processing of changing the auto-increment values
after changing the cluster size.

2. If wsrep auto_increment_control switched on during operation of
the node, then we should immediately update the auto_increment_increment
and auto_increment_offset global variables, without waiting of the next
invocation of the wsrep_view_handler_cb() callback. In the current version
these variables retain its initial values if wsrep_auto_increment_control
is switched on during operation of the node, which leads to inconsistent
results on the different nodes in some scenarios.

3. If wsrep auto_increment_control switched off during operation of the node,
then we must return the original values of the auto_increment_increment and
auto_increment_offset global variables, as the user has set. To make this
possible, we need to add a "shadow copies" of these variables (which stores
the latest values set by the user).
2018-08-15 14:17:28 +03:00
Julius Goryavsky
7e8ed15b95 Fixes after review and correction of the problems caused by the fact that during the SST innodb plugin is not yet initialized, as well as problems with running tests from the root user (not directly related to the MDEV-10754). 2018-08-09 02:24:12 +00:00
Jan Lindström
9b29bda0d6 Merge remote-tracking branch 'origin/5.5-galera' into 10.0-galera 2018-08-02 13:13:21 +03:00
Daniele Sciascia
4d2b552369 Fix FK constraint violation in applier, after ALTER TABLE ADD FK
Adding a FK constraint to an existing table (ALTER TABLE ADD FOREIGN
KEY) causes the applier to fail, if a concurrent DML statement that
violate the new constraint (i.e. a DELETE or UPDATE of record in the
parent table).

For exmaple, the following scenario causes a crash in the applier:

1. ALTER successfully adds FK constraint in node_1
2. On node_2 is UPDATE is in pre_commit() and has certified successfully
3. ALTER is delivered in node_2 and BF aborts DML
4. Applying UPDATE event causes FK violation in node_1

To avoid this situation it is necessary for UPDATE to fail during
certification. And for the UPDATE to fail certfication it is necessary
that ALTER appends certification keys for both the child and the parent
table. Before this patch, ALTER TABLE ADD FK only appended keys for
child table which is ALTERed.
2018-08-02 09:03:27 +03:00