An addition to fix for MDEV-5205, fixes server crash on shutdown.
Thread groups are destroyed asynchronously, that is kill server
thread sends shutdown request to all thread groups without waiting
for compeltion.
It means all_groups array must not be freed until all thread groups
are down. This patch suggests that all_groups is freed when last
thread group is destroyed.
Note 1: threadpool code doesn't surround atomic ops with atomic locks,
thus no locks for shutdown_group_count.
Note 2: this patch preserves old behaviour, but we may need to wait
until all thread groups are down before returning from tp_end().
modified:
storage/connect/filamtxt.h
storage/connect/filamzip.h
storage/connect/myconn.h
storage/connect/plgdbsem.h
storage/connect/taboccur.h
storage/connect/tabutil.h
storage/connect/tabxcl.h
- Add the possibility to execute several commands in one query of
an EXECSRC tables (using ...where command in (cmd list);)
modified:
storage/connect/ha_connect.cc
storage/connect/odbconn.cpp
storage/connect/odbconn.h
storage/connect/tabmysql.cpp
storage/connect/tabmysql.h
storage/connect/tabodbc.cpp
storage/connect/tabodbc.h
storage/connect/tabtbl.cpp
storage/connect/tabwmi.cpp
storage/connect/xtable.h
- Enhance retrieving column definitions in discovery:
From SRCDEF adding LIMIT 0 to the executed query
Testing if type, length, and precision are compatible
Making the distinction between CHAR and VARCHAR
modified:
storage/connect/ha_connect.cc
storage/connect/myconn.cpp
storage/connect/mysql-test/connect/r/mysql.result
storage/connect/mysql-test/connect/r/odbc_sqlite3.result
storage/connect/mysql-test/connect/r/odbc_sqlite3_grant.result
storage/connect/myutil.h
storage/connect/myutil.h
storage/connect/odbconn.cpp
storage/connect/value.h
MDEV-5217: Incorrect MyISAM event execution order causing incorrect parallel replication
In parallel replication, if transactions A,B group-commit together on the
master, we can execute them in parallel on a replication slave. But then, if
transaction C follows on the master, on the slave, we need to be sure that
both A and B have completed before starting on C to be sure to avoid
conflicts.
The necessary wait is implemented such that B waits for A to commit before it
commits itself (thus preserving commit order). And C waits for B to commit
before it itself can start executing. This way C does not start until both A
and B have completed.
The wait for B's commit on A happens inside the commit processing. However, in
the case of MyISAM with no binlog enabled on the slave, it appears that no
commit processing takes place (since MyISAM is non-transactional), and thus
the wait of B for A was not done. This allowed C to start before A, which can
lead to conflicts and incorrect replication.
Fixed by doing an extra wait for A at the end of B before signalling C.
MDEV-5217: Unlock of de-allocated mutex
There was a race in the code for wait_for_commit::wakeup().
Since the waiter does a dirty read of the waiting_for_commit
flag, it was possible for the waiter to complete and deallocate
the wait_for_commit object while the waitee was still running
inside wakeup(). This would cause the waitee to access invalid
memory.
Fixed by putting an extra lock/unlock in the destructor for
wait_for_commit, to ensure that waitee has finished with the
object before it is deallocated.
MDEV-5217: Incorrect event pos update leading to corruption of reading of events from relay log
The rli->event_relay_log_pos was sometimes undated incorrectly when using
parallel replication, especially around relay log rotates. This could cause
the SQL thread to seek into an invalid position in the relay log, resulting in
errors about invalid events or even random corruption in some cases.
include/CMakeLists.txt:
Added handler_ername.h
include/handler_ername.h:
Names of handler error messages
mysql-test/extra/binlog_tests/binlog.test:
Test using handler error names
mysql-test/r/mysqltest.result:
Update for new error message
mysql-test/t/auto_increment.test:
Test using handler error names
mysql-test/t/auto_increment_ranges.inc:
Test using handler error names
mysql-test/t/replace.test:
Test using handler error names
MDEV-5217: Last_sql_error lost in parallel replication.
For some reason, the query execution code in log_event.cc call
rli->clear_error for each event (part of clear_all_errors()).
This causes a problem in parallel replication, where the
execution in one worker thread could clear the error set by
another thread, causing the SQL thread to stop but leaving no
error visible in SHOW SLAVE STATUS.
There seems to be no reason to clear the global error code
in Relay_log_info for each event execution, from code review
and from running the test suite. So remove this clearing of
the error code to make things work also in the parallel case.
MDEV-5217: SQL thread hangs during stop if error occurs in the middle of an event group
Normally, when we stop the slave SQL thread in parallel replication, we want
the worker threads to continue processing events until the end of the current
event group. But if we stop due to an error that prevents further events from
being queued, such as an error reading the relay log, no more events can be
queued for the workers, so they have to abort even if they are in the middle
of an event group. There was a bug that we would deadlock, the workers
waiting for more events to be queued for the event group, the SQL thread
stopped and waiting for the workers to complete their current event group
before exiting.
Fixed by now signalling from the SQL thread to all workers when it is about
to exit, and cleaning up in all workers when so signalled.
This patch fixes one of multiple problems reported in MDEV-5217.
- thread_pool_size command line option upper limit increased to 100 000
(same as for max_connections)
- thread_pool_size system variable upper limit is maximum of 128 or
the value given at command line
- thread groups are now allocated dynamically
Different limit for command line option and system variable was done to
avoid additional mutex for all_groups and threadpool_max_size.
"
A maximum size of 10 Mb works in 5.1 because the initial
required size of ibdata1 was less than 10M. But in 5.5, a
change was made to allocate all 128 rollback segments at
bootstrap. Since then, the initial size has been 10M + the
default autoextend size of 8M.
In 5.6, worklog 6216 changes the autoextend size from 8M to
64M. This changes the initial size of ibdata1 from 18M in
5.5 and earlier releases of 5.6 to 74M in the current
mysql-5.6 and mysql-trunk. So this change is especially
needed in 5.6.
"
12M is enough to avoid autoextending during bootstrap
extend table names discovery (ha_discover_table_names() and Discovered_table_list) to return
or optionally filter out temporary tables ("#sql..."). SHOW commands and I_S tables
typically want temp table filtered out, while DROP DATABASE wants to see them too.
additonally, remove the supression for the warning "Invalid (old?) table or database name"
from mtr, and add it to .test files as needed (we need to test that this warning
does *not* happen in drop.test)
Updated --help text to declare --slave-parallel-threads as an alpha feature
mysql-test/r/mysqld--help.result:
Updated --help text
sql/slave.cc:
Added missing trans_retries++ that caused rpl_deadlock_innodb.test to fail.
This is safe as this part is never run in parallel.
sql/sql_base.cc:
Fixed temporary table handling (part of merge)
sql/sys_vars.cc:
Updated --help text to declare --slave-parallel-threads as an alpha feature
The merge is still missing a few hunks related to temporary tables and
InnoDB log file size. The associated code did not seem to exist in
10.0, so the merge of that needs more work. Until this is fixed, there
are a number of test failures as a result.
For compatibility purposes let InnoDB use DATA_INT for MYSQL_TYPE_ENUM and MYSQL_TYPE_SET.
Silence the warning for these types and let the index translation table to be built anyway.
Test case by Jeremy Cole.
In parallel replication, there are two kinds of events which are
executed in different ways.
Normal events that are part of event groups/transactions are executed
asynchroneously by being queued for a worker thread.
Other events like format description and rotate and such are executed
directly in the driver SQL thread.
If the direct execution of the other events were to update the old-style
position, then the position gets updated too far ahead, before the normal
events that have been queued for a worker thread have been executed. So
this patch adds some special cases to prevent such position updates ahead
of time, and instead queues dummy events for the worker threads, so that
they will at an appropriate time do the position updates instead.
(Also fix a race in a test case that happened to trigger while running
tests for this patch).
which makes it possible to add more world language collations
with very complex collation rules (e.g. Myanmar):
- Weight string for a single character in a user defined collation
was erroneously limited to 7 weights (instead of 8 weights).
Added an extra element in the user-defined weight arrays,
to fit 8 non-zero weights.
- Weight string limit for contractions was made two times longer (16 weights),
which allows longer contractions without affecting the performance
of filesort.
- A user-defined collation now refuses to initialize and reports an error
in case if a weight string gets longer than 8 weights for a single character,
or longer than 16 weights for a contraction. Previously weight strings
for such characters (and contractions) were cut, so a collation
could silently start with wrong rules.
- Fixed a bug in handling rules like "&a << b" in combination with
shift-after-method="expand". The primary weight for "b" was not
correctly calculated, which erroneously made "b" primary greater than "a"
instead of primary equal to "a".
Fix a couple of issues in MDEV-4506, Parallel replication:
- Missing mysql_cond_signal(), which could cause hangs.
- Fix incorrect update of old-style replication position.
- Change assertion to error handling (can trigger on manipulated/
corrupt binlog).
There was 2 problems:
1) coping/moving of the same type (usually casting) as sizeof() (solved in different ways depends on the cause);
2) using 'const' in SSL_CTX::getVerifyCallback() which return object (not reference) and so copy of the object will be created and 'const' has no sens.
