Don't double-check privileges for a column in the GROUP BY that refers to
the same column in SELECT clause. Privileges were already checked for SELECT clause.
Added MAX_STATEMENT_TIME user variable to automaticly kill queries after a given time limit has expired.
- Added timer functions based on pthread_cond_timedwait
- Added kill_handlerton() to signal storage engines about kill/timeout
- Added support for GRANT ... MAX_STATEMENT_TIME=#
- Copy max_statement_time to current user, if stored in mysql.user
- Added status variable max_statement_time_exceeded
- Added KILL_TIMEOUT
- Removed digest hash from performance schema tests as they change all the time.
- Updated test results that changed because of the new user variables or new fields in mysql.user
This functionallity is inspired by work done by Davi Arnaut at twitter.
Test case is copied from Davi's work.
Documentation can be found at
https://kb.askmonty.org/en/how-to-limittimeout-queries/
mysql-test/r/mysqld--help.result:
Updated for new help message
mysql-test/suite/perfschema/r/all_instances.result:
Added new mutex
mysql-test/suite/sys_vars/r/max_statement_time_basic.result:
Added testing of max_statement_time
mysql-test/suite/sys_vars/t/max_statement_time_basic.test:
Added testing of max_statement_time
mysql-test/t/max_statement_time.test:
Added testing of max_statement_time
mysys/CMakeLists.txt:
Added thr_timer
mysys/my_init.c:
mysys/mysys_priv.h:
Added new mutex and condition variables
Added new mutex and condition variables
mysys/thr_timer.c:
Added timer functions based on pthread_cond_timedwait()
This can be compiled with HAVE_TIMER_CREATE to benchmark agains timer_create()/timer_settime()
sql/lex.h:
Added MAX_STATEMENT_TIME
sql/log_event.cc:
Safety fix (timeout should be threated as an interrupted query)
sql/mysqld.cc:
Added support for timers
Added status variable max_statement_time_exceeded
sql/share/errmsg-utf8.txt:
Added ER_QUERY_TIMEOUT
sql/signal_handler.cc:
Added support for KILL_TIMEOUT
sql/sql_acl.cc:
Added support for GRANT ... MAX_STATEMENT_TIME=#
Copy max_statement_time to current user
sql/sql_class.cc:
Added timer functionality to THD.
Added thd_kill_timeout()
sql/sql_class.h:
Added timer functionality to THD.
Added KILL_TIMEOUT
Added max_statement_time variable in similar manner as long_query_time was done.
sql/sql_connect.cc:
Added handling of max_statement_time_exceeded
sql/sql_parse.cc:
Added starting and stopping timers for queries.
sql/sql_show.cc:
Added max_statement_time_exceeded for user/connects status in MariaDB 10.0
sql/sql_yacc.yy:
Added support for GRANT ... MAX_STATEMENT_TIME=# syntax, to be enabled in 10.0
sql/structs.h:
Added max_statement_time user resource
sql/sys_vars.cc:
Added max_statement_time variables
mysql-test/suite/roles/create_and_drop_role_invalid_user_table.test
Removed test as we require all fields in mysql.user table.
scripts/mysql_system_tables.sql
scripts/mysql_system_tables_data.sql
scripts/mysql_system_tables_fix.sql
Updated mysql.user with new max_statement_time field
- Fix the crash by making get_column_range_cardinality()
to handle the special case where Column_stats objects
is an all-zeros object (the question of what is the point
of having Field::read_stats point to such object remains a
mystery)
- Added a few comments. Learning the code still.
- if test_if_skip_sort_order() decides to switch to using an index, or
switch from using ref to using quick select, it should set all
members accordingly.
merge from MySQL-5.6, revision:
revno: 3677.2.1
committer: Alfranio Correia <alfranio.correia@oracle.com>
timestamp: Tue 2012-02-28 16:26:37 +0000
message:
BUG#13627921 - MISSING FLAGS IN SQL_COMMAND_FLAGS MAY LEAD TO REPLICATION PROBLEMS
Flags in sql_command_flags[command] are not correctly set for the following
commands:
. SQLCOM_SET_OPTION is missing CF_CAN_GENERATE_ROW_EVENTS;
. SQLCOM_BINLOG_BASE64_EVENT is missing CF_CAN_GENERATE_ROW_EVENTS;
. SQLCOM_REVOKE_ALL is missing CF_CHANGES_DATA;
. SQLCOM_CREATE_FUNCTION is missing CF_AUTO_COMMIT_TRANS;
This may lead to a wrong sequence of events in the binary log. To fix
the problem, we correctly set the flags in sql_command_flags[command].
The problem was that the internal temporary table created
for information_schema overflow to MyISAM because it has
a row width of > 3000 characters, which filled the in memory
temporary tables.
Fix was to increase size for the heap table.
Added comments
Ensure that tokudb test works even if jemalloc is not installed
Removed not referenced function Item::remove_fixed()
mysql-test/suite/rpl/t/rpl_gtid_reconnect.test:
Fixed race condition
sql/item.cc:
Indentation fix
sql/item.h:
Removed not used function
Added comment
sql/sql_select.cc:
Fixed indentation
storage/tokudb/mysql-test/rpl/include/have_tokudb.opt:
Ensure that tokudb test works even if jemalloc is not installed
storage/tokudb/mysql-test/tokudb/suite.opt:
Ensure that tokudb test works even if jemalloc is not installed
storage/tokudb/mysql-test/tokudb_add_index/suite.opt:
Ensure that tokudb test works even if jemalloc is not installed
storage/tokudb/mysql-test/tokudb_alter_table/suite.opt:
Ensure that tokudb test works even if jemalloc is not installed
storage/tokudb/mysql-test/tokudb_bugs/suite.opt:
Ensure that tokudb test works even if jemalloc is not installed
storage/tokudb/mysql-test/tokudb_mariadb/suite.opt:
Ensure that tokudb test works even if jemalloc is not installed
don't kill statements in the default connection, kill them in a connection that
will be closed - it'll guarantee that `KILL con_id` will not apply to unrelated statements.
We should assume that the store engine will report the first duplicate key for this case.
Old code of suppression of unsafe logging error with LIMIT didn't work, because of wrong usage of my_interval_timer().
Suppress unsafe logging errors to the error log if we get too many unsafe logging errors in a short time.
This is to not overflow the error log with meaningless errors.
- Each error code is suppressed and counted separately.
- We do a 5 minute suppression of new errors if we get more than 10 errors in that time.
Only print unsafe logging errors if log_warnings > 1.
mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
Update test results as INSERT ... ON DUPLICATE KEY UPDATE doesn't get logged anymore
mysql-test/suite/binlog/r/binlog_unsafe.result:
Update test results as INSERT ... ON DUPLICATE KEY UPDATE doesn't get logged anymore
mysql-test/suite/engines/README:
Fixed typos
mysql-test/suite/rpl/r/rpl_known_bugs_detection.result:
Update test results as INSERT ... ON DUPLICATE KEY UPDATE doesn't get logged anymore
sql/sql_base.cc:
Don't log warning if there are two unique keys used with INSERT .. ON DUPLICATE KEY UPDATE.
We should assume that the store engine will report the first duplicate key for this case.
sql/sql_class.cc:
Suppress error in binary log if we get too many unsafe logging errors in a short time.
Only print unsafe logging errors if log_warnings > 1
have 0x5C as the second byte in a multi-byte character.
- Adding big5 tests covering an unassigned character 0xC840
being stored into char/varchar/text/enum columns.
Fixed the bug itself.
Also, added "SET NAMES swe7" which was forgotten in the previous commit,
so latin1 was actually tested lati1 instead of swe7 in a mistake.
Now it tests swe7.
in the SQL parser.
Various backslash escapes and quote-quote escaped sequences are covered
in combination with single and multi-byte characters.
This is especially important for the character sets that can have 0x5C
as the second byte in a multi-byte character (big5, cp932, gbk, sjis).
swe7 is also a special character set, because in swe7 0x5C is used for
both escape character and for "LATIN CAPITAL LETTER O WITH DIAERESIS".
Fix test_if_skip_sort_order() logic:
WHEN
we use index X, which doesn't produce needed ordering, but there is
an index Y which does and has the same prefix as used prefix of X
THEN
don't just switch to using ref access on Y. If range(Y) would use more
key parts, use range(Y).
The problem was that my_hash_sort didn't properly delete end-space characters properly, so strings that should compare
identically was seen as different strings. (Space was handled correctly, but not NBSP)
This caused duplicate key errors when a heap table was converted to Aria as part of overflow in group by.
Fixed by removing all characters that compares as end space when creating a hash.
Other things:
- Fixed that --sorted_results also works for errors in mysqltest.
- Speed up hash by not comparing strings that has different hash.
- Speed up many my_hash_sort functions by using registers to calculate hash instead of pointers.
This was previously done for some functions, but not for all.
- Made a macro of the hash function, to simplify code and to be able to experiment with new hash functions.
client/mysqltest.cc:
Fixed that --sorted_results also works for error messages.
mysql-test/r/ctype_partitions.result:
New test to ensure that partitions on hash works
mysql-test/suite/multi_source/gtid.result:
Updated result
mysql-test/suite/multi_source/gtid.test:
Test that --sorted_result works for error messages
mysql-test/suite/multi_source/gtid_ignore_duplicates.result:
Updated result
mysql-test/suite/multi_source/gtid_ignore_duplicates.test:
Updated result
mysql-test/suite/multi_source/load_data.result:
Updated result
mysql-test/suite/multi_source/load_data.test:
Updated result
mysql-test/t/ctype_partitions.test:
New test to ensure that partitions on hash works
storage/heap/hp_write.c:
Speed up hash by not comparing strings that has different hash.
storage/maria/ma_check.c:
Extra debug
strings/ctype-bin.c:
Use macro for hash function
strings/ctype-latin1.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-mb.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-simple.c:
Use macro for hash function
Use same variable names as in other my_hash_sort functions.
Update my_hash_sort_simple() to properly remove end space (patch by Bar)
strings/ctype-uca.c:
Ignore duplicated space inside strings and end space in my_hash_sort_uca(). This fixed MDEV-6255
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-ucs2.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-utf8.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/strings_def.h:
Made a macro of the hash function, to simplify code and to be able to experiment with new hash functions.
Avoided exponential recursive calls of JOIN_CACHE::join_records() in the case
of non-nested outer joins.
A different solution is required to resolve this performance problem for
nested outer joins.
MDEV-6560 Assertion `! is_set() ' failed in Diagnostics_area::set_ok_status on killing CREATE OR REPLACE
MDEV-6525 Assertion `table->pos_in_locked _tables == __null || table->pos_in_locked_tables->table = table' failed in mark_used_tables_as_free_for_reuse, locking problems and binlogging problems on CREATE OR REPLACE under lock.
mysql-test/r/create_or_replace.result:
Added test for MDEV-6560
mysql-test/t/create_or_replace.test:
Added test for MDEV-6560
mysql-test/valgrind.supp:
Added suppression for OpenSuse 12.3
sql/sql_base.cc:
More DBUG
sql/sql_class.cc:
Changed that thd_sqlcom_can_generate_row_events() does not report that CREATE OR REPLACE is generating row events.
This is safe as this function is only used by InnoDB/XtraDB to check if a query is generating row events as part of another transaction. As CREATE is always run as it's own transaction, this isn't a problem.
This fixed MDEV-6525.
sql/sql_table.cc:
Remember if reopen_tables() generates an error (which can only happen in case of KILL).
This fixed MDEV-6560
the bug was introduced by CREATE OR REPLACE implementation.
CREATE IF NOT EXISTS ... SELECT was returning an error status to the caller,
while sending an ok packet to the user. SP code was not prepared for that
and trusted that error status means an error.
The Item_string constructors called set_name() on the source string,
which was wrong because in case of UCS2/UTF16/UTF32 the source value
might be a not well formed string (e.g. have incomplete leftmost character).
Now set_name() is called on str_value after its copied
(with optionally left zero padding) from the source string.
- MDEV-6694 Illegal mix of collation with a PS parameter
Item_param::convert_str_value() did not set repertoire.
Introducing a new structure MY_STRING_METADATA to collect
character length and repertoire of a string in a single loop,
to avoid two separate loops. Adding a new class Item_basic_value::Metadata
as a convenience wrapper around MY_STRING_METADATA, to reuse the
code between Item_string and Item_param.
The problem was caused by the following scenario:
- range optimizer picks an index IDX1 which doesn't match the ORDER BY ...
LIMIT clause.
- test_if_skip_sort_order() decides to switch to index IDX2 which matches
the ORDER BY ... LIMIT.
- it runs SQL_SELECT::test_quick_select() for the second time to produce
an quick select for IDX2.
- However, test_quick_select() would figure that full index scan on IDX1
is still cheaper (its calculations ignore the LIMIT n).
Fixed this by
- passing force_quick_range=true to test_quick_select()
- in test_quick_select, don't consider full index scans if the mentioned
parameter is true.
Numerous changes in .result files are caused by test_quick_select() being
run after "early/late NULLs filtering" feature has injected NOT NULL
condition.
If the slave gets a reconnect in the middle of a GTID event group, normally
it will re-fetch that event group, skipping the first part that was already
queued for the SQL thread.
However, if the master crashed while writing the event group, the group is
incomplete. This patch detects this case and makes sure that the
transaction is rolled back and nothing is skipped from any following
event groups.
Similarly, a network proxy might cause the reconnect to end up on a
different master server. Detect this by noticing a different server_id,
and similarly in this case roll back the partially received group.
Item_string::eq() and Item_param::eq() in string context behaved differently.
Introducing a new class Item_basic_value to share the eq() code between
literals (Item_int, Item_double, Item_string, Item_null) and Item_param.
MDEV-6666 Malformed result for CONCAT(utf8_column, binary_string)
Item_static_string_func::safe_charset_converter() and
Item_hex_string::safe_charset_converter() did not
handle character sets with mbminlen>1 properly, as well as
did not handle conversion from binary to multi-byte well.
Introducing Item::const_charset_converter(), to reuse it in a number
of Item_*::safe_charset_converter().
MDEV-6656: Test wsrep.variables hangs
Analysis: wsrep_applier_thread shutdown signaling does not always work
correctly causing a timing problem where main thread is waiting in a
condition variable a signal that all worker threads to end.
When ORDER BY ... LIMIT check whether it should switch from index IDX1
to index IDX2, it should not ignore the fact that IDX2 may have a
potential range or ref(const) access.
Istead, it should calculate their costs: there is now a saved range
optimizer cost and code to re-calculate what best_access_path()
calculated for ref(const).
/* in current cost model these two can be very different
numbers unfortunately */
Let range optimizer remove parts of OR-clauses for which range analysis
produced SEL_TREE(IMPOSSIBLE).
There is no need to remove parts of AND-clauses: either they are inside
of OR (and the whole AND-clause will be removed), or the AND-clause is
at the top level, in which case the whole WHERE (or ON) is always FALSE
and this is a degenerate case which receives special treatment.
The removal process takes care not to produce 1-way ORs (in that case
we substitute the OR for its remaining member).
Merged lp:maria/maria-10.0-galera up to revision 3880.
Added a new functions to handler API to forcefully abort_transaction,
producing fake_trx_id, get_checkpoint and set_checkpoint for XA. These
were added for future possiblity to add more storage engines that
could use galera replication.
Merged lp:maria/maria-10.0-galera up to revision 3879.
Added a new functions to handler API to forcefully abort_transaction,
producing fake_trx_id, get_checkpoint and set_checkpoint for XA. These
were added for future possiblity to add more storage engines that
could use galera replication.
Don't restore the whole of thd->server_status after a routine invocation,
only restore SERVER_STATUS_CURSOR_EXISTS and SERVER_STATUS_LAST_ROW_SENT,
as --ps --embedded needs.
In particular, don't restore SERVER_STATUS_IN_TRANS.
IS [NOT] NULL predicate is sargable within an outer join. Range
analysis only uses predicates from ON expressions, which have
regular semantics (without null-complemented rows, etc).
There is no reason not use IS [NOT] NULL predicates.
sporadically
Fix: Modify test to be smaller so that testcase timeout does not
trigger. We already have a test for --big-test setup
(innodb.innodb_simulate_comp_failures).
COLLATIONS ARE USED.
ISSUE :
-------
Code points of HALF WIDTH KATAKANA in SJIS/CP932 range from
A1 to DF. In function my_wildcmp_mb_bin_impl while comparing
such single byte code points, there is a code which compares
signed character with unsigned character. Because of this,
comparisons of two same code points representing a HALF
WIDTH KATAKANA character always fails.
Solution:
---------
A code point of HALF WIDTH KATAKANA at-least need 8 bits.
Promoting the variable from uchar to int will fix the issue.
mysql-test/t/ctype_cp932.test:
Tests which have conditions
LIKE 'STRING PATTERN WITH HALF WIDTH KATAKANA'.
strings/ctype-mb.c:
A code point of HALF WIDTH KATAKANA at-least need 8 bits.
Promoting the variable from uchar to int will fix the issue.
Follow-up patch, fixing a possible deadlock issue.
If the master crashes in the middle of an event group, there can be an active
transaction in a worker thread when we encounter the following master restart
format description event. In this case, we need to notify that worker thread
to abort and roll back the partial event group. Otherwise a deadlock occurs:
the worker thread waits for the commit that never arrives, and the SQL driver
thread waits for the worker thread to complete its event group, which it never
does.
mysqldump:
- Added --log-queries to allow one to disable logging for the dump
sql/log_event.cc:
- Removed setting of enable_slow_log as it's not required anymore.
sql/sql_parse.cc:
- Set enable_slow_log to value of thd->variables.sql_log_slow as this will speed up tests if slow log is disabled.
- opt_log_slow_admin_statements can now only disable slow log, not enable it.
sql/sql_explain.cc:
- Minor cleanup
Other things:
- Added sql_log_slow to system variables.
- Changed opt_slow_log to global_system_variables.sql_log_slow in all files
- Updated tests to reflect changes
Fix the bug properly (plugin cannot be unloaded as long as it's locked).
Enable and fix the test case.
Significantly reduce number of LOCK_plugin locks for semisync
(practically all locks were removed)
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
This bug only happens in case of paritioned tables used in LOCK TABLES and implicit_commit() was called
(as part of trying to execute a CREATE TABLE withing lock tables)
The problem was that Aria could not move the tables from one transaction to the new one, as thd->open_tables contained
a partitioned tables and not an Aria table.
Fix:
- Store a list of all open tables that are part of a share in share->open_tables
- In maria::implict_commit() use transaction->used_tables & share->open_tables to find out which tables
was part of the current transaction instead of using thd->open_tables, which may contain partitioned tables.
mysql-test/suite/maria/maria_partition.result:
Added test case
mysql-test/suite/maria/maria_partition.test:
Added test case
storage/maria/ha_maria.cc:
Use trn->used tables and share->open_tables to find out which tables was part of the current transaction instead of using thd->open_tables.
storage/maria/ma_close.c:
Remove closed table from share->open_list
storage/maria/ma_open.c:
Add table to share->open_list
storage/maria/ma_state.c:
Added comment
storage/maria/maria_def.h:
Added share->open_list, a list of all tables that is using this share.
- adjusted a test result according to the change made for MDEV-6100;
- added explicit timezone for engines/iuds, since MTR in
MariaDB does not set it like MySQL's, and tests with constant TIMESTAMP
can have different outcome
We don't support changing tc_log implementation at run time.
If the first XA-capable engine is loaded with INSTALL PLUGIN - disable its
XA capabilities with a warning
MySQL Bug#71095: Wrong results with PARTITION BY LIST COLUMNS()
MySQL Bug#72803: Wrong "Impossible where" with LIST partitioning
MDEV-6240: Wrong "Impossible where" with LIST partitioning
- Backprot the fix from MySQL Bug#71095.
- When traversing JOIN_TABs with first_linear_tab/next_linear_tab(), don't forget
to enter the semi-join nest when it is the first table in the join order.
Failure to do so could cause e.g. I_S tables not to be filled.
GOES AWAY, MYSQL QUITS WORKING.
Analysis:
-----------------
Issue in this bug and in bug 11907705 is, the socket file or
fifo file is set for general log at command line while starting
the server. But currently, only regular file can be set for the
general log. Instead of reporting any error, the provided files
are opened for writing and continued. Because of this issues
mentioned in the bug reports are seen.
As mentioned, only when any non-regular file is set for general
log at command line while starting the server, these issues are
seen. If general log file is set to non-regular file from CLI
using system variable general_log_file then error is reported.
These issues can also be faced with slow query log file, if it is
set to non-regular file.
Fix:
-----------------
Currently while starting the server if we fail to open log file
then we report an error, disable logging to file and continue.
To fix issue reported code is modified to check whether file
is regular file or not before opening it. If file is not a
regular file then error is logged to error log and logging to
file is disabled.
When a master server starts up, it logs a special format_description event at
the start of a new binlog to mark that is has restarted. This is used by a
slave to drop all temporary tables - this is needed in case the master crashed
and did not have a chance to send explicit DROP TEMPORARY TABLE statements to
the slave.
In parallel replication, we need to be careful when dropping the temporary
tables - we need to be sure that no prior events are still executing that
might be using the temporary tables to be dropped, _and_ that no following
events have started executing that might have created new temporary tables
that should not be dropped.
This was not handled correctly, which could cause errors about access to not
existing temporary tables or even crashes. This patch implements that such
format_description events cause serialisation of event execution; all prior
events are executed to completion first, then the format_description event is
executed, dropping temporary tables, then following events are queued for
execution.
Master restarts should be sufficiently infrequent that the resulting loss of
parallelism should be of minimal impact.
The problem occured when using parallel replication, and an error occured that
caused the SQL thread to stop when the IO thread had already reached a
following binlog file from the master (or otherwise performed a relay log
rotation).
In this case, the Rotate Event at the end of the relay log file could still be
executed, even though an earlier event in that relay log file had gotten an
error. This would cause the position to be incorrectly updated, so that upon
restart of the SQL thread, the event that had failed would be silently skipped
and ignored, causing replication corruption.
Fixed by checking before executing Rotate Event, whether an earlier event
has failed. If so, the Rotate Event is not executed, just dequeued, same as
for other normal events following a failing event.
If one uses 4 --verbose to mysql_upgrade it will also write out all mysqlcheck commands invoked.
mysql-test/r/mysql_upgrade.result:
Updated results from changing phases
mysql-test/r/mysql_upgrade_no_innodb.result:
dated results from changing phases
mysql-test/r/mysql_upgrade_ssl.result:
dated results from changing phases
The bug was that in some cases, if a replicated transaction was rolled back
due to deadlock, during the subsequent retry of that transaction, the
gtid_slave_pos would _not_ be updated with the new GTID, leaving the GTID
position of the slave incorrect.
Fix this by ensuring during the retry that we clear the flag that marks that
the GTID has already been recorded in gtid_slave_pos, so that the update of
gtid_slave_pos will be done again during the retry.
In the original bug, the symptom was an assertion due to OPTION_GTID_BEGIN not
being cleared during the retry of the transaction. The reason was some code in
handling of a COMMIT query event, which would not clear the flag when not
recording a GTID in gtid_slave_pos. This commit also fixes that code to always
clear the OPTION_GTID_BEGIN flag for clarity, though it is actually not
possible for OPTION_GTID_BEGIN to become set unless a GTID is pending for
update (after fixing the bug described above).
sporadically
Fix: Modify test to be smaller so that testcase timeout does not
trigger. We already have a test for --big-test setup
(innodb.innodb_simulate_comp_failures).
A new command line option "galera-sst-mode" was introduced
in mysqldump as part of fix for MDEV-6316. But, since the
fix was pushed to maria-10.0-galera branch, the mysqldump
tool supplied with mariadb client deb/rpm packages, does not
have this new opion.
This fix contains the same patch along with a test case.
INSTALL SONAME ignores attempts to load the same plugin twice, it's not an error
(because one can load one plugin by name and then install soname for the rest).
But Federated and FederatedX are different plugins, despite having the same name.
Now plugin_add() only considers two plugins identical if their names are the same
string (compared as pointers). Otherwise it reports an error.,
rpl.rpl_gtid_crash fail on PPC64
This is an addition to the original patch.
Restored show binlog events output and adjusted filters to
replace [\d-\d-\d,\d-\d-\d,\d-\d-\d] with [#-#-#].
Remove the "don't update the row for b'' and store uninitialized bytes on disk" change.
Update test cases to allow DEFAULT b'', because b'' is a valid expression elsewhere.
4229: MDEV-5670: Assertion failure in file buf0lru.c line 2355
Add more status information if repeatable.
4230: MDEV-5673: Crash while parallel dropping multiple tables under heavy load
Improve long semaphore wait output to include all semaphore waits
and try to find out if there is a sequence of waiters.
4233: Fix compiler errors on product build.
4237: Fix too agressive long semaphore wait output and add guard against introducing
compression failures on insert buffer.
4238: Fix test failure caused by simulated compression failure on
IBUF_DUMMY table.
Reason for the problem was that the hash of changed files in the key cache was too small (was 128). Fixed by making the hash size larger and changeable.
- Introduced key-cache-file-hash-size (default 512) for MyISAM and aria_pagecache_file_hash_size (default 512) for Aria.
- Added new status variable "Feature_delay_key_write" which counts number of tables opened that are using delay_key_write
mysql-test/r/features.result:
Added test of Feature_delay_key_write
mysql-test/r/key_cache.result:
Updated tests as the number of blocks has changed
mysql-test/r/mysqld--help.result:
Updated result
mysql-test/suite/maria/maria3.result:
Updated result
mysql-test/suite/sys_vars/r/key_cache_file_hash_size_basic.result:
Test new variable
mysql-test/suite/sys_vars/t/aria_pagecache_file_hash_size_basic.test:
Test new variable
mysql-test/suite/sys_vars/t/key_cache_file_hash_size_basic.test:
Test new variable
mysql-test/t/features.test:
Added test of Feature_delay_key_write
mysql-test/t/key_cache.test:
Updated tests as the number of blocks has changed
mysys/mf_keycache.c:
Made CHANGED_BLOCKS_HASH dynamic
sql/handler.cc:
Updated call to init_key_cache()
sql/mysqld.cc:
Added "Feature_delay_key_write"
Added support for key-cache-file-hash-size
sql/mysqld.h:
Added support for key-cache-file-hash-size
sql/sql_class.h:
Added feature_files_opened_with_delayed_keys
sql/sys_vars.cc:
Added key_cache_file_hash_size
storage/maria/ha_maria.cc:
Added pagecache_file_hash_size
Added counting of files with delay_key_write
storage/maria/ma_checkpoint.c:
Fixed compiler warning
storage/maria/ma_pagecache.c:
Made PAGECACHE_CHANGED_BLOCKS_HASH into a variable
storage/maria/ma_pagecache.h:
Made PAGECACHE_CHANGED_BLOCKS_HASH into a variable
storage/maria/ma_rt_test.c:
Updated parameters for init_pagecache()
storage/maria/ma_test1.c:
Updated parameters for init_pagecache()
storage/maria/ma_test2.c:
Updated parameters for init_pagecache()
storage/maria/ma_test3.c:
Updated parameters for init_pagecache()
storage/maria/maria_chk.c:
Updated parameters for init_pagecache()
storage/maria/maria_ftdump.c:
Updated parameters for init_pagecache()
storage/maria/maria_pack.c:
Updated parameters for init_pagecache()
storage/maria/maria_read_log.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_consist.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_rwconsist.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_rwconsist2.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_single.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
Updated parameters for init_pagecache()
storage/myisam/ha_myisam.cc:
Added counting of files with delay_key_write
storage/myisam/mi_check.c:
Updated call to init_key_cache()
storage/myisam/mi_test1.c:
Updated call to init_key_cache()
storage/myisam/mi_test2.c:
Updated call to init_key_cache()
storage/myisam/mi_test3.c:
Updated call to init_key_cache()
storage/myisam/mi_test_all.sh:
Fixed broken test
storage/myisam/myisam_ftdump.c:
Updated call to init_key_cache()
storage/myisam/myisamchk.c:
Updated call to init_key_cache()
storage/myisam/myisamlog.c:
Updated call to init_key_cache()
The bug was that my_real_read() called net_before_header_psi() multiple times for long packets.
Fixed by adding a flag when we are reading a header.
Did also some cleanups to interface of my_net_read() to avoid unnecessary calls if performance schema is not used.
- Added my_net_read_packet() as a replacement for my_net_read(). my_net_read() is still in the client library for old clients.
- Removed THD->m_server_idle (not needed anymore as this is now given as argument to my_net_read_packet()
- Added my_net_read_packet(), which is a new version of my_net_read() with a new parameter if we are doing a read for a new command from the server.
- Added tests for compressed protocol and big packets
include/mysql.h.pp:
Added my_net_read_packet() as a replacement for my_net_read()
include/mysql_com.h:
Added my_net_read_packet() as a replacement for my_net_read()
mysql-test/r/mysql_client_test_comp.result:
New test
mysql-test/t/mysql_client_test-master.opt:
Added max_allowed_packet to be able to test big packets and packet size overflows.
mysql-test/t/mysql_client_test_comp-master.opt:
New test
mysql-test/t/mysql_client_test_nonblock-master.opt:
Added max_allowed_packet to be able to test big packets and packet size overflows.
sql-common/client.c:
Use my_net_read_packet()
sql/mf_iocache.cc:
Use my_net_read_packet()
sql/mysqld.cc:
Removed THD->m_server_idle (not needed anymore as this is now given as argument to my_net_read_packet()
sql/net_serv.cc:
Added argument to my_real_read() to indicte if we are reading the first block of the next statement and should call performance schema.
Added 'compatibilty function' my_net_read().
Added my_net_read_packet(), which is a new version of my_net_read() with a new parameter if we are doing a read for a new command from the server.
sql/sql_class.cc:
Removed m_server_idle (not needed anymore)
sql/sql_class.h:
Removed m_server_idle (not needed anymore)
sql/sql_parse.cc:
Removed m_server_idle (not needed anymore)
tests/mysql_client_test.c:
Added tests for compressed protocol and big packets
Merge the patches into MariaDB 10.0 main.
With this patch, parallel replication will now automatically retry a
transaction that fails due to deadlock or other temporary error, same as
single-threaded replication.
We catch deadlocks with InnoDB transactions due to enforced commit order. If
T1 must commit before T2 in parallel replication and T1 ends up waiting for T2
inside InnoDB, we kill T2 and retry it later to resolve the deadlock
automatically.
Fix a bug discovered in Buildbot valgrind. The logic in checking for slave
init thread completion was reversed, so depending on thread scheduling server
startup could hang.
Also add another variant of SSL valgrind suppression, needed for different
library version.
When a MyISAM query is killed midway, the query is logged to the binlog marked
with the error.
The slave does not attempt to run the query, but aborts with a suitable error
message in the error log for the DBA to act on.
In this case, the parallel replication code would check the sql_errno() code,
even no my_error() had been set. In debug builds, this causes an assertion.
Fixed the code to check that we actually have an error set before querying for
an error code.
After-review changes.
For this patch in 10.0, we do not introduce a new public storage engine API,
we just fix the InnoDB/XtraDB issues. In 10.1, we will make a better public
API that can be used for all storage engines (MDEV-6429).
Eliminate the background thread that did deadlock kills asynchroneously.
Instead, we ensure that the InnoDB/XtraDB code can handle doing the kill from
inside the deadlock detection code (when thd_report_wait_for() needs to kill a
later thread to resolve a deadlock).
(We preserve the part of the original patch that introduces dedicated mutex
and condition for the slave init thread, to remove the abuse of
LOCK_thread_count for start/stop synchronisation of the slave init thread).
- In print_explain_row(), do not forget to print r_rows.
- Switch Explain_update from using its own counters to re-using
Table_access_tracker.
- Make ANALYZE UPDATE code structure uniform with ANALYZE DELETE.
The test case makes use of the fine DEBUG_SYNC facility. Furthermore,
since it needs synchronization on internal threads (dump and SQL
threads) the server code has DEBUG_SYNC commands internally deployed
and activated through the DBUG_EXECUTE_IF macro. The internal
DBUG_SYNC commands are then controlled from the test case through the
DEBUG variable.
There were three problems around the DEBUG + DEBUG_SYNC facility
usage:
1. When signaling the SQL thread to continue, the test would reset
immediately the DEBUG_SYNC variable. This could mean that the SQL
thread might loose the signal and continue to wait forever;
2. A similar scenario was happening with the dump thread on the
master. This thread was instructed to wait, and later it would be
signaled to continue, but immediately after the DEBUG_SYNC would be
reset. This could lead to the dump thread missing the signal and
wait forever;
3. The test was not cleaning itself up with respect to the
instrumentation of the dump thread. This would leave the
conditional execution of an internal DEBUG_SYNC command active
(through the usage of DBUG_EXECUTE_IF).
We fix#1 and #2 by waiting for the threads to receive the signal and
only then issue the reset. We fix#3 by reseting the DEBUG variable,
thus deactivating the dump thread internal DEBUG_SYNC command.
The sql_slave_skip_counter is important to be able to recover replication from
certain errors. Often, an appropriate solution is to set
sql_slave_skip_counter to skip over a problem event. But setting
sql_slave_skip_counter produced an error in GTID mode, with a suggestion to
instead set @@gtid_slave_pos to point past the problem event. This however is
not always possible; for example, in case of an INCIDENT event, that event
does not have any GTID to assign to @@gtid_slave_pos.
With this patch, sql_slave_skip_counter now works in GTID mode the same was as
in non-GTID mode. When set, that many initial events are skipped when the SQL
thread starts, plus as many extra events are needed to completely skip any
partially skipped event group. The GTID position is updated to point past the
skipped event(s).
MDEV-6344: mysqldump issues FLUSH TABLES, which gets written into binlog and replicated
Add a --gtid option (for compatibility, the original behaviour is preserved
when --gtid is not used).
With --gtid, --master-data and --dump-slave output the GTID position (the
old-style file/offset position is still output, but commented out). Also, a
CHANGE MASTER TO master_use_gtid=slave_pos is output to ensure a provisioned
slave is configured in GTID, as requested.
Without --gtid, the GTID position is still output, if available, but commented
out.
Also fix MDEV-6344, to avoid FLUSH TABLES getting into the binlog. Otherwise a
mysqldump on a slave server will silently inject a GTID which does not exist
on the master, which is highly undesirable.
Also fix an incorrect error handling around obtaining binlog position with
--master-data (was probably unlikely to trigger in most cases).
Analysis: For some reason table stats for a table pointed from a index
is not initialized. Added additional warning output on this situation
and table stats initialization. This is better than asserting.
These tests use search_pattern_in_file.inc to search the error log for
expected output. However, search_pattern_in_file.inc by default searched only
the first 50000 bytes, so if the error log grew too big the tests would fail.
This patch extends search_pattern_in_file.inc with an option to specify how
much of the file to search, and whether to search from the start of the file
or from the end. Then the rpl.rpl_checksum and rpl.rpl_gtid_errorlog test
cases are fixed to search the last 50000 bytes of the error log, which will
work no matter how large prior tests have made it.
The direct cause of the assertion was missing error handling in
record_gtid(). If ha_commit_trans() fails for the statement commit, there was
missing code to catch the error and do ha_rollback_trans() in this case; this
caused close_thread_tables() to assert.
Normally, this error case is not hit, but in this case it was triggered due to
another bug: When a transaction T1 fails during parallel replication, the code
would signal following transactions that they could start to run without
properly marking the error condition. This caused subsequent transactions to
incorrectly start replicating, only to get an error later during their own
commit step. This was particularly serious if the subsequent transactions were
DDL or MyISAM updates, which cannot be rolled back and would leave replication
in an inconsistent state.
Fixed by 1) in case of error, only signal following transactions to continue
once the error has been properly marked and those transactions will know not
to start; and 2) implement proper error handling in record_gtid() in the case
that statement commit fails.
If replication breaks in GTID mode, it is not trivial to determine the GTID of
the failing event group. This is a problem, as such GTID is needed eg. to
explicitly set @@gtid_slave_pos to skip to after that event group, or to
compare errors on different servers, etc.
Fix by ensuring that relevant slave errors logged to the error log include the
GTID of the event group containing the problem event.
This is MySQL Bug#59123. The message string stored in an INCIDENT event was
not zero-terminated. This caused any following checksum bytes (if enabled on
the master) to be output to the error log as trailing garbage when the message
was printed to the error log.
Backport the patch from MySQL 5.6:
revno: 2876.228.200
revision-id: zhenxing.he@sun.com-20110111051323-w2xnzvcjn46x6h6u
committer: He Zhenxing <zhenxing.he@sun.com>
timestamp: Tue 2011-01-11 13:13:23 +0800
message:
BUG#59123 rpl_stm_binlog_max_cache_size fails sporadically with found warnings
Also add a test case.
Bug#16415173 CRLF INSTEAD OF LF IN SQL-BENCH SCRIPTS
Correct perms and converts from Windows style to UNIX style line endings on some files.
Fix perms on installed ini files.
(MySQL 5.5 version)
- "ANALYZE $stmt" should discard select's output, but it should still
evaluate the output columns (otherwise, subqueries in select list
are not executed)
- SHOW EXPLAIN's code practice of calling JOIN::save_explain_data()
after JOIN::exec() is disastrous for ANALYZE, because it resets
all counters after the first execution. It is stopped
= "Late" test_if_skip_sort_order() calls explicitly update their part
of the query plan.
= Also, I had to rewrite I_S optimization to actually have optimization
and execution stages.
MySQL 5.6 implemented WL#344, which is about a MASTER_DELAY option to CHANGE
MASTER. But as part of this worklog, the format of the realy-log.info file was
changed. The new format is not understood by earlier versions, and nor by
MariaDB 10.0, so changing server to those versions would cause the slave to
abort with an error due to reading incorrect data out of relay-log.info.
Fix this by backporting from the WL#344 patch just the code that understands
the new relay-log.info format. We still write out the old format, and none of
the MASTER_DELAY feature is backported with this commit.
CORRUPTS FRM
Analysis:
---------
ALTER TABLE on a partitioned table resulted in the wrong
engine being written into the table's FRM file and displayed
in SHOW CREATE TABLE.
The prep_alter_part_table() modifies the partition_info object
for TABLE instance representing the old version of table.
If the ALTER TABLE ENGINE statement fails, the partition_info
object for the TABLE contains the altered storage engine name.
The SHOW CREATE TABLE uses the TABLE object to display the table
information, hence displays incorrect storage engine for the table.
Also a subsequent successful ALTER TABLE operation will write the
incorrect engine information into the FRM file.
Fix:
---
A copy of the partition_info object is created before modification so
that any changes would not cause the the original partition_info object
to be modified if the ALTER TABLE fails.(Backported part of the code
provided as fix for bug#14156617 in mysql-5.6.6).
* enum values to index different ACL tables, instead of hard-coded numbers
(even different in diffent functions).
* move TABLE_LIST initialization into open_grant_tables()
and use it everywhere
* change few my_bool's to bool's
* Don't write frm for tmp tables
* pass frm image down to open_table_uncached, when possible
* don't use truncate-by-recreate for temp tables - cannot recreate
without frm, and delete_all_rows is faster anyway
Auto-generate the allowed list of values for enum/set/flagset options
in --help output. But don't do that when the help text already has them.
Also, remove lists of values from help strings of various options, where
they were simply listed without any additional information.
it checked that the default is lz4. Which only worked on systems that
had lz4 and did not have lzo. Now it checks for the default to be zlib,
which works on systems that has neither lz4 or lzo. Like our package
builders in buildbot. This is intentional, we don't want introduce
additional dependencies (lz4, lzo) for our packages just yet.
This can (and will) be reconsidered, and this test can (and will)
be updated again.
The INCIDENT_EVENT always caused slave error and abort, without checking
--slave-skip-errors.
Now, if error 1590, ER_SLAVE_INCIDENT is included in the --slave-skip-errors
list, incident events will be ignored.
This is a merge of this MySQL 5.6 patch:
revision-id: frazer@mysql.com-20110314170916-ypgin17otj3ucx95
committer: Frazer Clement <frazer@mysql.com>
timestamp: Mon 2011-03-14 17:09:16 +0000
message:
Bug#11799671 NOT POSSIBLE TO SKIP INCIDENT ERRORS
DISCONNECT CON1 AND CON2
Problem:
The test suite/binlog/t/binlog_killed.test makes the connections
con1 and con2 but forgets to disconnect them + wait till that
operation is finished at test end.
This mistake has the potential to harm subsequent tests in
case these tests depend on the content of the processlist.
Solution:
Added disconnect + wait_until_disconnected.inc
within the test cleanup.
- Filesort has an optmization where it reads only columns that are
needed before the sorting is done.
- When ref(_or_null) is picked by the join optimizer, it may remove parts
of WHERE clause that are guaranteed to be true.
- However, if we use quick select, we must put all of the range columns into the
read set. Not doing so will may cause us to fail to detect the end of the range.
That particular part of slave connect to master was missing code to handle
retry in case of network errors. The same problem is present in MySQL 5.5, but
fixed in MySQL 5.6.
Fixed with this patch, by adding the code (mostly identical to MySQL 5.6), and
also adding a test case.
I checked other queries done towards master during slave connect, and they now
all seem to handle reconnect in case of network failures.
Adapt default_tmp_storage_engine implementation from mysql-5.6
New feature (as compared to 5.6), default_tmp_storage_engine=NULL
means that temporary tables will use default_storage_engine value.
This makes the behavior backward compatible.
When plugin=mysql_native_password (or mysql_old_password) take the password
from *either* password *or* authentication_string, whichever is set.
This makes no sense, but alas, that's what MySQL-5.6 does.
replication causing replication to fail.
Remove the temporary fix for MDEV-5914, which used READ COMMITTED for parallel
replication worker threads. Replace it with a better, more selective solution.
The issue is with certain edge cases of InnoDB gap locks, for example between
INSERT and ranged DELETE. It is possible for the gap lock set by the DELETE to
block the INSERT, if the DELETE runs first, while the record lock set by
INSERT does not block the DELETE, if the INSERT runs first. This can cause a
conflict between the two in parallel replication on the slave even though they
ran without conflicts on the master.
With this patch, InnoDB will ask the server layer about the two involved
transactions before blocking on a gap lock. If the server layer tells InnoDB
that the transactions are already fixed wrt. commit order, as they are in
parallel replication, InnoDB will ignore the gap lock and allow the two
transactions to proceed in parallel, avoiding the conflict.
Improve the fix for MDEV-6020. When InnoDB itself detects a deadlock, it now
asks the server layer for any preferences about which transaction to roll
back. In case of parallel replication with two transactions T1 and T2 fixed to
commit T1 before T2, the server layer will ask InnoDB to roll back T2 as the
deadlock victim, not T1. This helps in some cases to avoid excessive deadlock
rollback, as T2 will in any case need to wait for T1 to complete before it can
itself commit.
Also some misc. fixes found during development and testing:
- Remove thd_rpl_is_parallel(), it is not used or needed.
- Use KILL_CONNECTION instead of KILL_QUERY when a parallel replication
worker thread is killed to resolve a deadlock with fixed commit
ordering. There are some cases, eg. in sql/sql_parse.cc, where a KILL_QUERY
can be ignored if the query otherwise completed successfully, and this
could cause the deadlock kill to be lost, so that the deadlock was not
correctly resolved.
- Fix random test failure due to missing wait_for_binlog_checkpoint.inc.
- Make sure that deadlock or other temporary errors during parallel
replication are not printed to the the error log; there were some places
around the replication code with extra error logging. These conditions can
occur occasionally and are handled automatically without breaking
replication, so they should not pollute the error log.
- Fix handling of rgi->gtid_sub_id. We need to be able to access this also at
the end of a transaction, to be able to detect and resolve deadlocks due to
commit ordering. But this value was also used as a flag to mark whether
record_gtid() had been called, by being set to zero, losing the value. Now,
introduce a separate flag rgi->gtid_pending, so rgi->gtid_sub_id remains
valid for the entire duration of the transaction.
- Fix one place where the code to handle ignored errors called reset_killed()
unconditionally, even if no error was caught that should be ignored. This
could cause loss of a deadlock kill signal, breaking deadlock detection and
resolution.
- Fix a couple of missing mysql_reset_thd_for_next_command(). This could
cause a prior error condition to remain for the next event executed,
causing assertions about errors already being set and possibly giving
incorrect error handling for following event executions.
- Fix code that cleared thd->rgi_slave in the parallel replication worker
threads after each event execution; this caused the deadlock detection and
handling code to not be able to correctly process the associated
transactions as belonging to replication worker threads.
- Remove useless error code in slave_background_kill_request().
- Fix bug where wfc->wakeup_error was not cleared at
wait_for_commit::unregister_wait_for_prior_commit(). This could cause the
error condition to wrongly propagate to a later wait_for_prior_commit(),
causing spurious ER_PRIOR_COMMIT_FAILED errors.
- Do not put the binlog background thread into the processlist. It causes
too many result differences in mtr, but also it probably is not useful
for users to pollute the process list with a system thread that does not
really perform any user-visible tasks...
SLOW/CRASHES SEMAPHORE
Problem:
There are 2 lakh tables - fk_000001, fk_000002 ... fk_200000. All of them
are related to the same parent_table through a foreign key constraint.
When the parent_table is loaded into the dictionary cache, all the child table
will also be loaded. This is taking lot of time. Since this operation happens
when the dictionary latch is taken, the scenario leads to "long semaphore wait"
situation and the server gets killed.
Analysis:
A simple performance analysis showed that the slowness is because of the
dict_foreign_find() function. It does a linear search on two linked list
table->foreign_list and table->referenced_list, looking for a particular
foreign key object based on foreign->id as the key. This is called two
times for each foreign key object.
Solution:
Introduce a rb tree in table->foreign_rbt and table->referenced_rbt, which
are some sort of index on table->foreign_list and table->referenced_list
respectively, using foreign->id as the key. These rbt structures will be
solely used by dict_foreign_find().
rb#5599 approved by Vasil
The method JOIN_CACHE::init may fail (return 1) if some conditions on the
used join buffer is not satisfied. For example it fails if join_buffer_size
is greater than join_buffer_space_limit. The conditions should be checked
when running the EXPLAIN command for the query. That's why the method
JOIN_CACHE::init has to be called for EXPLAIN commands as well.
Part#1.
table_cond_selectivity() should discount selectivity of table'
conditions only when ity counts that selectivity to begin with.
For non-ref-based access methods (ALL/range/index_merge/etc),
we start with sel=1.0 and hence do not need to discount any
selectivities.
