Repairing MyISAM table with fulltext indexes and low
myisam_sort_buffer_size may crash the server.
Estimation of number of index entries was done incorrectly,
causing further assertion failure or server crash.
Docs note: min value for myisam_sort_buffer_size has been
changed from 4 to 4096.
mysql-test/r/fulltext.result:
A test case for BUG#51866.
mysql-test/r/myisam.result:
Min value for myisam_sort_buffer_size is 4096.
mysql-test/r/variables.result:
Min value for myisam_sort_buffer_size is 4096.
mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic_32.result:
Min value for myisam_sort_buffer_size is 4096.
mysql-test/t/fulltext.test:
A test case for BUG#51866.
sql/mysqld.cc:
Min value for myisam_sort_buffer_size is 4096.
storage/myisam/mi_check.c:
When estimating number of index entries for external
fulltext parser, take into account that key_length may
be bigger than myisam_sort_buffer_size. Reuse logic
from _create_index_by_sort(): force MIN_SORT_BUFFER to
be min value for myisam_sort_buffer_size.
Another problem is that ftkey_nr has no other meaning
than serial number of fulltext index starting with 1.
We can't say if this key using built-in or external
parser basing on it's value. In other words we always
entered if-branch for external parser. At this point,
the only way to check if we use default parser is to
compare keyinfo::parser with &ft_default_parser.
storage/myisam/sort.c:
Get rid of MIN_SORT_MEMORY, use MIN_SORT_BUFFER instead
(defined in myisamdef.h, has the same value and purpose).
Conflicts:
Text conflict in client/mysqlbinlog.cc
Text conflict in mysql-test/Makefile.am
Text conflict in mysql-test/collections/default.daily
Text conflict in mysql-test/r/mysqlbinlog_row_innodb.result
Text conflict in mysql-test/suite/rpl/r/rpl_typeconv_innodb.result
Text conflict in mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test
Text conflict in mysql-test/suite/rpl/t/rpl_row_create_table.test
Text conflict in mysql-test/suite/rpl/t/rpl_slave_skip.test
Text conflict in mysql-test/suite/rpl/t/rpl_typeconv_innodb.test
Text conflict in mysys/charset.c
Text conflict in sql/field.cc
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_func.cc
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/rpl_utility.cc
Text conflict in sql/rpl_utility.h
Text conflict in sql/set_var.cc
Text conflict in sql/share/Makefile.am
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_plugin.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_table.cc
Text conflict in storage/example/ha_example.h
Text conflict in storage/federated/ha_federated.cc
Text conflict in storage/myisammrg/ha_myisammrg.cc
Text conflict in storage/myisammrg/myrg_open.c
When mysqlbinlog was given the --database=X flag, it always printed
'ROLLBACK TO', but the corresponding 'SAVEPOINT' statement was not
printed. The replicated filter(replicated-do/ignore-db) and binlog
filter (binlog-do/ignore-db) has the same problem. They are solved
in this patch together.
After this patch, We always check whether the query is 'SAVEPOINT'
statement or not. Because this is a literal check, 'SAVEPOINT' and
'ROLLBACK TO' statements are also binlogged in uppercase with no
any comments.
The binlog before this patch can be handled correctly except one case
that any comments are in front of the keywords. for example:
/* bla bla */ SAVEPOINT a;
/* bla bla */ ROLLBACK TO a;
The log event of 'CREATE EVENT' was being binlogged with garbage
at the end of the query if 'CREATE EVENT' is followed by another SQL statement
and they were executed as one command.
for example:
DELIMITER |;
CREATE EVENT e1 ON EVERY DAY DO SELECT 1; SELECT 'a';
DELIMITER ;|
When binlogging 'CREATE EVENT', we always create a new statement with definer
and write it into the log event. The new statement is made from cpp_buf(preprocessed buffer).
which is not a c string(end with '\0'), but it is copied as a c string.
In this patch, cpp_buf is copied with its length.
A failed REVOKE statement is logged with error=0, thus causing
the slave to stop. The slave should not stop as this was an
expected error. Given that the execution failed on the master as
well the error code should be logged so that the slave can replay
the statement, get an error and compare with the master's
execution outcome. If errors match, then slave can proceed with
replication, as the error it got, when replaying the statement,
was expected.
In this particular case, the bug surfaces because the error code
is pushed to the THD diagnostics area after writing the event to
the binary log. Therefore, it would be logged with the THD
diagnostics area clean, hence its error code would not contain
the correct code.
We fix this by moving the error reporting ahead of the call to
the routine that writes the event to the binary log.
Conflicts:
Text conflict in mysql-test/r/partition_innodb.result
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_cmpfunc.h
Text conflict in sql/item_sum.h
Text conflict in sql/log_event_old.cc
Text conflict in sql/protocol.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_yacc.yy
DBUG_SYNC_POINT has at least one strong limitation that it's not defined
on all platforms. It has issues cooperating with @@debug.
All in all its functionality is superseded by DEBUG_SYNC facility and
there is no reason to maintain the old less flexible one.
Fixed with adding debug_sync_set_action() function as a facility to set up
a sync-action in the server sources code and re-writing existing simulations
(found 3) to use it.
Couple of tests have been reworked as well.
The patch offers a pattern for setting sync-points in replication threads
where the standard DEBUG_SYNC does not suffice to reach goals.
mysql-test/extra/rpl_tests/rpl_get_master_version_and_clock.test:
rewriting the test from GET_LOCK()-based to DEBUG_SYNC-based;
a pattern of usage DEBUG_SYNC for replication testing is provided.
mysql-test/suite/rpl/r/rpl_get_master_version_and_clock.result:
results are changed.
mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test:
rewriting the test from GET_LOCK()-based to DEBUG_SYNC-based;
limiting the test to run only with MIXED binlog-format as the test last
some 10 secs sensitively contributing to the total of tests run.
mysql-test/suite/rpl/t/rpl_show_slave_running.test:
rewriting the test from GET_LOCK()-based to DEBUG_SYNC-based.
sql/debug_sync.cc:
adding debug_sync_set_action() function as a facility to set up
a sync-action in the server sources code.
sql/debug_sync.h:
externalizing debug_sync_set_action().
sql/item_func.cc:
purging sources from DBUG_SYNC_POINT.
sql/mysql_priv.h:
purging sources from DBUG_SYNC_POINT.
sql/slave.cc:
rewriting failure simulations to base on DEBUG_SYNC rather than GET_LOCK()-based DBUG_SYNC_POINT.
sql/sql_repl.cc:
removing an orphan failure simulation line because no counterpart in tests existing.
All numeric operators and functions on integer, floating point
and DECIMAL values now throw an 'out of range' error rather
than returning an incorrect value or NULL, when the result is
out of supported range for the corresponding data type.
Some test cases in the test suite had to be updated
accordingly either because the test case itself relied on a
value returned in case of a numeric overflow, or because a
numeric overflow was the root cause of the corresponding bugs.
The latter tests are no longer relevant, since the expressions
used to trigger the corresponding bugs are not valid anymore.
However, such test cases have been adjusted and kept "for the
record".
mysql-test/r/func_math.result:
Added test cases for bug #8433.
Updated results of the test case for bug #31236.
mysql-test/r/func_misc.result:
Streamlined test cases.
mysql-test/r/func_test.result:
Streamlined test cases.
mysql-test/r/select.result:
Streamlined test cases.
mysql-test/r/sp.result:
Streamlined test cases.
mysql-test/r/strict.result:
Streamlined test cases.
mysql-test/r/type_newdecimal.result:
Streamlined test cases.
mysql-test/suite/sys_vars/r/sql_slave_skip_counter_basic.result:
Streamlined test cases.
mysql-test/suite/sys_vars/t/sql_slave_skip_counter_basic.test:
Streamlined test cases.
mysql-test/t/func_math.test:
Added test cases for bug #8433.
Updated results of the test case for bug #31236.
mysql-test/t/func_misc.test:
Streamlined test cases.
mysql-test/t/func_test.test:
Streamlined test cases.
mysql-test/t/select.test:
Streamlined test cases.
mysql-test/t/sp.test:
Streamlined test cases.
mysql-test/t/strict.test:
Streamlined test cases.
mysql-test/t/type_newdecimal.test:
Streamlined test cases.
sql/item_create.cc:
Changed Item_func_cot() to be defined as a standalone Item
rather than a combination of "1 / TAN(x)".
sql/item_func.cc:
Throw an 'out of range' error rather than returning an
incorrect value or NULL, when the result of a numeric
operator or a function is out of supported range for
the corresponding data type.
sql/item_func.h:
Added validation helpers as inline methods of Item_func.
sql/share/errmsg-utf8.txt:
New ER_DATA_OUT_OF_RANGE error.
for general use.
mysql-test/Makefile.am:
Adding directories of additional test suites
mysql-test/mysql-stress-test.pl:
Adding check for additional errors checking during test run
for InnoDB
The class Field_bit_as_char stores the metadata for the
field incorrecly because bytes_in_rec and bit_len are set
to (field_length + 7 ) / 8 and 0 respectively, while
Field_bit has the correct values field_length / 8 and
field_length % 8.
Solved the problem by re-computing the values for the
metadata based on the field_length instead of using the
bytes_in_rec and bit_len variables.
To handle compatibility with old server, a table map
flag was added to indicate that the bit computation is
exact. If the flag is clear, the slave computes the
number of bytes required to store the bit field and
compares that instead, effectively allowing replication
*without conversion* from any field length that require
the same number of bytes to store.
mysql-test/suite/rpl/t/rpl_typeconv_innodb.test:
Adding test to check compatibility for bit field
replication when using InnoDB
sql/field.cc:
Extending compatible_field_size() with flags from
table map to allow fields to check master info.
sql/field.h:
Extending compatible_field_size() with flags from
table map to allow fields to check master info.
sql/log.cc:
Removing table map flags since they are not used
outside table map class.
sql/log_event.cc:
Removing flags parameter from table map constructor
since it is not used and does not have to be exposed.
sql/log_event.h:
Adding flag to denote that bit length for bit field type
is exact and not potentially rounded to even bytes.
sql/rpl_utility.cc:
Adding fields to table_def to store table map flags.
sql/rpl_utility.h:
Removing obsolete comment and adding flags to store
table map flags from master.
concurrent I_S query
There were two problem:
1) MYSQL_LOCK_IGNORE_FLUSH also ignored name locks
2) there was a race between abort_and_upgrade_locks and
alter_close_tables
(i.e. remove_table_from_cache and
close_data_files_and_morph_locks)
Which allowed the table to be opened with MYSQL_LOCK_IGNORE_FLUSH flag
resulting in renaming a partition that was already in use,
which could cause the table to be unusable.
Solution was to not allow IGNORE_FLUSH to skip waiting for
a named locked table.
And to not release the LOCK_open mutex between the
calls to remove_table_from_cache and
close_data_files_and_morph_locks by merging the functions
abort_and_upgrade_locks and alter_close_tables.
mysql-test/suite/parts/r/partition_debug_sync_innodb.result:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Added test result
mysql-test/suite/parts/t/partition_debug_sync_innodb-master.opt:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Added test option
mysql-test/suite/parts/t/partition_debug_sync_innodb.test:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Added test file
sql/authors.h:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Time to be acknowledged :)
sql/ha_partition.cc:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Added DEBUG_SYNC for deterministic testing
sql/mysql_priv.h:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Renamed function since merging alter_close_tables into
abort_and_upgrade_lock.
sql/sql_base.cc:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Changed MYSQL_LOCK_IGNORE_FLUSH to not ignore name locks
(open_placeholder).
Merged alter_close_tables into abort_and_upgrade_locks
(and added _and_close_table to the name)
to not release LOCK_open between remove_table_from_cache
and close_data_files_and_morph_locks.
Added DEBUG_SYNC for deterministic testing.
sql/sql_partition.cc:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Removed alter_close_tables, (merged it into
abort_and_upgrad_lock) so that LOCK_open never is released
between remove_table_from_cache and
close_data_files_and_morph_locks.
sql/sql_show.cc:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Added DEBUG_SYNC for deterministic testing
In BUG#49562 we fixed the case where numeric user var events
would not serialize the flag stating whether the value was signed
or unsigned (unsigned_flag). This fixed the case that the slave
would get an overflow while treating the unsigned values as
signed.
In this bug, we find that the unsigned_flag can sometimes change
between the moment that the user value is recorded for binlogging
purposes and the actual binlogging time. Since we take the
unsigned_flag from the runtime variable data, at binlogging time,
and the variable value is comes from the copy taken earlier in
the execution, there may be inconsistency in the
User_var_log_event between the variable value and its
unsigned_flag.
We fix this by also copying the unsigned_flag of the
user_var_entry when its value is copied, for binlogging
purposes. Later, at binlogging time, we use the copied
unsigned_flag and not the one in the runtime user_var_entry
instance.
Non-determinism of the test was caused by lack of setting a proper value to hb period,
actually fixed by BUG@50767.
These fixes aim at possible non-determinism in comparison of received
hb events by master and slave in the circular part of the test.
Even though the HB periods ratio was choosen to be as high as 10, it's still incorrect
to compare number of hb-events basing only a relation between their periods.
Yet another issue is relatively short 60 secs timeout of wait_for_status_var.inc
makes valgrind runs to fail.
Fixed with deploying wait_for_slave_io_to_start afront of calling wait_for_status_var.
The test is made runnable only with MIXED binlog-format as it has close to 1 min
total exec time and there is nothing format specific in it.
mysql-test/suite/rpl/r/rpl_heartbeat_basic.result:
results are changed.
mysql-test/suite/rpl/t/rpl_heartbeat_basic.test:
Reducing the test env to run in only with MIXED mode;
Simplifying logics of the circular setup to verify only
that HB flows both directions.
Conflicts:
Text conflict in client/mysqlbinlog.cc
Text conflict in mysql-test/r/explain.result
Text conflict in mysql-test/r/subselect.result
Text conflict in mysql-test/r/subselect3.result
Text conflict in mysql-test/r/type_datetime.result
Text conflict in sql/share/Makefile.am
Split rpl_row_charset into:
- rpl_row_utf16.
- rpl_row_utf32.
This way these tests can run independently if server supports
either one of the charsets but not both.
Cleaned up rpl_row_utf32 which had a spurious instruction:
-- let $reset_slave_type_conversions= 0
In BUG#51787 we were using the wrong charset to print out the
data. We were using the field charset for the string that would
hold the information. This caused the assertion, because the
string length was not aligned with UTF32 bytes requirements for
storage.
We fix this by using &my_charset_latin1 in the string object
instead of the field->charset(). As a side-effect, we needed to
extend the show_sql_type interface so that it took the field
charset is now passed as a parameter, so that one is able to
calculate the correct field size.
In BUG#51716 we had issues with Field_string::pack and
Field_string::unpack. When packing, the length was incorrectly
calculated. When unpacking, the padding the string would be
padded with the wrong bytes (a few bytes less than it should).
We fix this by resorting to charset abstractions (functions) that
calculate the correct length when packing and pad correctly the
string when unpacking.
The test case crashes and recovers the server twice, and while
recovering InnoDB sets innodb_file_format_check to different
format on second restart. This makes the test case clean up
check to fail.
We fix this by deploying the option "--innodb_file_format_check='ON'"
in slave startup configuration. This makes the innodb_file_format_check
to not change on secod restart.
The test case added in previous patch missed a RESET MASTER on
test start up. Without it, showing binary log contents can
sometimes show spurious entries from previously executed tests,
ultimately causing test failure - result mismatch.
The test file was added in:
revid:luis.soares@sun.com-20100224190153-k0bpdx9abe88uoo2
This patch also moves the test case into binlog_innodb_row.test
file. This way we avoid having yet another test file,
binlog_row_innodb_truncate.test, whose only purpose is to host
one test case. This had been actually suggested during original
patch review, but somehow the binlog_innodb_row was missed when
searching for a file to host the test case.
Before this fix, the performance schema instrumentation
in mdl.h / mdl.cc was incomplete, causing:
- build warnings,
- no data collection for the performance schema
This fix:
- added instrumentation helpers for the new preferred
reader read write lock, mysql_prlock_*
- implemented completely the performance schema
instrumentation of mdl.h / mdl.cc
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/r/explain.result
Text conflict in mysql-test/r/having.result
Text conflict in mysql-test/suite/rpl/t/disabled.def
Text conflict in mysql-test/suite/rpl/t/rpl_slave_skip.test
Text conflict in storage/federated/ha_federated.cc
Before this fix, mysql_upgrade would always drop and re create
the performance_schema database.
This in theory could destroy user data created using 5.1 or older versions.
With this fix, mysql_upgrade checks the content of the
performance_schema database before droping it.
had broken the 5.1 behaviour of --log-error: --log-error without argument sent to stderr instead of writing
to a file with an autogenerated name.
mysql-test/suite/sys_vars/t/log_error_func.test:
test that error log is created and shown in SHOW VARIABLES.
Interestingly the error log's path is apparently relative if --log-error=argument is used, but
may be absolute or relative if --log-error(no argument) is used (because then the path is derived from
that of pidfile_name, which can be absolute or relative, depending on if autogenerated or not).
mysql-test/suite/sys_vars/t/log_error_func2.test:
test that error log is created and shown in SHOW VARIABLES
mysql-test/suite/sys_vars/t/log_error_func3.test:
test that error log is empty in SHOW VARIABLES
sql/mysql_priv.h:
id for option --log-error
sql/mysqld.cc:
No --log-error means "write errors to stderr", whereas --log-error
without argument means "write errors to a file". So we cannot use the default logic
of class sys_var_charptr, which treats "option not used" the same as "option used
without argument" and uses the same default for both. We need to catch "option used",
in mysqld_get_one_option(), and then "without argument". Setting to "" makes sure
that init_server_components() will create the log, with an autogenerated name.
sql/sys_vars.cc:
need to give the option a numeric id so that we can catch it in mysqld_get_one_option()
auto_increment on duplicate entry
The bug was that when INSERT_ID was used and the storage
engine was told to release any reserved but not used
auto_increment values, it set the highest auto_increment
value to INSERT_ID.
The fix was to check if the auto_increment value was forced
by user (INSERT_ID) or by slave-thread, i.e. not auto-
generated. So that it is only allowed to release generated
values.
mysql-test/r/partition_error.result:
Bug#50392: insert_id is not reset for partitioned tables
auto_increment on duplicate entry
updated result
mysql-test/suite/parts/inc/partition_auto_increment.inc:
Bug#50392: insert_id is not reset for partitioned tables
auto_increment on duplicate entry
Added test
mysql-test/suite/parts/r/partition_auto_increment_archive.result:
Bug#50392: insert_id is not reset for partitioned tables
auto_increment on duplicate entry
Added result, note that archive does only allow increasing
auto_increment values
mysql-test/suite/parts/r/partition_auto_increment_blackhole.result:
Bug#50392: insert_id is not reset for partitioned tables
auto_increment on duplicate entry
Added result, note that blackhole accepts all inserts :)
mysql-test/suite/parts/r/partition_auto_increment_innodb.result:
Bug#50392: insert_id is not reset for partitioned tables
auto_increment on duplicate entry
Added result, note that innodb rolls back inserts on error,
but keeps the auto_increment value.
mysql-test/suite/parts/r/partition_auto_increment_memory.result:
Bug#50392: insert_id is not reset for partitioned tables
auto_increment on duplicate entry
Added result, note that memory and myisam inserts all rows
before the error.
mysql-test/suite/parts/r/partition_auto_increment_myisam.result:
Bug#50392: insert_id is not reset for partitioned tables
auto_increment on duplicate entry
Added result, note that memory and myisam inserts all rows
before the error.
mysql-test/suite/parts/r/partition_auto_increment_ndb.result:
Bug#50392: insert_id is not reset for partitioned tables
auto_increment on duplicate entry
Added result, note that NDB does not seem to handle
INSERT_ID as other engines. (Martin will look into it).
mysql-test/t/partition_error.test:
Bug#50392: insert_id is not reset for partitioned tables
auto_increment on duplicate entry
Added test
sql/ha_partition.cc:
Bug#50392: insert_id is not reset for partitioned tables
auto_increment on duplicate entry
If the next_insert_id comes from non generated (i.e
forced by INSERT_ID or slave-thread) then we cannot
lower the reserved auto_increment value, since it have
not reserved any values.
mode
When the master was executing in sql_mode='traditional' (which
implies that really_abort_on_warning returns TRUE - because of
MODE_STRICT_ALL_TABLES), the error code (ER_DUP_ENTRY in the
reported case) was not being set in the
Query_log_event. Therefore, even if a failure was to be expected
when replaying the statement on the slave, a failure would occur,
because the Query_log_event was not transporting the expected
error code, but 0 instead.
This was because when the master was getting the error code to
set it in the Query_log_event, the executing thread would be
assumed to have been killed:
THD::killed==THD::KILL_BAD_DATA. This would make the error code
fetch routine not to check thd->main_da.sql_errno(), but instead
the thd->killed value. What's more, is that the server would
thd->killed value if thd->killed == THD::KILL_BAD_DATA and return
0 instead. So this is a double inconsistency, as the we should
not even check thd->killed but rather thd->main_da.sql_errno().
We fix this by extending the condition used to choose whether to
check the thd->main_da.sql_errno() or thd->killed, so that it
takes into consideration the case when:
thd->killed==THD::KILL_BAD_DATA.
+ failing statements
Implicit DROP event for temporary table is not getting
LOG_EVENT_THREAD_SPECIFIC_F flag, because, in the previous
executed statement in the same thread, which might even be a
failed statement, the thread_specific_used flag is set to
FALSE (in mysql_reset_thd_for_next_command) and not set to TRUE
before connection is shutdown. This means that implicit DROP
event will take the FALSE value from thread_specific_used and
will not set LOG_EVENT_THREAD_SPECIFIC_F in the event header. As
a consequence, mysqlbinlog will not print the pseudo_thread_id
from the DROP event, because one of the requirements for the
printout is that this flag is set to TRUE.
We fix this by setting thread_specific_used whenever we are
binlogging a DROP in close_temporary_tables, and resetting it to
its previous value afterward.
START SLAVE UNTIL MASTER ... specifies only SQL thread to stop.
rpl_slave_skip erronously deployed waiting for stop of both threads.
Corrected with deploying the correct macro.
Notice, earlier a similar bug@47749 was fixed in mysql-trunk.
mysql-test/suite/rpl/t/rpl_slave_skip.test:
changing two waiting calls for offline of both threads into waiting for SQL to stop.
