The ALTER PARTITION and SELECT seemed to be deadlocked
when having innodb_thread_concurrency = 1.
Problem was that there was unreleased latches
in the ALTER PARTITION thread which was needed
by the SELECT thread to be able to continue.
Solution was to release the latches by commit
before requesting upgrade to exclusive MDL lock.
Updated according to reviewers comments (3).
mysql-test/r/partition_innodb.result:
updated test result
mysql-test/t/partition_innodb.test:
added test
sql/sql_partition.cc:
Moved implicit commit into mysql_change_partition
so that if latches are taken, they are always released
before waiting on exclusive lock.
sql/sql_table.cc:
refactored the code to prepare and commit
around copy_data_between_tables, to be able
to reuse it in mysql_change_partitions
sql/sql_table.h:
exporting mysql_trans_prepare/commit_alter_copy_data
corruption on ADD PARTITION and LOCK TABLE
Bug#53770: Server crash at handler.cc:2076 on
LOAD DATA after timed out COALESCE PARTITION
5.5 fix for:
Bug#51042: REORGANIZE PARTITION can leave table in an
inconsistent state in case of crash
Needs to be back-ported to 5.1
5.5 fix for:
Bug#50418: DROP PARTITION does not interact with
transactions
Main problem was non-persistent operations done
before meta-data lock was taken (53770+53676).
And 53676 needed to keep the table/partitions opened and locked
while copying the data to the new partitions.
Also added thorough tests to spot some additional bugs
in the ddl_log code, which could result in bad state
between the .frm and partitions.
Collapsed patch, includes all fixes required from the reviewers.
mysql-test/r/partition_innodb.result:
updated result with new test
mysql-test/suite/parts/inc/partition_crash.inc:
crash test include file
mysql-test/suite/parts/inc/partition_crash_add.inc:
test all states in fast_alter_partition_table
ADD PARTITION branch
mysql-test/suite/parts/inc/partition_crash_change.inc:
test all states in fast_alter_partition_table
CHANGE PARTITION branch
mysql-test/suite/parts/inc/partition_crash_drop.inc:
test all states in fast_alter_partition_table
DROP PARTITION branch
mysql-test/suite/parts/inc/partition_fail.inc:
recovery test including injecting errors
mysql-test/suite/parts/inc/partition_fail_add.inc:
test all states in fast_alter_partition_table
ADD PARTITION branch
mysql-test/suite/parts/inc/partition_fail_change.inc:
test all states in fast_alter_partition_table
CHANGE PARTITION branch
mysql-test/suite/parts/inc/partition_fail_drop.inc:
test all states in fast_alter_partition_table
DROP PARTITION branch
mysql-test/suite/parts/inc/partition_mgm_crash.inc:
include file that runs all crash and failure injection tests.
mysql-test/suite/parts/r/partition_debug_innodb.result:
new test result file
mysql-test/suite/parts/r/partition_debug_myisam.result:
new test result file
mysql-test/suite/parts/r/partition_special_innodb.result:
updated result
mysql-test/suite/parts/r/partition_special_myisam.result:
updated result
mysql-test/suite/parts/t/partition_debug_innodb-master.opt:
opt file for using with crashing tests of partitioned innodb
mysql-test/suite/parts/t/partition_debug_innodb.test:
partitioned innodb test that require debug builds
mysql-test/suite/parts/t/partition_debug_myisam-master.opt:
opt file for using with crashing tests of partitioned myisam
mysql-test/suite/parts/t/partition_debug_myisam.test:
partitioned myisam test that require debug builds
mysql-test/suite/parts/t/partition_special_innodb-master.opt:
added innodb-file-per-table to easier verify partition status on disk
mysql-test/suite/parts/t/partition_special_innodb.test:
added test case
mysql-test/suite/parts/t/partition_special_myisam.test:
added test case
mysql-test/t/partition_innodb.test:
added test case
sql/sql_base.cc:
Moved alter_close_tables to sql_partition.cc
sql/sql_base.h:
removed some non existing and duplicated functions.
sql/sql_partition.cc:
fast_alter_partition_table:
Spletted abort_and_upgrad_lock_and_close_table
to its parts (wait_while_table_is_used and
alter_close_tables) and always have
wait_while_table_is_used before any persistent
operations (including logs, which will be executed
on failure) and alter_close_tables after
create/read/write operations and before
drop operations.
moved alter_close_tables here from sql_base.cc
Added error injections for better test coverage.
write_log_final_change_partition:
fixed a log_entry linking bug (delete_frm was not
linked to change/drop partition)
and drop partition must be executed before
change partition (change partition can rename a
partition to an old name, like REORG p1 INTO (p1,p2).
write_log_add_change_partition:
need to use drop_frm first, and relinking that entry
and reusing its execute entry.
sql/sql_table.cc:
added initialization of next_active_log_entry.
sql/table.h:
removed a duplicate declaration.
/*![:version:] Query Code */, where [:version:] is a sequence of 5
digits representing the mysql server version(e.g /*!50200 ... */),
is a special comment that the query in it can be executed on those
servers whose versions are larger than the version appearing in the
comment. It leads to a security issue when slave's version is larger
than master's. A malicious user can improve his privileges on slaves.
Because slave SQL thread is running with SUPER privileges, so it can
execute queries that he/she does not have privileges on master.
This bug is fixed with the logic below:
- To replace '!' with ' ' in the magic comments which are not applied on
master. So they become common comments and will not be applied on slave.
- Example:
'INSERT INTO t1 VALUES (1) /*!10000, (2)*/ /*!99999 ,(3)*/
will be binlogged as
'INSERT INTO t1 VALUES (1) /*!10000, (2)*/ /* 99999 ,(3)*/
mysql-test/suite/rpl/t/rpl_conditional_comments.test:
Test the patch for this bug.
sql/mysql_priv.h:
Rename inBuf as rawBuf and remove the const limitation.
sql/sql_lex.cc:
To replace '!' with ' ' in the magic comments which are not applied on
master.
sql/sql_lex.h:
Remove the const limitation on parameter buff, as it can be modified in the function since
this patch.
Add member function yyUnput for Lex_input_stream. It set a character back the query buff.
sql/sql_parse.cc:
Rename inBuf as rawBuf and remove the const limitation.
sql/sql_partition.cc:
Remove the const limitation on parameter part_buff, as it can be modified in the function since
this patch.
sql/sql_partition.h:
Remove the const limitation on parameter part_buff, as it can be modified in the function since
this patch.
sql/table.h:
Remove the const limitation on variable partition_info, as it can be modified since
this patch.
This patch also fixes Bug#55452 "SET PASSWORD is
replicated twice in RBR mode".
The goal of this patch is to remove the release of
metadata locks from close_thread_tables().
This is necessary to not mistakenly release
the locks in the course of a multi-step
operation that involves multiple close_thread_tables()
or close_tables_for_reopen().
On the same token, move statement commit outside
close_thread_tables().
Other cleanups:
Cleanup COM_FIELD_LIST.
Don't call close_thread_tables() in COM_SHUTDOWN -- there
are no open tables there that can be closed (we leave
the locked tables mode in THD destructor, and this
close_thread_tables() won't leave it anyway).
Make open_and_lock_tables() and open_and_lock_tables_derived()
call close_thread_tables() upon failure.
Remove the calls to close_thread_tables() that are now
unnecessary.
Simplify the back off condition in Open_table_context.
Streamline metadata lock handling in LOCK TABLES
implementation.
Add asserts to ensure correct life cycle of
statement transaction in a session.
Remove a piece of dead code that has also become redundant
after the fix for Bug 37521.
mysql-test/r/variables.result:
Update results: set @@autocommit and statement transaction/
prelocked mode.
mysql-test/r/view.result:
A harmless change in CHECK TABLE <view> status for a broken view.
If previously a failure to prelock all functions used in a view
would leave the connection in LTM_PRELOCKED mode, now we call
close_thread_tables() from open_and_lock_tables()
and leave prelocked mode, thus some check in mysql_admin_table() that
works only in prelocked/locked tables mode is no longer activated.
mysql-test/suite/rpl/r/rpl_row_implicit_commit_binlog.result:
Fixed Bug#55452 "SET PASSWORD is replicated twice in
RBR mode": extra binlog events are gone from the
binary log.
mysql-test/t/variables.test:
Add a test case: set autocommit and statement transaction/prelocked
mode.
sql/event_data_objects.cc:
Simplify code in Event_job_data::execute().
Move sp_head memory management to lex_end().
sql/event_db_repository.cc:
Move the release of metadata locks outside
close_thread_tables().
Make sure we call close_thread_tables() when
open_and_lock_tables() fails and remove extra
code from the events data dictionary.
Use close_mysql_tables(), a new internal
function to properly close mysql.* tables
in the data dictionary.
Contract Event_db_repository::drop_events_by_field,
drop_schema_events into one function.
When dropping all events in a schema,
make sure we don't mistakenly release all
locks acquired by DROP DATABASE. These
include locks on the database name
and the global intention exclusive
metadata lock.
sql/event_db_repository.h:
Function open_event_table() does not require an instance
of Event_db_repository.
sql/events.cc:
Use close_mysql_tables() instead of close_thread_tables()
to bootstrap events, since the latter no longer
releases metadata locks.
sql/ha_ndbcluster.cc:
- mysql_rm_table_part2 no longer releases
acquired metadata locks. Do it in the caller.
sql/ha_ndbcluster_binlog.cc:
Deploy the new protocol for closing thread
tables in run_query() and ndb_binlog_index
code.
sql/handler.cc:
Assert that we never call ha_commit_trans/
ha_rollback_trans in sub-statement, which
is now the case.
sql/handler.h:
Add an accessor to check whether THD_TRANS object
is empty (has no transaction started).
sql/log.cc:
Update a comment.
sql/log_event.cc:
Since now we commit/rollback statement transaction in
mysql_execute_command(), we need a mechanism to communicate
from Query_log_event::do_apply_event() to mysql_execute_command()
that the statement transaction should be rolled back, not committed.
Ideally it would be a virtual method of THD. I hesitate
to make THD a virtual base class in this already large patch.
Use a thd->variables.option_bits for now.
Remove a call to close_thread_tables() from the slave IO
thread. It doesn't open any tables, and the protocol
for closing thread tables is more complicated now.
Make sure we properly close thread tables, however,
in Load_data_log_event, which doesn't
follow the standard server execution procedure
with mysql_execute_command().
@todo: this piece should use Server_runnable
framework instead.
Remove an unnecessary call to mysql_unlock_tables().
sql/rpl_rli.cc:
Update Relay_log_info::slave_close_thread_tables()
to follow the new close protocol.
sql/set_var.cc:
Remove an unused header.
sql/slave.cc:
Remove an unnecessary call to
close_thread_tables().
sql/sp.cc:
Remove unnecessary calls to close_thread_tables()
from SP DDL implementation. The tables will
be closed by the caller, in mysql_execute_command().
When dropping all routines in a database, make sure
to not mistakenly drop all metadata locks acquired
so far, they include the scoped lock on the schema.
sql/sp_head.cc:
Correct the protocol that closes thread tables
in an SP instruction.
Clear lex->sphead before cleaning up lex
with lex_end to make sure that we don't
delete the sphead twice. It's considered
to be "cleaner" and more in line with
future changes than calling delete lex->sphead
in other places that cleanup the lex.
sql/sp_head.h:
When destroying m_lex_keeper of an instruction,
don't delete the sphead that all lex objects
share.
@todo: don't store a reference to routine's sp_head
instance in instruction's lex.
sql/sql_acl.cc:
Don't call close_thread_tables() where the caller will
do that for us.
Fix Bug#55452 "SET PASSWORD is replicated twice in RBR
mode" by disabling RBR replication in change_password()
function.
Use close_mysql_tables() in bootstrap and ACL reload
code to make sure we release all metadata locks.
sql/sql_base.cc:
This is the main part of the patch:
- remove manipulation with thd->transaction
and thd->mdl_context from close_thread_tables().
Now this function is only responsible for closing
tables, nothing else.
This is necessary to be able to easily use
close_thread_tables() in procedures, that
involve multiple open/close tables, which all
need to be protected continuously by metadata
locks.
Add asserts ensuring that TABLE object
is only used when is protected by a metadata lock.
Simplify the back off condition of Open_table_context,
we no longer need to look at the autocommit mode.
Make open_and_lock_tables() and open_normal_and_derived_tables()
close thread tables and release metadata locks acquired so-far
upon failure. This simplifies their usage.
Implement close_mysql_tables().
sql/sql_base.h:
Add declaration for close_mysql_tables().
sql/sql_class.cc:
Remove a piece of dead code that has also become redundant
after the fix for Bug 37521.
The code became dead when my_eof() was made a non-protocol method,
but a method that merely modifies the diagnostics area.
The code became redundant with the fix for Bug#37521, when
we started to cal close_thread_tables() before
Protocol::end_statement().
sql/sql_do.cc:
Do nothing in DO if inside a substatement
(the assert moved out of trans_rollback_stmt).
sql/sql_handler.cc:
Add comments.
sql/sql_insert.cc:
Remove dead code.
Release metadata locks explicitly at the
end of the delayed insert thread.
sql/sql_lex.cc:
Add destruction of lex->sphead to lex_end(),
lex "reset" method called at the end of each statement.
sql/sql_parse.cc:
Move close_thread_tables() and other related
cleanups to mysql_execute_command()
from dispatch_command(). This has become
possible after the fix for Bug#37521.
Mark federated SERVER statements as DDL.
Next step: make sure that we don't store
eof packet in the query cache, and move
the query cache code outside mysql_parse.
Brush up the code of COM_FIELD_LIST.
Remove unnecessary calls to close_thread_tables().
When killing a query, don't report "OK"
if it was a suicide.
sql/sql_parse.h:
Remove declaration of a function that is now static.
sql/sql_partition.cc:
Remove an unnecessary call to close_thread_tables().
sql/sql_plugin.cc:
open_and_lock_tables() will clean up
after itself after a failure.
Move close_thread_tables() above
end: label, and replace with close_mysql_tables(),
which will also release the metadata lock
on mysql.plugin.
sql/sql_prepare.cc:
Now that we no longer release locks in close_thread_tables()
statement prepare code has become more straightforward.
Remove the now redundant check for thd->killed() (used
only by the backup project) from Execute_server_runnable.
Reorder code to take into account that now mysql_execute_command()
performs lex->unit.cleanup() and close_thread_tables().
sql/sql_priv.h:
Add a new option to server options to interact
between the slave SQL thread and execution
framework (hack). @todo: use a virtual
method of class THD instead.
sql/sql_servers.cc:
Due to Bug 25705 replication of
DROP/CREATE/ALTER SERVER is broken.
Make sure at least we do not attempt to
replicate these statements using RBR,
as this violates the assert in close_mysql_tables().
sql/sql_table.cc:
Do not release metadata locks in mysql_rm_table_part2,
this is done by the caller.
Do not call close_thread_tables() in mysql_create_table(),
this is done by the caller.
Fix a bug in DROP TABLE under LOCK TABLES when,
upon error in wait_while_table_is_used() we would mistakenly
release the metadata lock on a non-dropped table.
Explicitly release metadata locks when doing an implicit
commit.
sql/sql_trigger.cc:
Now that we delete lex->sphead in lex_end(),
zero the trigger's sphead in lex after loading
the trigger, to avoid double deletion.
sql/sql_udf.cc:
Use close_mysql_tables() instead of close_thread_tables().
sql/sys_vars.cc:
Remove code added in scope of WL#4284 which would
break when we perform set @@session.autocommit along
with setting other variables and using tables or functions.
A test case added to variables.test.
sql/transaction.cc:
Add asserts.
sql/tztime.cc:
Use close_mysql_tables() rather than close_thread_tables().
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
client/mysqldump.c:
Pass my_free directly as its signature is compatible with the
callback type -- which wasn't the case for free_table_ent.
Add -Wall to gcc/g++
Fix most warnings reported in dbg and opt mode.
cmd-line-utils/libedit/filecomplete.c:
Remove unused auto variables.
configure.cmake:
Add -Wall to gcc.
extra/comp_err.c:
Cast to correct type.
extra/perror.c:
Fix segfault (but warnings about deprecated features remain)
extra/yassl/taocrypt/include/runtime.hpp:
Comparing two literals was reported as undefined behaviour.
include/my_global.h:
Add a template for aligning character buffers.
mysys/lf_alloc-pin.c:
Initialize pointer.
sql/mysqld.cc:
Use UNINIT_VAR rather than LINT_INIT.
sql/partition_info.cc:
Use UNINIT_VAR rather than LINT_INIT.
sql/rpl_handler.cc:
Use char[] rather than unsigned long[] array for placement buffer.
sql/spatial.cc:
Use char[] rather than unsigned void*[] array for placement buffer.
sql/spatial.h:
Use char[] rather than unsigned void*[] array for placement buffer.
sql/sql_partition.cc:
Initialize auto variable.
sql/sql_table.cc:
Initialize auto variables.
Add parens around assignment within if()
sql/sys_vars.cc:
Use UNINIT_VAR.
storage/innobase/os/os0file.c:
Init first slot in auto variable.
storage/myisam/mi_create.c:
Use UNINIT_VAR rather than LINT_INIT.
storage/myisam/mi_open.c:
Remove (wrong) casting.
storage/myisam/mi_page.c:
Remove (wrong) casting.
storage/myisam/mi_search.c:
Cast to uchar* rather than char*.
strings/ctype-ucs2.c:
Use UNINIT_VAR rather than LINT_INIT.
Add (uchar*) casting.
Item_hex_string::Item_hex_string
The status of memory allocation in the Lex_input_stream (called
from the Parser_state constructor) was not checked which led to
a parser crash in case of the out-of-memory error.
The solution is to introduce new init() member function in
Parser_state and Lex_input_stream so that status of memory
allocation can be returned to the caller.
mysql-test/r/error_simulation.result:
Added a test case for bug #42064.
mysql-test/t/error_simulation.test:
Added a test case for bug #42064.
mysys/my_alloc.c:
Added error injection code for the regression test.
mysys/my_malloc.c:
Added error injection code for the regression test.
mysys/safemalloc.c:
Added error injection code for the regression test.
sql/event_data_objects.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/mysqld.cc:
Added error injection code for the regression test.
sql/sp.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/sql_lex.cc:
Moved memory allocation from constructor to the separate init()
member function.
Added error injection code for the regression test.
sql/sql_lex.h:
Moved memory allocation from constructor to the separate init()
member function.
sql/sql_parse.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/sql_partition.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/sql_prepare.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/sql_trigger.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures.
sql/sql_view.cc:
Use the new init() member function of Parser_state and check
its return value to handle memory allocation failures..
sql/thr_malloc.cc:
Added error injection code for the regression test.
partition if muliple columns used
Problem was that range scanning through the sorted array of
the column list values did not use a correct index calculation.
Fixed by also taking the number of columns in the calculation.
mysql-test/r/partition_column.result:
Bug#52815: LIST COLUMNS doesn't insert rows in correct
partition if muliple columns used
Added test result for bug.
mysql-test/t/partition_column.test:
Bug#52815: LIST COLUMNS doesn't insert rows in correct
partition if muliple columns used
Added test for bug.
sql/sql_partition.cc:
Bug#52815: LIST COLUMNS doesn't insert rows in correct
partition if muliple columns used
list_col_array is a matrix of field values, so one must also
use the number of columns to calculate the correct partition
id.
This patch:
- Moves all definitions from the mysql_priv.h file into
header files for the component where the variable is
defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h
into partitioned MyISAM table
Problem was that the ha_data structure was introduced in 5.1
and only used for partitioning first, but with the intention
of be of use for others engines as well, and when used by other
engines it would clash if it also was partitioned.
Solution is to move the partitioning specific data to a separate
structure, with its own mutex (which is used for auto_increment).
Also did rename PARTITION_INFO to PARTITION_STATS since there
already exist a class named partition_info, also cleaned up
some related variables.
mysql-test/r/partition_binlog_stmt.result:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
New result file
mysql-test/t/partition_binlog_stmt.test:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
New result file
sql/ha_ndbcluster.cc:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Rename of PARTITION_INFO to PARTITION_STATS to better
match the use (and there is also a class named
partition_info...)
sql/ha_ndbcluster.h:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Rename of PARTITION_INFO to PARTITION_STATS to better
match the use (and there is also a class named
partition_info...)
sql/ha_partition.cc:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Removed the partitioning engines use of ha_data in
TABLE_SHARE and added ha_part_data instead, since
they collide if used in the same time.
Rename of PARTITION_INFO to PARTITION_STATS to better
match the use (and there is also a class named
partition_info...)
Removed some dead code.
sql/ha_partition.h:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Removed some dead code.
Rename of PARTITION_INFO to PARTITION_STATS to better
match the use (and there is also a class named
partition_info...)
Removed the partitioning engines use of ha_data in
TABLE_SHARE and added ha_part_data instead, since
they collide if used in the same time.
sql/handler.cc:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Rename of PARTITION_INFO to PARTITION_STATS to better
match the use (and there is also a class named
partition_info...)
sql/handler.h:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Rename of PARTITION_INFO to PARTITION_STATS to better
match the use (and there is also a class named
partition_info...)
sql/mysql_priv.h:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Removed the partitioning engines use of ha_data in
TABLE_SHARE and added ha_part_data instead, since
they collide if used in the same time.
Added key_PARTITION_LOCK_auto_inc for instrumentation.
sql/mysqld.cc:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Removed the partitioning engines use of ha_data in
TABLE_SHARE and added ha_part_data instead, since
they collide if used in the same time.
Added key_PARTITION_LOCK_auto_inc for instrumentation.
sql/partition_info.h:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Removed part_state* since it was not in use.
sql/sql_partition.cc:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Removed part_state* since it was not in use.
sql/sql_partition.h:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Cleaned up old commented out code.
Removed part_state* since it was not in use.
sql/sql_show.cc:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Rename of PARTITION_INFO to PARTITION_STATS to better
match the use (and there is also a class named
partition_info...)
Renamed partition_info to partition_info_str, since
partition_info is a name of a class.
sql/sql_table.cc:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Renamed partition_info to partition_info_str, since
partition_info is a name of a class.
sql/table.cc:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Removed the partitioning engines use of ha_data in
TABLE_SHARE and added ha_part_data instead, since
they collide if used in the same time.
Renamed partition_info to partition_info_str, since
partition_info is a name of a class.
removed part_state* since it was not in use.
sql/table.h:
Bug#51851: Server with SBR locks mutex twice on LOAD DATA
into partitioned MyISAM table
Removed the partitioning engines use of ha_data in
TABLE_SHARE and added ha_part_data instead, since
they collide if used in the same time.
Renamed partition_info to partition_info_str, since
partition_info is a name of a class.
removed part_state* since it was not in use.
Conflicts:
Text conflict in client/mysqlbinlog.cc
Text conflict in mysql-test/Makefile.am
Text conflict in mysql-test/collections/default.daily
Text conflict in mysql-test/r/mysqlbinlog_row_innodb.result
Text conflict in mysql-test/suite/rpl/r/rpl_typeconv_innodb.result
Text conflict in mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test
Text conflict in mysql-test/suite/rpl/t/rpl_row_create_table.test
Text conflict in mysql-test/suite/rpl/t/rpl_slave_skip.test
Text conflict in mysql-test/suite/rpl/t/rpl_typeconv_innodb.test
Text conflict in mysys/charset.c
Text conflict in sql/field.cc
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_func.cc
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/rpl_utility.cc
Text conflict in sql/rpl_utility.h
Text conflict in sql/set_var.cc
Text conflict in sql/share/Makefile.am
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_plugin.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_table.cc
Text conflict in storage/example/ha_example.h
Text conflict in storage/federated/ha_federated.cc
Text conflict in storage/myisammrg/ha_myisammrg.cc
Text conflict in storage/myisammrg/myrg_open.c
Conflicts:
Text conflict in mysql-test/r/partition_innodb.result
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_cmpfunc.h
Text conflict in sql/item_sum.h
Text conflict in sql/log_event_old.cc
Text conflict in sql/protocol.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_yacc.yy
concurrent I_S query
There were two problem:
1) MYSQL_LOCK_IGNORE_FLUSH also ignored name locks
2) there was a race between abort_and_upgrade_locks and
alter_close_tables
(i.e. remove_table_from_cache and
close_data_files_and_morph_locks)
Which allowed the table to be opened with MYSQL_LOCK_IGNORE_FLUSH flag
resulting in renaming a partition that was already in use,
which could cause the table to be unusable.
Solution was to not allow IGNORE_FLUSH to skip waiting for
a named locked table.
And to not release the LOCK_open mutex between the
calls to remove_table_from_cache and
close_data_files_and_morph_locks by merging the functions
abort_and_upgrade_locks and alter_close_tables.
mysql-test/suite/parts/r/partition_debug_sync_innodb.result:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Added test result
mysql-test/suite/parts/t/partition_debug_sync_innodb-master.opt:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Added test option
mysql-test/suite/parts/t/partition_debug_sync_innodb.test:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Added test file
sql/authors.h:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Time to be acknowledged :)
sql/ha_partition.cc:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Added DEBUG_SYNC for deterministic testing
sql/mysql_priv.h:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Renamed function since merging alter_close_tables into
abort_and_upgrade_lock.
sql/sql_base.cc:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Changed MYSQL_LOCK_IGNORE_FLUSH to not ignore name locks
(open_placeholder).
Merged alter_close_tables into abort_and_upgrade_locks
(and added _and_close_table to the name)
to not release LOCK_open between remove_table_from_cache
and close_data_files_and_morph_locks.
Added DEBUG_SYNC for deterministic testing.
sql/sql_partition.cc:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Removed alter_close_tables, (merged it into
abort_and_upgrad_lock) so that LOCK_open never is released
between remove_table_from_cache and
close_data_files_and_morph_locks.
sql/sql_show.cc:
Bug#50561: ALTER PARTITIONS does not have adequate lock, breaks with
concurrent I_S query
Added DEBUG_SYNC for deterministic testing
(regression)
Problem was that partition pruning did not exclude the
last partition if the range was beyond it
(i.e. not using MAXVALUE)
Fix was to not include the last partition if the
partitioning function value was not within the partition
range.
mysql-test/r/partition_innodb.result:
Bug#51830: Incorrect partition pruning on range partition
(regression)
Updated result
mysql-test/r/partition_pruning.result:
Bug#51830: Incorrect partition pruning on range partition
(regression)
Updated result
mysql-test/t/partition_innodb.test:
Bug#51830: Incorrect partition pruning on range partition
(regression)
Added test for pruning in InnoDB, since it does not show
for MyISAM due to 'Impossible WHERE noticed after reading
const tables'.
mysql-test/t/partition_pruning.test:
Bug#51830: Incorrect partition pruning on range partition
(regression)
Added test
sql/sql_partition.cc:
Bug#51830: Incorrect partition pruning on range partition
(regression)
Also increase the partition id if not inside the last partition
(and no MAXVALUE is defined).
Added comments and DBUG_ASSERT.
There was no check for foreign keys when altering partitioned
tables.
Added check for FK when altering partitioned tables.
mysql-test/r/partition_innodb.result:
Bug#50104: Partitioned table with just 1 partion works with fk
Updated test result
mysql-test/t/partition_innodb.test:
Bug#50104: Partitioned table with just 1 partion works with fk
Added test for adding FK on partitioned tables (both 1 and 2
partitions)
sql/sql_partition.cc:
Bug#50104: Partitioned table with just 1 partion works with fk
Disabled adding foreign key when altering a partitioned table.
on decimal column
The problem was that there was no check to disallow DECIMAL
columns in the code (it was accepted as if it was INTEGER).
Solution was to correctly disallow DECIMAL columns in
COLUMNS partitioning. As documented.
mysql-test/r/partition_column.result:
Bug#51347: assertion with show create table + partition by columns
on decimal column
updated test result
mysql-test/t/partition_column.test:
Bug#51347: assertion with show create table + partition by columns
on decimal column
Added test to verify column types that is not supported
sql/sql_partition.cc:
Bug#51347: assertion with show create table + partition by columns
on decimal column
Moved DECIMAL types to be disallowed as column types in COLUMNS
partitioning
mysql-test/t/disabled.def:
Restore disabled ssl tests: SSL certificates were updated.
Disable sp_sync.test, the test case can't work in next-4284.
mysql-test/t/partition_innodb.test:
Disable parsing of the test case for Bug#47343,
the test can not work in next-4284.
mysql-test/t/ps_ddl.test:
Update results (CREATE TABLE IF NOT EXISTS takes
into account existence of the temporary table).
Cherry-pick a fix Bug#37148 from next-mr, to preserve
file ids of the added files, and ensure that all the necessary
changes have been pulled.
Since initially Bug#37148 was null-merged into 6.0,
the changeset that is now being cherry-picked was likewise
null merged into next-4284.
Now that Bug#37148 has been reapplied to 6.0, try to make
it work with next-4284. This is also necessary to be able
to pull other changes from 5.1-rep into next-4284.
To resolve the merge issues use this changeset applied
to 6.0:
revid:jperkin@sun.com-20091216103628-ylhqf7s6yegui2t9
revno: 3776.1.1
committer: He Zhenxing <zhenxing.he@sun.com>
branch nick: 6.0-codebase-bugfixing
timestamp: Thu 2009-12-17 17:02:50 +0800
message:
Fix merge problem with Bug#37148
REORGANIZE PARTITION
There were several problems which lead to this this,
all related to bad error handling.
1) There was several bugs preventing the ddl-log to be used for
cleaning up created files on error.
2) The error handling after the copy partition rows did not close
and unlock the tables, resulting in deletion of partitions
which were in use, which lead InnoDB to put the partition to
drop in a background queue.
sql/ha_partition.cc:
Bug#47343: InnoDB fails to clean-up after lock wait timeout on
REORGANIZE PARTITION
Better error handling, if partition has been created/opened/locked
then make sure it is unlocked and closed before returning error.
The delete of the newly created partition is handled by the ddl-log.
sql/sql_parse.cc:
Bug#47343: InnoDB fails to clean-up after lock wait timeout on
REORGANIZE PARTITION
Fix a bug found when experimenting, thd could really be NULL here,
as mentioned in the function header.
sql/sql_partition.cc:
Bug#47343: InnoDB fails to clean-up after lock wait timeout on
REORGANIZE PARTITION
Used the correct .frm shadow name to put into the ddl-log.
Really use the ddl-log to handle errors.
sql/sql_table.cc:
Bug#47343: InnoDB fails to clean-up after lock wait timeout on
REORGANIZE PARTITION
Fixes of the ddl-log when used as error recovery (no crash).
When executing an entry from memory (not read from disk)
the name_len was not set correctly.
Problem was when calculating the range of partitions for
pruning.
Solution was to get the calculation correct. I also simplified
it a bit for easier understanding.
mysql-test/r/partition_pruning.result:
Bug#49742: Partition Pruning not working correctly for RANGE
Added results.
mysql-test/t/partition_pruning.test:
Bug#49742: Partition Pruning not working correctly for RANGE
Added tests to prevent regressions.
sql/sql_partition.cc:
Bug#49742: Partition Pruning not working correctly for RANGE
Simplified calculation for partition id for ranges.
Easier to get right and understand.
Added comments.
Bug#16565 mysqld --help --verbose does not order variablesBug#20413 sql_slave_skip_counter is not shown in show variables
Bug#20415 Output of mysqld --help --verbose is incomplete
Bug#25430 variable not found in SELECT @@global.ft_max_word_len;
Bug#32902 plugin variables don't know their names
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#34829 No default value for variable and setting default does not raise error
Bug#34834 ? Is accepted as a valid sql mode
Bug#34878 Few variables have default value according to documentation but error occurs
Bug#34883 ft_boolean_syntax cant be assigned from user variable to global var.
Bug#37187 `INFORMATION_SCHEMA`.`GLOBAL_VARIABLES`: inconsistent status
Bug#40988 log_output_basic.test succeeded though syntactically false.
Bug#41010 enum-style command-line options are not honoured (maria.maria-recover fails)
Bug#42103 Setting key_buffer_size to a negative value may lead to very large allocations
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#44797 plugins w/o command-line options have no disabling option in --help
Bug#46314 string system variables don't support expressions
Bug#46470 sys_vars.max_binlog_cache_size_basic_32 is broken
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#47212 Crash in DBUG_PRINT in mysqltest.cc when trying to print octal number
Bug#48758 mysqltest crashes on sys_vars.collation_server_basic in gcov builds
Bug#49417 some complaints about mysqld --help --verbose output
Bug#49540 DEFAULT value of binlog_format isn't the default value
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
Bug#49646 mysql --show-warnings crashes when server dies
CMakeLists.txt:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
client/mysql.cc:
don't crash with --show-warnings when mysqld dies
config/ac-macros/plugins.m4:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
include/my_getopt.h:
comments
include/my_pthread.h:
fix double #define
mysql-test/mysql-test-run.pl:
run sys_vars suite by default
properly recognize envirinment variables (e.g. MTR_MAX_SAVE_CORE) set to 0
escape gdb command line arguments
mysql-test/suite/sys_vars/r/rpl_init_slave_func.result:
init_slave+utf8 bug
mysql-test/suite/sys_vars/t/rpl_init_slave_func.test:
init_slave+utf8 bug
mysys/my_getopt.c:
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
mysys/typelib.c:
support for flagset
sql/ha_ndbcluster.cc:
backport from telco tree
sql/item_func.cc:
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
sql/sql_builtin.cc.in:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
sql/sql_plugin.cc:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#32902 plugin variables don't know their names
Bug#44797 plugins w/o command-line options have no disabling option in --help
sql/sys_vars.cc:
all server variables are defined here
storage/myisam/ft_parser.c:
remove unnecessary updates of param->quot
storage/myisam/ha_myisam.cc:
myisam_* variables belong here
strings/my_vsnprintf.c:
%o and %llx
unittest/mysys/my_vsnprintf-t.c:
%o and %llx tests
vio/viosocket.c:
bugfix: fix @@wait_timeout to work with socket timeouts (vs. alarm thread)
timestamp primary key
Since TIMESTAMP values are adjusted by the current time zone
settings in both numeric and string contexts, using any
expressions involving TIMESTAMP values as a
(sub)partitioning function leads to undeterministic behavior of
partitioned tables. The effect may vary depending on a storage
engine, it can be either incorrect data being retrieved or
stored, or an assertion failure. The root cause of this is the
fact that the calculated partition ID may differ from a
previously calculated ID for the same data due to timezone
adjustments of the partitioning expression value.
Fixed by disabling any expressions involving TIMESTAMP values
to be used in partitioning functions with the follwing two
exceptions:
1. Creating or altering into a partitioned table that violates
the above rule is not allowed, but opening existing such tables
results in a warning rather than an error so that such tables
could be fixed.
2. UNIX_TIMESTAMP() is the only way to get a
timezone-independent value from a TIMESTAMP column, because it
returns the internal representation (a time_t value) of a
TIMESTAMP argument verbatim. So UNIX_TIMESTAMP(timestamp_column)
is allowed and should be used to fix existing tables if one
wants to use TIMESTAMP columns with partitioning.
mysql-test/r/partition_bug18198.result:
Corrected the error.
mysql-test/r/partition_error.result:
Corrected error texts.
Added test cases for bug #42849.
mysql-test/t/partition_bug18198.test:
Corrected error code.
mysql-test/t/partition_error.test:
Corrected error codes.
Added test cases for bug #42849.
sql/item.h:
Added is_timezone_dependent_processor() to Item.
sql/item_func.h:
Added has_timestamp_args() and the implementation of
is_timezone_dependent_processor() for Item_func.
sql/item_timefunc.h:
Added is_timezone_dependent_processor() to
Item_func_unix_timestamp.
sql/share/errmsg.txt:
Renamed ER_CONST_EXPR_IN_PARTITION_FUNC_ERROR to
ER_WRONG_EXPR_IN_PARTITION_FUNC_ERROR to better reflect the
meaning. Adjusted the error message.
sql/sql_partition.cc:
Modified fix_fields_part_func() to walk through partitioning
expression tree with is_timezone_dependent_processor() and issue
a warning/error if it depends on the timezone settings.
Changed fix_fields_part_func() to a static function since it is
not used anywhere except sql_partition.cc
sql/sql_partition.h:
Removed the unneeded declaration of fix_fields_part_func()
since it is now a static function.
sql/sql_yacc.yy:
ER_CONST_EXPR_IN_PARTITION_FUNC_ERROR ->
ER_WRONG_EXPR_IN_PARTITION_FUNC_ERROR.
