XA START may cause assertion failure/server crash when it is called
after unilateral roll back issued by the Resource Manager (both
in regular transaction and after XA transaction).
The problem was that rm_error variable wasn't set/reset properly.
Conflicts
=========
Text conflict in .bzr-mysql/default.conf
Text conflict in libmysqld/CMakeLists.txt
Text conflict in libmysqld/Makefile.am
Text conflict in mysql-test/collections/default.experimental
Text conflict in mysql-test/extra/rpl_tests/rpl_row_sp006.test
Text conflict in mysql-test/suite/binlog/r/binlog_tmp_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata_fatal.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_create_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_sp006_InnoDB.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_log.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_circular_simplex.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_sp006.result
Text conflict in mysql-test/t/mysqlbinlog.test
Text conflict in sql/CMakeLists.txt
Text conflict in sql/Makefile.am
Text conflict in sql/log_event_old.cc
Text conflict in sql/rpl_rli.cc
Text conflict in sql/slave.cc
Text conflict in sql/sql_binlog.cc
Text conflict in sql/sql_lex.h
21 conflicts encountered.
NOTE
====
mysql-5.1-rpl-merge has been made a mirror of mysql-next-mr:
- "mysql-5.1-rpl-merge$ bzr pull ../mysql-next-mr"
This is the first cset (merge/...) committed after pulling
from mysql-next-mr.
2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29,
2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and
some other minor revisions.
This patch implements:
WL#4264 "Backup: Stabilize Service Interface" -- all the
server prerequisites except si_objects.{h,cc} themselves (they can
be just copied over, when needed).
WL#4435: Support OUT-parameters in prepared statements.
(and all issues in the initial patches for these two
tasks, that were discovered in pushbuild and during testing).
Bug#39519: mysql_stmt_close() should flush all data
associated with the statement.
After execution of a prepared statement, send OUT parameters of the invoked
stored procedure, if any, to the client.
When using the binary protocol, send the parameters in an additional result
set over the wire. When using the text protocol, assign out parameters to
the user variables from the CALL(@var1, @var2, ...) specification.
The following refactoring has been made:
- Protocol::send_fields() was renamed to Protocol::send_result_set_metadata();
- A new Protocol::send_result_set_row() was introduced to incapsulate
common functionality for sending row data.
- Signature of Protocol::prepare_for_send() was changed: this operation
does not need a list of items, the number of items is fully sufficient.
The following backward incompatible changes have been made:
- CLIENT_MULTI_RESULTS is now enabled by default in the client;
- CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
When a storage engine failed to initialize before allocated slot number,
the slot number would be 0, and when later finalizing this plugin, it would
accidentally unplug the storage engine currently uses slot 0.
This patch fixed this problem by add a new macro value HA_SLOT_UNDEF to
distinguish undefined slot number from slot 0.
There is a missing check for memory allocation failure when allocating
memory for the handlerton structure. If the handlerton init function
tries to de-reference the pointer, it will cause a segmentation fault
and crash the server.
This patch fixes the problem by not calling the init function if memory
allocation failed, and instead prints an informative error message and
reports the error to the caller.
Create temporary InnoDB table fails on case insensitive
filesystems, when lower_case_table_names is 2 (e.g. OS X)
and temporary directory path contains upper case letters.
The problem was that tmpdir prefix was converted to lower
case when table was created, but was passed as is when
table was opened.
Fixed by leaving tmpdir prefix part intact.
bzr branch mysql-5.1-performance-version mysql-trunk # Summit
cd mysql-trunk
bzr merge mysql-5.1-innodb_plugin # which is 5.1 + Innodb plugin
bzr rm innobase # remove the builtin
Next step: build, test fixes.
those keywords do nothing in 5.1 (they are meant for future versions, for example featuring the Maria engine)
so they are here removed from the syntax. Adding those keywords to future versions when needed is:
- WL#5034 "Add TRANSACTIONA=0|1 and PAGE_CHECKSUM=0|1 clauses to CREATE TABLE"
- WL#5037 "New ROW_FORMAT value for CREATE TABLE: PAGE"
Problem was that a failing rename just left the partitions at the state
it was at the failure.
Solution was to try to revert the started rename if a failure occured.
it returns misleading 'table is full'
Innodb returns a misleading error message "table is full"
when the number of active concurrent transactions is greater
than 1024.
Fixed by adding errorcode "ER_TOO_MANY_CONCURRENT_TRXS" to the
error codes. Innodb should return HA_TOO_MANY_CONCURRENT_TRXS
to mysql which is then mapped to ER_TOO_MANY_CONCURRENT_TRXS
Note: testcase is not written as this was reproducible only by
changing innodb code.
NOTE: This undoes changes by BUG#42829 in sql_class.cc:binlog_query().
I will revert the change in a post-push fix (the binlog filter should
be checked in sql_base.cc:decide_logging_format()).
General overview:
The logic for switching to row format when binlog_format=MIXED had
numerous flaws. The underlying problem was the lack of a consistent
architecture.
General purpose of this changeset:
This changeset introduces an architecture for switching to row format
when binlog_format=MIXED. It enforces the architecture where it has
to. It leaves some bugs to be fixed later. It adds extensive tests to
verify that unsafe statements work as expected and that appropriate
errors are produced by problems with the selection of binlog format.
It was not practical to split this into smaller pieces of work.
Problem 1:
To determine the logging mode, the code has to take several parameters
into account (namely: (1) the value of binlog_format; (2) the
capabilities of the engines; (3) the type of the current statement:
normal, unsafe, or row injection). These parameters may conflict in
several ways, namely:
- binlog_format=STATEMENT for a row injection
- binlog_format=STATEMENT for an unsafe statement
- binlog_format=STATEMENT for an engine only supporting row logging
- binlog_format=ROW for an engine only supporting statement logging
- statement is unsafe and engine does not support row logging
- row injection in a table that does not support statement logging
- statement modifies one table that does not support row logging and
one that does not support statement logging
Several of these conflicts were not detected, or were detected with
an inappropriate error message. The problem of BUG#39934 was that no
appropriate error message was written for the case when an engine
only supporting row logging executed a row injection with
binlog_format=ROW. However, all above cases must be handled.
Fix 1:
Introduce new error codes (sql/share/errmsg.txt). Ensure that all
conditions are detected and handled in decide_logging_format()
Problem 2:
The binlog format shall be determined once per statement, in
decide_logging_format(). It shall not be changed before or after that.
Before decide_logging_format() is called, all information necessary to
determine the logging format must be available. This principle ensures
that all unsafe statements are handled in a consistent way.
However, this principle is not followed:
thd->set_current_stmt_binlog_row_based_if_mixed() is called in several
places, including from code executing UPDATE..LIMIT,
INSERT..SELECT..LIMIT, DELETE..LIMIT, INSERT DELAYED, and
SET @@binlog_format. After Problem 1 was fixed, that caused
inconsistencies where these unsafe statements would not print the
appropriate warnings or errors for some of the conflicts.
Fix 2:
Remove calls to THD::set_current_stmt_binlog_row_based_if_mixed() from
code executed after decide_logging_format(). Compensate by calling the
set_current_stmt_unsafe() at parse time. This way, all unsafe statements
are detected by decide_logging_format().
Problem 3:
INSERT DELAYED is not unsafe: it is logged in statement format even if
binlog_format=MIXED, and no warning is printed even if
binlog_format=STATEMENT. This is BUG#45825.
Fix 3:
Made INSERT DELAYED set itself to unsafe at parse time. This allows
decide_logging_format() to detect that a warning should be printed or
the binlog_format changed.
Problem 4:
LIMIT clause were not marked as unsafe when executed inside stored
functions/triggers/views/prepared statements. This is
BUG#45785.
Fix 4:
Make statements containing the LIMIT clause marked as unsafe at
parse time, instead of at execution time. This allows propagating
unsafe-ness to the view.
The problem is that the one phase commit function failed to
properly end a empty transaction. The solution is to ensure
that the transaction cleanup procedure is invoked even for
empty transactions.
The problem: described in the bug report.
The fix:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the first patch, fixing a number
of the warnings, predominantly "suggest using parentheses
around && in ||", and empty for and while bodies.
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the first patch, fixing a number
of the warnings, predominantly "suggest using parentheses
around && in ||", and empty for and while bodies.
The problem is that when a optimization of read-only transactions
(bypass 2-phase commit) was implemented, it removed the code that
reseted the XID once a transaction wasn't active anymore:
sql/sql_parse.cc:
- bzero(&thd->transaction.stmt, sizeof(thd->transaction.stmt));
- if (!thd->active_transaction())
- thd->transaction.xid_state.xid.null();
+ thd->transaction.stmt.reset();
This mostly worked fine as the transaction commit and rollback
functions (in handler.cc) reset the XID once the transaction is
ended. But those functions wouldn't reset the XID in case of
a empty transaction, leading to a assertion when a new starting
a new XA transaction.
The solution is to ensure that the XID state is reset when empty
transactions are ended (by either commit or rollback). This is
achieved by reorganizing the code so that the transaction cleanup
routine is invoked whenever a transaction is ended.
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
innodb_flush_method=wrong_value causes crash
Problem: after a failed plugin initialization, incompletely
initialized data remained in the plugin and handlerton data
structures. These were used later and caused the crash.
Fix: clean-up plugin related data if initialization failed.
Note: no test case added, hand tested.
When a MEMORY table is full the error is returned to client but not written
to error log.
Fixed the handler api to write the error mesage to error log when the table is
full.
Note: No TestCase included as testing the error log is non-trivial.
Cache size is truncated via 32bit ulong in ha_init_key_cache() and
ha_resize_key_cache()
This change fixes the cast to size_t instead of ulong. This cast is safe,
because key_buffer_size parameter is limited to SIZE_T_MAX
Problem was that partitioning cached the table flags.
These flags could change due to TRANSACTION LEVEL changes.
Solution was to remove the cache and always return the table flags
from the first partition (if the handler was initialized).
The problem was that the server did not robustly handle a
unilateral roll back issued by the Resource Manager (RM)
due to a resource deadlock within the transaction branch.
By not acknowledging the roll back, the server (TM) would
eventually corrupt the XA transaction state and crash.
The solution is to mark the transaction as rollback-only
if the RM indicates that it rolled back its branch of the
transaction.
The problem was that the server did not robustly handle a
unilateral roll back issued by the Resource Manager (RM)
due to a resource deadlock within the transaction branch.
By not acknowledging the roll back, the server (TM) would
eventually corrupt the XA transaction state and crash.
The solution is to mark the transaction as rollback-only
if the RM indicates that it rolled back its branch of the
transaction.
collation change made in 5.1.24-rc
Problem: 'CHECK TABLE ... FOR UPGRADE' did not check for
incompatible collation changes made in MySQL 5.1.24-rc.
Fix: add the check.
upgrade from <=5.0.46 to >=5.0.48
Problem: 'check table .. for upgrade' doesn't detect
incompatible collation changes made in 5.0.48.
Fix: check for incompatible collation changes.
The failure was caused by executing a CREATE-SELECT statement that creates a
table in another database than the current one. In row-based logging, the
CREATE statement was written to the binary log without the database, hence
creating the table in the wrong database, causing the following inserts to
fail since the table didn't exist in the given database.
Fixed the bug by adding a parameter to store_create_info() that will make
the function print the database name before the table name and used that
in the calls that write the CREATE statement to the binary log. The database
name is only printed if it is different than the currently selected database.
The output of SHOW CREATE TABLE has not changed and is still printed without
the database name.
"Trigger fired multiple times leads to gaps in auto_increment sequence".
The bug was that if a trigger fired multiple times inside a top
statement (for example top-statement is a multi-row INSERT,
and trigger is ON INSERT), and that trigger inserted into an auto_increment
column, then gaps could be observed in the auto_increment sequence,
even if there were no other users of the database (no concurrency).
It was wrong usage of THD::auto_inc_intervals_in_cur_stmt_for_binlog.
Note that the fix changes "class handler", I'll tell the Storage Engine API team.
This patch contains fixes for two problems:
1. As originally reported, the server crashed on Mac OS X when trying to access
an EXAMPLE table after the EXAMPLE plugin was installed.
It turned out that the dynamically loaded EXAMPLE plugin called the
function hash_earch() from a Mac OS X system library, instead of
hash_earch() from MySQL's mysys library. Makefile.am in storage/example
does not include libmysys. So the Mac OS X linker arranged the hash_search()
function to be linked to the system library when the shared object is
loaded.
One possible solution would be to include libmysys into the linkage of
dynamic plugins. But then we must have a libmysys.so, which must be
used by the server too. This could have a minimal performance impact,
but foremost the change seems to bee too risky at the current state of
MySQL 5.1.
The selected solution is to rename MySQL's hash_search() to my_hash_search()
like it has been done before with hash_insert() and hash_reset().
Since this is the third time, we need to rename a hash_*() function,
I did renamed all hash_*() functions to my_hash_*().
To avoid changing a zillion calls to these functions, and announcing
this to hundreds of developers, I added defines that map the old names
to the new names.
This change is in hash.h and hash.c.
2. The other problem was improper implementation of the handlerton-to-plugin
mapping. We use a fixed-size array to hold a plugin reference for each
handlerton. On every install of a handler plugin, we allocated a new slot
of the array. On uninstall we did not free it. After some uninstall/install
cycles the array overflowed. We did not check for overflow.
One fix is to check for overflow to stop the crashes.
Another fix is to free the array slot at uninstall and search for a free slot
at plugin install.
This change is in handler.cc.
SET col
When reporting a duplicate key error the server was making incorrect assumptions
on what the state of the value string to include in the error is.
Fixed by accessing the data in this string in a "safe" way (without relying on it
having a terminating 0).
Detected by code analysis and fixed a similar problem in reporting the foreign key
duplicate errors.
partition is corrupt
The main problem was that ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR
PARTITION took another code path (over mysql_alter_table instead of
mysql_admin_table) which differs in two ways:
1) alter table opens the tables in a different way than admin tables do
resulting in returning with error before it tried the command
2) alter table does not start to send any diagnostic rows to the client
which the lower admin functions continue to use -> resulting in
assertion crash
The fix:
Remapped ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR PARTITION to use
the same code path as ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE t.
Adding check in mysql_admin_table to setup the partition list for
which partitions that should be used.
Partitioned tables will still not work with
REPAIR TABLE/PARTITION USE_FRM, since that requires moving partitions
to tables, REPAIR TABLE t USE_FRM, and check that the data still
fulfills the partitioning function and then move the table back to
being a partition.
NOTE: I have removed the following functions from the handler
interface:
analyze_partitions, check_partitions, optimize_partitions,
repair_partitions
Since they are not longer needed.
THIS ALTERS THE STORAGE ENGINE API
Problem was that ha_partition had HA_FILE_BASED flag set
(since it uses a .par file), but after open it uses the first partitions
flags, which results in different case handling for create and for
open.
Solution was to change the underlying partition name so it was consistent.
(Only happens when lower_case_table_names = 2, i.e. Mac OS X and storage
engines without HA_FILE_BASED, like InnoDB and Memory.)
(Recommit after adding rename of check_lowercase_names to
get_canonical_filename, and moved it from handler.h to mysql_priv.h)
NOTE: if a mixed case name for a partitioned table was created when
lower_case_table_name = 2 it should be renamed or dropped before using
the updated version (See bug#37402 for more info)
The REPAIR TABLE ... USE_FRM query silently corrupts data of tables
with old .FRM file version.
The mysql_upgrade client program or the REPAIR TABLE query (without
the USE_FRM clause) can't prevent this trouble, because in the
common case they don't upgrade .FRM file to compatible structure.
1. Evaluation of the REPAIR TABLE ... USE_FRM query has been
modified to reject such tables with the message:
"Failed repairing incompatible .FRM file".
2. REPAIR TABLE query (without USE_FRM clause) evaluation has been
modified to upgrade .FRM files to current version.
3. CHECK TABLE ... FOR UPGRADE query evaluation has been modified
to return error status when .FRM file has incompatible version.
4. mysql_upgrade and mysqlcheck client programs call CHECK TABLE
FOR UPGRADE and REPAIR TABLE queries, so their behaviors have
been changed too to upgrade .FRM files with incompatible
version numbers.
a SELECT doesn't cause ROLLBACK of statem".
The idea of the fix is to ensure that we always commit the current
statement at the end of dispatch_command(). In order to not issue
redundant disc syncs, an optimization of the two-phase commit
protocol is implemented to bypass the two phase commit if
the transaction is read-only.
Replacing a template function with a normal static function.
The template parameter, which previously was the class to
find a binlogging function in, is now passed as a pointer to
the actual binlogging function instead.
The patch requires change of indention, but that is submitted
as a separate patch.
The bug was that handler::clone/handler::ha_open() call caused allocation of
cloned_copy->ref on the handler->table->mem_root. The allocated memory could not
be reclaimed until the table is flushed, so it was possible to exhaust memory by
repeatedly running index_merge queries without doing table flushes.
The fix:
- make handler::clone() allocate new_handler->ref on the passed mem_root
- make handler::ha_open() not allocate this->ref if it has already been allocated
There is no testcase as it is not possible to check small leaks from testsuite.
