Improve the range estimation algorithm.
Previously:
For a given level the algo knows the number of pages in the requested range and the n
With this change:
Same idea, but peek a few (10) of the intermediate pages to get a better estimate of
In the bug report one of the examples has a btree with a snippet of the leaf level li
page1(899 records), page2(1 record), page3(1 record), page4(1 record)
so when trying to estimate, the previous algo, assumed there are average (899+1)/2=45
Fix Bug#53761 RANGE estimation for matched rows may be 200 times different
Improve the range estimation algorithm.
Previously:
For a given level the algo knows the number of pages in the requested range
and the number of records on the leftmost and the rightmost page. Then it
assumes all pages in between contain the average between the two border pages
and multiplies this average number by the number of intermediate pages.
With this change:
Same idea, but peek a few (10) of the intermediate pages to get a better
estimate of the average number of records per page. If there are less than 10
intermediate pages then all of them will be scanned and the result will be
precise, not an estimation.
In the bug report one of the examples has a btree with a snippet of the leaf
level like this:
page1(899 records), page2(1 record), page3(1 record), page4(1 record)
so when trying to estimate, the previous algo, assumed there are average
(899+1)/2=450 records per page which went terribly wrong. With this change
page2 and page3 will be read and the exact number of records will be returned.
Approved by: Sunny (rb://401)
example files)
The system variable 'thread_concurrency' has been
(re-)enabled on all platforms, to prevent startup
errors.
'thread_concurrency' is unused and has no effect,
on any platform, in MySQL 5.1 and later versions. It
will be deprecated, and removed, in context of
worklog WL#5265
mysql-test/include/have_thread_concurrency.inc:
include/have_thread_concurrency.inc was only needed
to support platforms that didn't have the
thread_concurrency variable available. Since the
variable is now available on all platforms, the file
is removed.
locks on the table
Fixing the partitioning specifics after TRUNCATE TABLE in
bug-42643 was fixed.
Reorganize of code to decrease the size of the giant switch
in mysql_execute_command, and to prepare for future parser
reengineering. Moved code into Sql_statement objects.
Updated patch according to davi's review comments.
libmysqld/CMakeLists.txt:
Added new files.
libmysqld/Makefile.am:
Added new files.
mysql-test/r/not_partition.result:
now returning error on partitioning commands
if partitioning is not enabled.
mysql-test/r/partition_disabled.result:
There is no partition handlerton, so it cannot
find the specified engine in the .frm file.
mysql-test/r/partition_truncate.result:
Updated test results.
mysql-test/suite/parts/inc/partition_mgm.inc:
Added check that TRUNCATE PARTITION does not delete on failure.
mysql-test/suite/parts/r/partition_debug_sync_innodb.result:
updated results.
mysql-test/suite/parts/r/partition_mgm_lc0_archive.result:
updated results.
mysql-test/suite/parts/r/partition_mgm_lc1_archive.result:
updated results.
mysql-test/suite/parts/r/partition_mgm_lc2_archive.result:
updated results.
mysql-test/suite/parts/t/partition_debug_sync_innodb.test:
Test case for this bug.
mysql-test/t/not_partition.test:
Added check for TRUNCATE PARTITION without partitioning.
mysql-test/t/partition_truncate.test:
Added test of TRUNCATE PARTITION on non partitioned table.
sql/CMakeLists.txt:
Added new files.
sql/Makefile.am:
Added new files.
sql/datadict.cc:
Moved out the storage engine check into an own
function, including assert for lock.
sql/datadict.h:
added dd_frm_storage_engine.
sql/sql_alter_table.cc:
moved the code for SQLCOM_ALTER_TABLE in mysql_execute_command
into its own file, and using the Sql_statement object to
prepare for future parser reengineering.
sql/sql_alter_table.h:
Created Sql_statement object for ALTER TABLE.
sql/sql_lex.cc:
resetting m_stmt.
sql/sql_lex.h:
Temporary hack for forward declaration of enum_alter_table_change_level.
sql/sql_parse.cc:
Moved out ALTER/ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE
from the giant switch into their own Sql_statement
objects.
sql/sql_parse.h:
Exporting check_merge_table_access.
sql/sql_partition_admin.cc:
created Sql_statement for
ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR/TRUNCATE
PARTITION. To be able to reuse the TABLE equivalents.
sql/sql_partition_admin.h:
Added Sql_statement of partition admin statements.
sql/sql_table.cc:
Moved table maintenance code into sql_table_maintenance.cc
sql/sql_table.h:
Moved table maintenance code into sql_table_maintenance.h
exporting functions used by sql_table_maintenance.
sql/sql_table_maintenance.cc:
Moved table maintenance code from sql_table.cc
sql/sql_table_maintenance.h:
Sql_statement objects for ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE.
Also declaring the keycache functions.
sql/sql_truncate.cc:
Moved code from SQLCOM_TRUNCATE in mysql_execute_command into
Truncate_statement::execute.
Added check for partitioned table on TRUNCATE PARTITION.
Moved locking fix for partitioned table into
Alter_table_truncate_partition::execute.
sql/sql_truncate.h:
Truncate_statement declaration (sub class of Sql_statement).
sql/sql_yacc.yy:
Using the new Sql_statment objects.
pushdown.
NDB supports only a limited set of item nodes for use in engine condition
pushdown. Because of this adding cache for const expression effectively
disabled this optimization.
The ndb_serialize_cond function is extended to support Item_cache and treat
it as a constant values.
A helper function called ndb_serialize_const is added. It is used to create
Ndb_cond value node from given const item.
mysql-test/suite/ndb/t/disabled.def:
Bug#49746: Const expression caching led to NDB not using engine condition
pushdown.
Enabled ndb_condition_pushdown test after fixing appropriate bug.
sql/ha_ndbcluster_cond.cc:
Bug#49746: Const expression caching led to NDB not using engine condition
pushdown.
The ndb_serialize_cond function is extended to support Item_cache and treat
it as a constant values.
A helper function called ndb_serialize_const is added. It is used to create
Ndb_cond value node from given const item.
sql/item.cc:
Bug#49746: Const expression caching led to NDB not using engine condition
pushdown.
The Item::cache_const_expr_analyzer function is adjusted to not create
cache for Item_int_with_ref objects.
sql/item.h:
Bug#49746: Const expression caching led to NDB not using engine condition
pushdown.
The result_type() method is added to Item_cache class.
The Item_cache_str now initializes its collation.
The bug was fixed by the patch for Bug 52044.
Add a test case.
mysql-test/r/flush.result:
Update results (Bug#52117)
mysql-test/t/flush.test:
Add a test case for Bug#52117 "Pending FLUSH TALBES <list> aborts
transactions unnecessarily"
variable assignments
The assert() that is firing is checking if expressions that can't be
null return a NULL when evaluated.
MAKEDATE() function can return NULL if the second argument is
less then or equal to 0. Thus its nullability depends not only on
the nullability of its arguments but also on their values.
Fixed by (overoptimistically) setting MAKEDATE() to be nullable
despite the nullability of its arguments.
Test added.
Had to update one test result to reflect the metadata change.
The server was not checking for errors generated during
the execution of Item::val_xxx() methods when copying
data to the group, order, or distinct temp table's row.
Fixed by extending the copy_funcs() to return an error
code and by checking for that error code on the places
copy_funcs() is called.
Test case added.
The problem was that SHOW CREATE EVENT released all metadata locks
held by the current transaction. This made any exisiting savepoints
invalid, triggering the assert when ROLLBACK TO SAVEPOINT later
was executed.
This patch fixes the problem by making sure SHOW CREATE EVENT only
releases metadata locks acquired by the statement itself.
Test case added to event_trans.test.
corruption on ADD PARTITION and LOCK TABLE
Bug#53770: Server crash at handler.cc:2076 on
LOAD DATA after timed out COALESCE PARTITION
5.5 fix for:
Bug#51042: REORGANIZE PARTITION can leave table in an
inconsistent state in case of crash
Needs to be back-ported to 5.1
5.5 fix for:
Bug#50418: DROP PARTITION does not interact with
transactions
Main problem was non-persistent operations done
before meta-data lock was taken (53770+53676).
And 53676 needed to keep the table/partitions opened and locked
while copying the data to the new partitions.
Also added thorough tests to spot some additional bugs
in the ddl_log code, which could result in bad state
between the .frm and partitions.
Collapsed patch, includes all fixes required from the reviewers.
mysql-test/r/partition_innodb.result:
updated result with new test
mysql-test/suite/parts/inc/partition_crash.inc:
crash test include file
mysql-test/suite/parts/inc/partition_crash_add.inc:
test all states in fast_alter_partition_table
ADD PARTITION branch
mysql-test/suite/parts/inc/partition_crash_change.inc:
test all states in fast_alter_partition_table
CHANGE PARTITION branch
mysql-test/suite/parts/inc/partition_crash_drop.inc:
test all states in fast_alter_partition_table
DROP PARTITION branch
mysql-test/suite/parts/inc/partition_fail.inc:
recovery test including injecting errors
mysql-test/suite/parts/inc/partition_fail_add.inc:
test all states in fast_alter_partition_table
ADD PARTITION branch
mysql-test/suite/parts/inc/partition_fail_change.inc:
test all states in fast_alter_partition_table
CHANGE PARTITION branch
mysql-test/suite/parts/inc/partition_fail_drop.inc:
test all states in fast_alter_partition_table
DROP PARTITION branch
mysql-test/suite/parts/inc/partition_mgm_crash.inc:
include file that runs all crash and failure injection tests.
mysql-test/suite/parts/r/partition_debug_innodb.result:
new test result file
mysql-test/suite/parts/r/partition_debug_myisam.result:
new test result file
mysql-test/suite/parts/r/partition_special_innodb.result:
updated result
mysql-test/suite/parts/r/partition_special_myisam.result:
updated result
mysql-test/suite/parts/t/partition_debug_innodb-master.opt:
opt file for using with crashing tests of partitioned innodb
mysql-test/suite/parts/t/partition_debug_innodb.test:
partitioned innodb test that require debug builds
mysql-test/suite/parts/t/partition_debug_myisam-master.opt:
opt file for using with crashing tests of partitioned myisam
mysql-test/suite/parts/t/partition_debug_myisam.test:
partitioned myisam test that require debug builds
mysql-test/suite/parts/t/partition_special_innodb-master.opt:
added innodb-file-per-table to easier verify partition status on disk
mysql-test/suite/parts/t/partition_special_innodb.test:
added test case
mysql-test/suite/parts/t/partition_special_myisam.test:
added test case
mysql-test/t/partition_innodb.test:
added test case
sql/sql_base.cc:
Moved alter_close_tables to sql_partition.cc
sql/sql_base.h:
removed some non existing and duplicated functions.
sql/sql_partition.cc:
fast_alter_partition_table:
Spletted abort_and_upgrad_lock_and_close_table
to its parts (wait_while_table_is_used and
alter_close_tables) and always have
wait_while_table_is_used before any persistent
operations (including logs, which will be executed
on failure) and alter_close_tables after
create/read/write operations and before
drop operations.
moved alter_close_tables here from sql_base.cc
Added error injections for better test coverage.
write_log_final_change_partition:
fixed a log_entry linking bug (delete_frm was not
linked to change/drop partition)
and drop partition must be executed before
change partition (change partition can rename a
partition to an old name, like REORG p1 INTO (p1,p2).
write_log_add_change_partition:
need to use drop_frm first, and relinking that entry
and reusing its execute entry.
sql/sql_table.cc:
added initialization of next_active_log_entry.
sql/table.h:
removed a duplicate declaration.
"MySQL server is not restarted properly after RPM upgrade"
The problem is that with the general spec file cleanup and
alignment we also did a name change, dropping the "-community"
part from the package file name.
As a result of this, RPM (some versions of it) will report
file conflicts, because it considers this name difference
to imply different packages.
To avoid this, the spec file explicitly "obsoletes" the old
packages (with "-community" in the file name).
Now, RPM will first install these packages and the remove the
old ones, and part of that removal is running the "%preun"
section which stops the server and uninstalls the service
(removes the symlinks to "/etc/init.d/mysql" from the run
level directories).
This stop/uninstall will affect the new server!
The fix is to define a "%triggerpostun" in this spec file
which will watch for removal of the "-community" server.
If this is done (as part of this install/upgrade), the
trigger code will re-install the service and restart the
server process.
In addition, the "sleep" calls after starting the server
have been cleaned up: Rather than doing 2* "sleep 2",
it is now 1 "sleep 5".
it couldn't parse the --ssl option.
client/mysql_upgrade.c:
mysql_upgrade parses its options and passes some of them to the underlying
tools (mysqlcheck etc). To do this passdown, it reconstructs a
command-line-suitable text from the my_option object (which
contains the option's name and option's value). For options
which expect no parameter, it just had to use the option's name;
for other options, it had to concatenate the option's name,
a "=" symbol, and the option's value; it had code to handle
this latter case, but only for GET_STR options (options taking a
string as value). But since the work on WL 4738, the --ssl
option, a GET_BOOL, which used to have no parameter (NO_ARG), can
now have one (OPT_ARG), so with --ssl we came to the "default"
label, error. Fixed by constructing the command-line-suitable
representation for GET_BOOL too. For --ssl it will produce
--ssl=1 ; for --ssl=0, it will produce --ssl=0.
mysql-test/include/mysql_upgrade_preparation.inc:
handles requirements of tests which use mysql_upgrade
mysql-test/r/mysql_upgrade_ssl.result:
result; without the code fix we would get "internal error".
mysql-test/t/mysql_upgrade.test:
This test has requirements before running; moved them
to an include file in order to share with mysql_upgrade_ssl.
mysql-test/t/mysql_upgrade_ssl.test:
test for bug. Couldn't go into mysql_upgrade.test as this new test requires
SSL support. --force is needed, in case mysql_upgrade.test run before
(in which case mysql_upgrade_ssl would say that upgrade has already been
done); --force forces the upgrade in all cases.
feature
The test for bug no 50939 was put in range.test which isn't such a good idea
since it requires partitioning. Fixed by moving the test case to
partitioning_range.test.
Implement a few simple asserts in my_rwlock_t locks.
include/my_pthread.h:
Declare two simple assert functions.
include/mysql/psi/mysql_thread.h:
Add wrappers for new assert functions.
mysys/thr_rwlock.c:
Add asserts.
sql/sql_base.cc:
Silence a compiler warning for the case when
SAFE_MUTEX is not ON.
A label statement needs to be followed by at least
one primary expression. If built without
WITH_PARTITION_STORAGE_ENGINE set, the block would
be empty.
Added ';' as a dummy statement to fix it.
After BUG#36649, warnings for sub-statements are cleared when a
new sub-statement is started. This is problematic since it suppresses
warnings for unsafe statements in some cases. It is important that we
always give a warning to the client, because the user needs to know
when there is a risk that the slave goes out of sync.
We fixed the problem by generating warning messages for unsafe statements
while returning from a stored procedure, function, trigger or while
executing a top level statement.
We also started checking unsafeness when both performance and log tables are
used. This is necessary after the performance schema which does a distinction
between performance and log tables.
mysql-test/extra/rpl_tests/create_recursive_construct.inc:
Changed the order of the calls in the procedure because the code
that checks if a warning message is printed out expects that the
first statement gives an warning what is not the case for INSERT
INTO ta$CRC_ARG_level VALUES (47);
mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
Updated the result file.
mysql-test/suite/binlog/r/binlog_unsafe.result:
There are several changes here:
(1) - Changed the CREATE PROCEDURE $CRC.
(2) - The procedure $CRC was failing and the content of the binlog
was being printed out, after fix (1) the failure disappeared.
(3) - The warning message for unsafeness due to auto-increment collumns was
changed.
(4) - The warning message for unsafeness due to VERSION(), RAND() was changed.
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning.test:
Tested filters.
mysql-test/suite/binlog/t/binlog_unsafe.test:
Reenabled the test case binlog_unsafe.
mysql-test/suite/binlog/t/disabled.def:
Reenabled the test case binlog_unsafe.
mysql-test/suite/rpl/r/rpl_begin_commit_rollback.result:
Updated the result file.
mysql-test/suite/rpl/r/rpl_non_direct_stm_mixing_engines.result:
Updated the result file.
mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result:
Updated the result file.
sql/sql_class.cc:
Moved the stmt_accessed_table_flag variable and related information to the
LEX as we need the variable reset after each statement even inside a stored
procedure, what did not happen if the information was in the THD.
Changed the routine in the THD::binlog_query that prints the warning
messages to avoid trying to print them when inside a stored procedure,
function or trigger.
Checked for unsafeness when both performance and log tables where used.
After the introduction of the performance schema, we need to check both.
with open HANDLER
This patch changes the code for table renames to not drop metadata
locks. Since table renames are done as a part of ALTER DATABASE ...
UPGRADE, dropping metadata locks in the middle of execution can
result in wrong binlog order since it means that no locks are held
when the binlog is written to.
The RENAME TABLE statement is unafffected since it auto commits and
therefore already drops metadata locks at the end of execution.
This patch also reverts the regression test for Bug#48940 back to
its original version. The test was temporarily changed due to the
issue mentioned above.
Reduce ibuf_mutex and ibuf_pessimistic_insert_mutex contention further.
Protect ibuf->empty by the insert buffer root page latch, not ibuf_mutex.
ibuf_tree_root_get(): Assert that ibuf_mutex is owned by the
caller. Assert that the stamped page number is correct. Assert that
ibuf->empty agrees with the root page.
ibuf_size_update(): Do not update ibuf->empty.
ibuf_init_at_db_start(): Update ibuf->empty while holding the root page latch.
ibuf_add_free_page(): Return TRUE/FALSE instead of DB_SUCCESS/DB_STRONG_FAIL.
ibuf_remove_free_page(): Release ibuf_pessimistic_insert_mutex as
early as possible.
ibuf_contract_ext(): Rely on a dirty read of ibuf->empty, unless the
server is being shut down. Never acquire ibuf_mutex. Eliminate n_stored.
ibuf_contract_after_insert(): Never acquire ibuf_mutex. Perform dirty
reads of ibuf->size and ibuf->max_size.
ibuf_insert_low(): Only acquire ibuf_mutex for mode==BTR_MODIFY_TREE.
Perform dirty reads of ibuf->size and ibuf->max_size. Update
ibuf->empty while holding the root page latch.
ibuf_delete_rec(): Update ibuf->empty while holding the root page latch.
ibuf_is_empty(): Release ibuf_mutex earlier.
Fix by adding "DEPENDS gen_lex_hash" to ADD_CUSTOM_COMMAND(),
so it regenerates lex_hash.h whenever gen_lex_lash changes.
Also, make sql dependent on GenServerSource to avoid concurrent
generation of lex_hash.h (once for sql , and in parallel for embedded)
Remove acquisition of LOCK_open around file system operations,
since such operations are now protected by metadata locks.
Rework table discovery algorithm to not require LOCK_open.
No new tests added since all MDL locking operations are covered
in lock.test and mdl_sync.test, and as long as these tests
pass despite the increased concurrency, consistency must be
unaffected.
mysql-test/t/disabled.def:
Disable NDB tests due to Bug#55799.
sql/datadict.cc:
No longer necessary to protect ha_create_table() with
LOCK_open. Serial execution is now ensured by metadata
locks.
sql/ha_ndbcluster.cc:
Do not manipulate with LOCK_open in cluster code.
sql/ha_ndbcluster_binlog.cc:
Do not manipulate with LOCK_open in cluster code.
sql/ha_ndbcluster_binlog.h:
Update function signature.
sql/handler.cc:
Implement ha_check_if_table_exists().
@todo: some engines provide ha_table_exists_in_engine()
handlerton call, for those we perhaps shouldn't
call ha_discover(), to be more efficient.
Since currently it's only NDB, postpone till
integration with NDB.
sql/handler.h:
Declare ha_check_if_table_exists() function.
sql/mdl.cc:
Remove an obsolete comment.
sql/sql_base.cc:
Update to a new signature of close_cached_tables():
from now on we always call it without LOCK_open.
Update comments.
Remove get_table_share_with_create(), we should
not attempt to create a table under LOCK_open.
Introduce get_table_share_with_discover() instead,
which would request a back off action if the table
exists in engine.
Remove acquisition of LOCK_open for
data dictionary operations, such as check_if_table_exists().
Do not use get_table_share_with_create/discover for views,
where it's not needed.
Make tdc_remove_table() optionally acquire LOCK_open
to simplify usage of this function.
Use the right mutex in the partitioning code when
manipulating with thd->open_tables.
sql/sql_base.h:
Update signatures of changes functions.
sql/sql_insert.cc:
Do not wrap quick_rm_table() with LOCK_open acquisition,
this is unnecessary.
sql/sql_parse.cc:
Update to the new calling convention of tdc_remove_table().
Update to the new signature of close_cached_tables().
Update comments.
sql/sql_rename.cc:
Update to the new calling convention of tdc_remove_table().
Remove acquisition of LOCK_open around filesystem
operations.
sql/sql_show.cc:
Remove get_trigger_table_impl().
Do not acquire LOCK_open for a dirty read of the trigger
file.
sql/sql_table.cc:
Do not acquire LOCK_open for filesystem operations.
sql/sql_trigger.cc:
Do not require LOCK_open for trigger file I/O.
sql/sql_truncate.cc:
Update to the new signature of tdc_remove_table().
sql/sql_view.cc:
Do not require LOCK_open for view I/O.
Use tdc_remove_table() to expel view share.
Update comments.
sql/sys_vars.cc:
Update to the new signature of close_cached_tables().
