Reduced number of my_hash_sort_bin() calls from 4 to 1 per query.
Reduced number of memory accesses done by my_hash_sort_bin().
Details:
- let MDL subsystem use pre-calculated hash value for hash
inserts and deletes
- let table cache use pre-calculated MDL hash value
- MDL namespace is excluded from hash value calculation, so that
hash value can be used by table cache as is
- hash value for MDL is calculated as resulting hash value + MDL
namespace
- extended hash implementation to accept user defined hash function
Clean up and improve the parallel implementation code, mainly related to
scheduling of work to threads and handling of stop and errors.
Fix a lot of bugs in various corner cases that could lead to crashes or
corruption.
Fix that a single replication domain could easily grab all worker threads and
stall all other domains; now a configuration variable
--slave-domain-parallel-threads allows to limit the number of
workers.
Allow next event group to start as soon as previous group begins the commit
phase (as opposed to when it ends it); this allows multiple event groups on
the slave to participate in group commit, even when no other opportunities for
parallelism are available.
Various fixes:
- Fix some races in the rpl.rpl_parallel test case.
- Fix an old incorrect assertion in Log_event iocache read.
- Fix repeated malloc/free of wait_for_commit and rpl_group_info objects.
- Simplify wait_for_commit wakeup logic.
- Fix one case in queue_for_group_commit() where killing one thread would
fail to correctly signal the error to the next, causing loss of the
transaction after slave restart.
- Fix leaking of pthreads (and their allocated stack) due to missing
PTHREAD_CREATE_DETACHED attribute.
- Fix how one batch of group-committed transactions wait for the previous
batch before starting to execute themselves. The old code had a very
complex scheduling where the first transaction was handled differently,
with subtle bugs in corner cases. Now each event group is always scheduled
for a new worker (in a round-robin fashion amongst available workers).
Keep a count of how many transactions have started to commit, and wait for
that counter to reach the appropriate value.
- Fix slave stop to wait for all workers to actually complete processing;
before, the wait was for update of last_committed_sub_id, which happens a
bit earlier, and could leave worker threads potentially accessing bits of
the replication state that is no longer valid after slave stop.
- Fix a couple of places where the test suite would kill a thread waiting
inside enter_cond() in connection with debug_sync; debug_sync + kill can
crash in rare cases due to a race with mysys_var_current_mutex in this
case.
- Fix some corner cases where we had enter_cond() but no exit_cond().
- Fix that we could get failure in wait_for_prior_commit() but forget to flag
the error with my_error().
- Fix slave stop (both for normal stop and stop due to error). Now, at stop
we pick a specific safe point (in terms of event groups executed) and make
sure that all event groups before that point are executed to completion,
and that no event group after start executing; this ensures a safe place to
restart replication, even for non-transactional stuff/DDL. In error stop,
make sure that all prior event groups are allowed to execute to completion,
and that any later event groups that have started are rolled back, if
possible. The old code could leave eg. T1 and T3 committed but T2 not, or
it could even leave half a transaction not rolled back in some random
worker, which would cause big problems when that worker was later reused
after slave restart.
- Fix the accounting of amount of events queued for one worker. Before, the
amount was reduced immediately as soon as the events were dequeued (which
happens all at once); this allowed twice the amount of events to be queued
in memory for each single worker, which is not what users would expect.
- Fix that an error set during execution of one event was sometimes not
cleared before executing the next, causing problems with the error
reporting.
- Fix incorrect handling of thd->killed in worker threads.
This is port of fix for MySQL BUG#17647863.
revno: 5572
revision-id: jon.hauglid@oracle.com-20131030232243-b0pw98oy72uka2sj
committer: Jon Olav Hauglid <jon.hauglid@oracle.com>
timestamp: Thu 2013-10-31 00:22:43 +0100
message:
Bug#17647863: MYSQL DOES NOT COMPILE ON OSX 10.9 GM
Rename test() macro to MY_TEST() to avoid conflict with libc++.
mysql-test/r/lowercase_table2.result:
Updated result
(The change happend because we don't try to open the table anymore as part of create table)
mysql-test/suite/rpl/r/create_or_replace_mix.result:
Fixed result file
mysql-test/suite/rpl/r/create_or_replace_row.result:
Fixed result file
mysql-test/suite/rpl/r/create_or_replace_statement.result:
Fixed result file
mysql-test/suite/rpl/t/create_or_replace.inc:
Drop open temporary table
mysys/my_delete.c:
Added missing newline
plugin/metadata_lock_info/mysql-test/metadata_lock_info/r/user_lock.result:
Fixed result
(Lock names was before off by one. Was corrected by my previous patch)
sql/sql_select.cc:
Fixed compiler warnings by adding missing casts
storage/connect/ha_connect.cc:
Fixed compiler warnings
storage/innobase/os/os0file.cc:
Fixed compiler warnings
storage/xtradb/btr/btr0btr.cc:
Fixed compiler warnings
storage/xtradb/handler/ha_innodb.cc:
removed not used function
strings/ctype-uca.c:
Fixed compiler warnings
support-files/compiler_warnings.supp:
Added suppression for warnings that are wrong or are not serious andthat we don't plan to fix.
mysql_upgrade --help now also prints out --default options and variable values.
mysql_upgrade now prints permission errors.
mysql_upgrade doesn't print some non essential info if --silent is used.
Added handler error message about incompatible versions
Fixed that mysqlbug and mysql_install_db have the executable flag set.
Removed executable flag for some non executable files.
Changed in mysql_install_db askmonty.org to mariadb.com.
Ensured that all client executables prints --default options the same way.
Allow REPAIR ... USE_FRM for old .frm files if the are still compatible.
Extended shown error for storage engine messages.
client/mysql.cc:
print_defaults() should be first (as in all other programs)
client/mysql_upgrade.c:
--help now also prints out --default options and variable values
Print out error if wrong permissions
Don't print info if --silent
client/mysqladmin.cc:
print_defaults() should be first (as in all other programs)
client/mysqlbinlog.cc:
Added print_defaults() to --help
client/mysqlcheck.c:
Added empty line in --help
client/mysqlimport.c:
Added empty line in --help
client/mysqlshow.c:
Made --help compatible
client/mysqlslap.c:
Made --help compatible
client/mysqltest.cc:
Added print_defaults() to --help
include/handler_ername.h:
Added handler error message
include/my_base.h:
Added handler error message
mysql-test/r/mysql_upgrade.result:
Updated results
mysql-test/r/repair.result:
Added test case for better error messages
mysql-test/std_data/host_old.MYD:
Added test case for better error messages
mysql-test/std_data/host_old.MYI:
Added test case for better error messages
mysql-test/std_data/host_old.frm:
Added test case for better error messages
mysql-test/t/repair.test:
Added test case for better error messages
mysys/my_handler_errors.h:
Added handler error message
scripts/CMakeLists.txt:
Fixed that mysqlbug and mysql_install_db have the executable flag set
scripts/mysql_install_db.sh:
askmonty.org -> mariadb.com
sql/ha_partition.cc:
Sometimes table_type() can be called for errors even if partition didn't manage to open any files
sql/handler.cc:
Write clear text for not handled, but defined error messages.
sql/share/errmsg-utf8.txt:
Extended shown error for storage engine messages
sql/sql_admin.cc:
Allow REPAIR ... USE_FRM for old .frm files if the are still compatible
storage/myisam/ha_myisam.cc:
Use new error message
- Added MALLOC_LIBRARY variable to hold name of malloc library
- Back ported valgrind related fixes from jemalloc 3.4.1 to the included jemalloc 3.3.1
- Renamed bitmap_init() and bitmap_free() to my_bitmap_init() and my_bitmap_free() to avoid clash with jemalloc 3.4.1
- Use option --soname-synonyms=somalloc=NON to valgrind when using jemalloc
- Show version related variables in mysqld --help
-- Added SHOW_VALUE_IN_HELP marker
Increased back_log to 150 as the original value was a bit too small
CMakeLists.txt:
Added MALLOC_LIBRARY variable to hold name of malloc library
cmake/jemalloc.cmake:
Added MALLOC_LIBRARY variable to hold name of malloc library
config.h.cmake:
Added MALLOC_LIBRARY variable to hold name of malloc library
extra/jemalloc/ChangeLog:
Updates changelog
extra/jemalloc/include/jemalloc/internal/arena.h:
Backported valgrind fixes from jemalloc 3.4.1
extra/jemalloc/include/jemalloc/internal/jemalloc_internal.h.in:
Backported valgrind fixes from jemalloc 3.4.1
extra/jemalloc/include/jemalloc/internal/private_namespace.h:
Backported valgrind fixes from jemalloc 3.4.1
extra/jemalloc/include/jemalloc/internal/tcache.h:
Backported valgrind fixes from jemalloc 3.4.1
extra/jemalloc/src/arena.c:
Backported valgrind fixes from jemalloc 3.4.1
include/my_bitmap.h:
Renamed bitmap_init() and bitmap_free() to my_bitmap_init() and my_bitmap_free() to avoid clash with jemalloc 3.4.1
mysql-test/mysql-test-run.pl:
Use option --soname-synonyms=somalloc=NON to valgrind when using jemalloc
mysql-test/valgrind.supp:
Supression of memory leak in OpenSuse 12.3
mysys/my_bitmap.c:
Renamed bitmap_init() and bitmap_free() to my_bitmap_init() and my_bitmap_free()
sql/ha_ndbcluster_binlog.cc:
Renames
sql/ha_ndbcluster_cond.h:
Renames
sql/ha_partition.cc:
Renames
sql/handler.cc:
Renames
sql/item_subselect.cc:
Renames
sql/log_event.cc:
Renames
sql/log_event_old.cc:
Renames
sql/mysqld.cc:
Renames
Show version related variables in mysqld --help
sql/opt_range.cc:
Renames
sql/opt_table_elimination.cc:
Renames
sql/partition_info.cc:
Renames
sql/rpl_injector.h:
Renames
sql/set_var.h:
Renames
sql/slave.cc:
Renames
sql/sql_bitmap.h:
Renames
sql/sql_insert.cc:
Renames
sql/sql_lex.h:
Renames
sql/sql_parse.cc:
Renames
sql/sql_partition.cc:
Renames
sql/sql_select.cc:
Renames
sql/sql_show.cc:
Renames
sql/sql_update.cc:
Renames
sql/sys_vars.cc:
Show version related variables in mysqld --help
sql/sys_vars.h:
Added SHOW_VALUE_IN_HELP marker for variables that should be shown in --help
sql/table.cc:
Renames
sql/table.h:
Removed not used bitmap_init_value
storage/connect/ha_connect.cc:
Removed compiler warning
storage/maria/ma_open.c:
Renames
unittest/mysys/bitmap-t.c:
Renames
Bug#68338 RFE: make tmpdir a build-time configurable option
Background: Some distributions use tmpfs for mounting /tmp by
default, which has some advantages, but brings also new
issues. Fedora started using tmpfs on /tmp in version 18 for
example. If not configured otherwise in my.cnf, MySQL uses
system's constant P_tmpdir expanded to /tmp on Linux. This can
introduce some problems with limited space in /tmp and also some
data loss in case of replication slave [1].
In case distributions would like to use /var/tmp, which should be
better for MySQL purposes, then we have to patch the source or
change tmpdir option in my.cnf, which is however not updated in
case it has already existed.
Thus, it would be useful to be able to specify default tmpdir
path using a configure option, while using P_tmpdir in case it is
not defined explicitly.
Based on a contribution from Honza Horak
- Character set code & tests from Alexander Barkov
- Integration with ALTER TABLE, REPAIR and open_table from Monty
The problem was that MySQL 5.6 added some croatian and vitanamese character set collations that are incompatible with MariaDB.
The fix is to move the MariaDB conflicting collation numbers out of the region that MySQL is likely to use.
mysql_upgrade, REPAIR TABLE or ALTER TABLE will fix the collations.
If one tries to access and old incompatible table, one will get the error "Table upgrade required...."
After this patch, MariaDB supports all the MySQL character set collations and the old MariaDB croatian collations, which are closer to the latest standard than the MySQL versions.
New character sets:
ucs2_croatian_mysql561_uca_ci
utf8_croatian_mysql561_uca_ci
utf16_croatian_mysql561_uca_ci
utf32_croatian_mysql561_uca_ci
utf8mb4_croatian_mysql561_uca_ci
Other things:
- Fixed some compiler warnings
- mysql_upgrade prints information about repaired tables.
- Increased version number
VERSION:
Increased VERSION number
client/mysqlcheck.c:
Print repaired table name when using --verbose
include/m_ctype.h:
Add new MariaDB collation regions that are not likely to conflict with MySQL
include/my_base.h:
Added flag to detect if table was opened for ALTER TABLE
mysql-test/r/ctype_ldml.result:
Updated result
mysql-test/r/ctype_uca.result:
Updated result
mysql-test/r/ctype_upgrade.result:
Updated result
mysql-test/r/ctype_utf16_uca.result:
Updated result
mysql-test/r/ctype_utf32_uca.result:
Updated result
mysql-test/r/ctype_utf8mb4_uca.result:
Updated result
mysql-test/std_data/ctype_upgrade:
Test files for testing upgrading of conflicting collations
mysql-test/suite/engines/funcs/r/db_alter_collate_ascii.result:
New collations added
mysql-test/suite/engines/funcs/r/db_alter_collate_utf8.result:
New collations added
mysql-test/suite/innodb/r/innodb_ctype_ldml.result:
Updated test result
mysql-test/suite/innodb/t/innodb_ctype_ldml.test:
Updated test result
mysql-test/suite/plugins/r/show_all_plugins.result:
Updated version number
mysql-test/suite/roles/create_and_drop_role_invalid_user_table.result:
Updated version number
mysql-test/t/ctype_ldml.test:
Updated test
mysql-test/t/ctype_uca.test:
Testing of new collations
mysql-test/t/ctype_upgrade.test:
Testing of upgrading tables with old collations
The test ensures that:
- We will get an error if we try to open a table with old collations.
- CHECK TABLE will detect that the table needs to be upgraded.
- ALTER TABLE and REPAIR will fix the table.
- mysql_upgrade works as expected
mysql-test/t/ctype_utf16_uca.test:
Testing of new collations
mysql-test/t/ctype_utf32_uca.test:
Testing of new collations
mysql-test/t/ctype_utf8mb4_uca.test:
Testing of new collations
mysys/charset-def.c:
Added new character sets
mysys/charset.c:
Always give an error, if requested, if a character set didn't exist
sql/handler.cc:
- Added upgrade_collation() to check if collation is compatible with old version
- check_collation_compatibility() checks if we are using an old collation from MariaDB 5.5 or MySQL 5.6
- ha_check_for_upgrade() returns HA_ADMIN_NEEDS_ALTER if we have an incompatible collation
sql/handler.h:
Added new prototypes
sql/sql_table.cc:
- Mark that tables are opened for ALTER TABLE
- If table needs to be upgraded, ensure we are not using online alter table.
sql/table.cc:
- If we are using an old incompatible collation, change to use the new one and mark table as incompatible.
- Give an error if we try to open an incompatible table.
sql/table.h:
Added error that table needs to be rebuild
storage/connect/ha_connect.cc:
Fixed compiler warning
strings/ctype-uca.c:
New character sets
The merge is still missing a few hunks related to temporary tables and
InnoDB log file size. The associated code did not seem to exist in
10.0, so the merge of that needs more work. Until this is fixed, there
are a number of test failures as a result.
