Also, implement MDEV-11027 a little differently from 5.5 and 10.0:
recv_apply_hashed_log_recs(): Change the return type back to void
(DB_SUCCESS was always returned).
Report progress also via systemd using sd_notifyf().
- Used same fix as for MyISAM: High level collation byte stored in unused
bit_end position.
- Moved language from header to base_info
- Removed unused bit_end part in HA_KEY_SEG
Both aria and myisam storage engines feature a logic path in
thr_find_all_keys that leads to undefined behaviour by bypassing the
initialization code of variables after my_thread_init().
By refactoring the nested logic into a separate function, this problem
is resolved.
Problem was that insert-order (enforced by the optimizer) did not handle
the case where the bitmap changed to a new one.
Fixed by remembering the last bitmap page used and to force usage of this when inserting new rows
Due to a hack that has propagated to the maria storage engine, undefined
behaviour would result by bypassing the initialization code of variables
after my_thread_init().
By refactoring the nested logic into a separate function, this problem
is resolved.
This bug only happens in case of paritioned tables used in LOCK TABLES and implicit_commit() was called
(as part of trying to execute a CREATE TABLE withing lock tables)
The problem was that Aria could not move the tables from one transaction to the new one, as thd->open_tables contained
a partitioned tables and not an Aria table.
Fix:
- Store a list of all open tables that are part of a share in share->open_tables
- In maria::implict_commit() use transaction->used_tables & share->open_tables to find out which tables
was part of the current transaction instead of using thd->open_tables, which may contain partitioned tables.
mysql-test/suite/maria/maria_partition.result:
Added test case
mysql-test/suite/maria/maria_partition.test:
Added test case
storage/maria/ha_maria.cc:
Use trn->used tables and share->open_tables to find out which tables was part of the current transaction instead of using thd->open_tables.
storage/maria/ma_close.c:
Remove closed table from share->open_list
storage/maria/ma_open.c:
Add table to share->open_list
storage/maria/ma_state.c:
Added comment
storage/maria/maria_def.h:
Added share->open_list, a list of all tables that is using this share.
The problem was that a big record was allocated on the stack, which casued stack to run out.
Fixed by using my_safe_alloca() instead of my_alloca() when allocating records.
Now only records <= 16384 are allocated on the stack.
mysql-test/r/stack-crash.result:
Added test case
mysql-test/t/stack-crash.test:
Added test case
storage/maria/ma_blockrec.c:
Use my_safe_alloca() instead of my_alloca()
storage/maria/ma_dynrec.c:
Use my_safe_alloca() instead of my_alloca()
storage/maria/maria_def.h:
Added MARIA_MAX_RECORD_ON_STACK
storage/maria/maria_pack.c:
Use my_safe_alloca() instead of my_alloca()
The problem was that a big record was allocated on the stack, which casued stack to run out.
Fixed by using my_safe_alloca() instead of my_alloca() when allocating records.
Now only records <= 16384 are allocated on the stack.
mysql-test/r/stack-crash.result:
Added test case
mysql-test/t/stack-crash.test:
Added test case
storage/maria/ma_blockrec.c:
Use my_safe_alloca() instead of my_alloca()
storage/maria/ma_dynrec.c:
Use my_safe_alloca() instead of my_alloca()
storage/maria/maria_def.h:
Added MARIA_MAX_RECORD_ON_STACK
storage/maria/maria_pack.c:
Use my_safe_alloca() instead of my_alloca()
Update InnoDB to 5.6.14
Apply MySQL-5.6 hack for MySQL Bug#16434374
Move Aria-only HA_RTREE_INDEX from my_base.h to maria_def.h (breaks an assert in InnoDB)
Fix InnoDB memory leak
Fixed the following compilation errors and test failures:
- maria SE: "stage_waiting_for_a_resource" wasn't declared w/o PFS
- sql_repl.h: PSI_mutex_key is not available in non-PFS builds
- mysqld.cc: pfs_param is not available in non-PFS builds
- mysqld.cc: init_show_explain_psi_keys() is not available in non-PFS builds
- mysqld.cc: call net_before_header_psi, net_after_header_psi even if PFS
is not available so that thread enters stage_init at proper time. Fixes
sp-threads and a few tests in funcs_1.
- myisam_file_io.opt: added missing loose prefix
includes:
* remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
* introduce LOCK_share, now LOCK_ha_data is strictly for engines
* rea_create_table() always creates .par file (even in "frm-only" mode)
* fix a 5.6 bug, temp file leak on dummy ALTER TABLE
This could happen when using Aria for internal temporary files (default case) and using DISTINCT.
_ma_scan_restore_block_record() didn't work correctly if there was rows inserted, updated or deleted on the handler
between calls to _ma_scan_remember_block_record() and _ma_scan_restore_block_record().
The effect was that some DISTINCT queries that used remove_dup_with_compare() could fail.
.bzrignore:
Ignore sql_yacc.hh
mysql-test/suite/maria/r/distinct.result:
Test case for MDEV-4280
mysql-test/suite/maria/t/distinct.test:
Test case for MDEV-4280
mysql-test/t/mysql.test:
Fixed test suite (we could get error -1 in some cases)
sql/sql_select.cc:
Break loop if restart_rnd_next() gives an error
storage/maria/ha_maria.cc:
scan_restore_pos() can return disk fault error.
storage/maria/ma_blockrec.c:
_ma_scan_remember_block_record() did incorrectly update scan.dir instead of scan_save.dir .
_ma_scan_restore_block_record() didn't work correctly if there was rows inserted,updated or deleted on the handler
between calls to _ma_scan_remember_block_record() and _ma_scan_restore_block_record().
Fixed by adding counters for row changes and reading the current scan page if changes had been made.
storage/maria/ma_blockrec.h:
scan_restore_pos() can return disk fault error.
storage/maria/ma_delete.c:
Increment row_changes
storage/maria/ma_scan.c:
scan_restore_pos() can return disk fault error.
storage/maria/ma_update.c:
Increment row_changes
storage/maria/ma_write.c:
Increment row_changes
storage/maria/maria_def.h:
scan_restore_pos() can return disk fault error.
- If one specifies --force twice to myisamchk and aria_chk, then we will try to finnish the repair even if sort_buffer would be too small.
This was done by dynamically allocate buffer handler objects as long as memory lasts.
- New option for myisamchk and aria_chk: create-missing-keys
- Changed default size of myisam_sort_buffer_size from 8M to 128M.
- Changed default size of sort_buffer_size in aria_chk from 128M to 256M.
- Increased information in error message about 'sort_buffer_size' beeing to small.
- Print also to 'show warnings' if repair was retried.
- Increased size of internal sort-buffer-readers from 16K to 128K
- Changed printing of 'number of records' to use %ll instead of casting to long
- Changed buffer sizes for myisam and aria to use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines.
include/my_global.h:
Added MY_ALIGN_DOWN() to get previous alignment (for big memory areas)
include/myisam.h:
Increased size of types to be able to handle more records
include/myisamchk.h:
Increased size of types to be able to handle more records
Added T_FORCE_SORT_MEMORY to force repair to work even if sort_buffer would not be big enough
mysql-test/r/myisam.result:
Updated result
mysql-test/r/mysqld--help.result:
Updated result
mysql-test/r/repair.result:
Updated result
mysql-test/suite/maria/maria.result:
Added test cases for some fixed bugs in MyISAM to verify that Aria doesn't have them too.
mysql-test/suite/maria/maria.test:
Added test cases for some fixed bugs in MyISAM to verify that Aria doesn't have them too.
mysql-test/suite/maria/maria3.result:
Updated result after sort buffer size increase
mysql-test/suite/sys_vars/r/aria_sort_buffer_size_basic.result:
Updated result after sort buffer size increase
mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic.result:
Updated result after sort buffer size increase
mysql-test/suite/sys_vars/t/myisam_sort_buffer_size_basic.test:
Updated result after sort buffer size increase
mysql-test/t/myisam.test:
Fixed error messages to not print system specific data
mysql-test/t/repair.test:
Fixed error messages to not print system specific data
storage/maria/ha_maria.cc:
Print also to 'show warnings' if repair was retried
Changed default size of sort_buffer_size from 128M to 256M (same as in mysqld)
storage/maria/ma_check.c:
Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/ma_sort.c:
Increased size of internal sort-buffer-readers from 16K to 128K
Increased size of types to be able to handle more records
Added support for T_FORCE_SORT_MEMORY
Don't allocate too many extra BUFFPEK at a time (they are probably not needed)
Improved error message for "sort_buffer_size is too small"
Changed printing of 'number of records' to use %ll instead of casting to long
Fixed bug where maria_update_key_parts() was called too early.
Fixed bug in detecting result from read_to_buffer().
Added 'out of memory' checking when calling 'alloc_dynamic()'.
storage/maria/maria_chk.c:
Added --create-missing-keys
If one specifies --force twice then we will try to finnish the repair even if sort_buffer would be too small.
check_param.sort_buffer_length varialble was used with wrong type.
storage/maria/maria_def.h:
Increased size of types to be able to handle more records
Use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines
Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/maria_ftdump.c:
Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/maria_read_log.c:
Use PAGE_BUFFER_INIT for page cache
storage/myisam/ha_myisam.cc:
Changed default size of myisam_sort_buffer_size from 8M to 128M
storage/myisam/mi_check.c:
Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisam_ftdump.c:
Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisamchk.c:
Added --create-missing-keys
If one specifies --force twice then we will try to finnish the repair even if sort_buffer would be too small.
check_param.sort_buffer_length varialble was used with wrong type.
Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisamdef.h:
Increased SORT_BUFFER_INIT to 64M
(speeds up repair a lot and most machines have nowadays a lot of memory)
Use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines
Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/sort.c:
Increased size of internal sort-buffer-readers from 16K to 128K
Increased size of types to be able to handle more records
Added support for T_FORCE_SORT_MEMORY
Don't allocate too many extra BUFFPEK at a time (they are probably not needed)
Improved error message for "sort_buffer_size is too small"
Changed printing of 'number of records' to use %ll instead of casting to long
Fixed bug in detecting result from read_to_buffer().
Added 'out of memory' checking when calling 'alloc_dynamic()'.
The patch lifts the limitation of the current implementation
of ALTER TABLE that does not allow to build unique/primary
indexes by sort for MyISAM and Aria engines.
and small collateral changes
mysql-test/lib/My/Test.pm:
somehow with "print" we get truncated writes sometimes
mysql-test/suite/perfschema/r/digest_table_full.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/dml_handler.result:
host table is not ported over yet
mysql-test/suite/perfschema/r/information_schema.result:
host table is not ported over yet
mysql-test/suite/perfschema/r/nesting.result:
this differs, because we don't rewrite general log queries, and multi-statement
packets are logged as a one entry. this result file is identical to what mysql-5.6.5
produces with the --log-raw option.
mysql-test/suite/perfschema/r/relaylog.result:
MariaDB modifies the binlog index file directly, while MySQL 5.6 has a feature "crash-safe binlog index" and modifies a special "crash-safe" shadow copy of the index file and then moves it over. That's why this test shows "NONE" index file writes in MySQL and "MANY" in MariaDB.
mysql-test/suite/perfschema/r/server_init.result:
MariaDB initializes the "manager" resources from the "manager" thread, and starts this thread only when --flush-time is not 0. MySQL 5.6 initializes "manager" resources unconditionally on server startup.
mysql-test/suite/perfschema/r/stage_mdl_global.result:
this differs, because MariaDB disables query cache when query_cache_size=0. MySQL does not
do that, and this causes useless mutex locks and waits.
mysql-test/suite/perfschema/r/statement_digest.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_consumers.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_long_query.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/rpl/r/rpl_mixed_drop_create_temp_table.result:
will be updated to match 5.6 when alfranio.correia@oracle.com-20110512172919-c1b5kmum4h52g0ni and anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y are merged
mysql-test/suite/rpl/r/rpl_non_direct_mixed_mixing_engines.result:
will be updated to match 5.6 when anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y is merged
- Don't use SAFEMALLOC on valgrind builds (slows things down)
- Added back lost option from 5.3: debug-mutex-deadlock-detector
- Flush pages before taking lock mutex (speeds up closing of Aria tables).
BUILD/SETUP.sh:
- Don't use SAFEMALLOC on valgrind builds (slows things down)
sql/lock.cc:
Make default argument explicit (improves readability)
sql/mysqld.cc:
Removed compiler warnings
Sorted debug options alphabetically
Added back lost option from 5.3: debug-mutex-deadlock-detector
storage/maria/ma_close.c:
Flush pages before taking lock mutex (speeds up closing of Aria tables).
storage/maria/ma_open.c:
More DBUG_PRINT
storage/maria/maria_def.h:
Better DBUG_PRINT
storage/maria/trnman.c:
Better DBUG_PRINT
Honor unique/not unique when creating keys for internal tempory tables.
Added new variables to be used to limit how keys are created for internal temporary tables.
include/maria.h:
Added maria_max_key_length() and maria_max_key_segments()
include/myisam.h:
Added myisam_max_key_length() and myisam_max_key_segments()
mysql-test/r/mysql.result:
Drop all used tables
mysql-test/r/subselect4.result:
Added test case for lp:879939
mysql-test/t/mysql.test:
Drop all used tables
mysql-test/t/subselect4.test:
Added test case for lp:879939
sql/mysql_priv.h:
Added internal_tmp_table_max_key_length and internal_tmp_table_max_key_segments to be used to limit how keys for derived tables are created.
sql/mysqld.cc:
Added internal_tmp_table_max_key_length and internal_tmp_table_max_key_segments to be used to limit how keys for derived tables are created.
sql/share/errmsg.txt:
Added new error message for internal errors
sql/sql_select.cc:
Give error if we try to create a wrong key (this error should never happen)
Honor unique/not unique when creating keys for internal tempory tables.
storage/maria/ha_maria.cc:
Added change_table_ptr() to ensure that external_ref points always to the correct table.
(Not having this caused an assert in the included test)
storage/maria/ha_maria.h:
Added change_table_ptr() to ensure that external_ref points always to the correct table.
storage/maria/ma_check.c:
Fixed bug in Duplicate key error printing (now row position is printed correctly)
storage/maria/ma_create.c:
maria_max_key_length() -> _ma_max_key_length()
storage/maria/ma_info.c:
Added extern function maria_max_key_length() to calculate the max key length based on current block size.
storage/maria/ma_open.c:
maria_max_key_length() -> _ma_max_key_length()
storage/maria/maria_def.h:
maria_max_key_length() -> _ma_max_key_length()
storage/myisam/ha_myisam.cc:
Added change_table_ptr() to ensure that external_ref points always to the correct table.
(Not having this caused an assert in the included test)
storage/myisam/ha_myisam.h:
Added change_table_ptr() to ensure that external_ref points always to the correct table.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
Added more error printing to log if log_warnings > 2
Give an error if checkpoint record is not correct,
mysql-test/suite/maria/r/compat_aliases.result:
Added ARIA_CHECKPOINT_LOG_ACTIVITY
mysql-test/suite/maria/r/maria3.result:
Added ARIA_CHECKPOINT_LOG_ACTIVITY
storage/maria/ha_maria.cc:
Added ARIA_CHECKPOINT_LOG_ACTIVITY
Added more error printing to log if log_warnings > 2
Added db and table name to error message when printing to log
storage/maria/ma_check.c:
Fixed bug where we didn't reset some variables between repair() calls
storage/maria/ma_checkpoint.c:
Made maria_checkpoint_min_activity global.
Don't do checkpoint if no data logged.
Changed test for if we should do checkpoint to test separately for if log has grown or if we have had a lot of of cache writes.
storage/maria/ma_recovery.c:
Give an error if checkpoint record is not correct
storage/maria/trnman.c:
Don't print not needed long_transaction_id entries for checkpoints.
The issues was:
- For some tables with a lot of not packed fields, we didn't allocate enough memory in head page which caused DBUG_ASSERT's
- Removed wrong DBUG_ASSERT()
- Fixed a problem with underflow() where it generates a key page where all keys didn't fit.
- Max key length is now limited by block_size/3 (was block_size /2). This is required for underflow() to work with packed keys.
mysql-test/lib/v1/mysql-test-run.pl:
Remove --alignment=8 as this doesn't work on 64 bit systems
mysql-test/suite/maria/r/small_blocksize.result:
Test case for Aria bug
mysql-test/suite/maria/t/small_blocksize-master.opt:
Test case for Aria bug
mysql-test/suite/maria/t/small_blocksize.test:
Test case for Aria bug
storage/maria/ha_maria.cc:
Fixed comment
storage/maria/ma_bitmap.c:
Fixed wrong variable usage in find_where_to_split_row() where we allocated too little memory for head page.
We did not take into account space for head extents (long VARCHAR) when trying to split row on head page. This caused us to allocate too little space from bitmap which lead to ASSERT failures later.
storage/maria/ma_blockrec.c:
Made some argument const (to ensure they was not accidently changed)
Removed wrong DBUG_ASSERT()
storage/maria/ma_blockrec.h:
Removed not used variable
storage/maria/ma_delete.c:
Added my_afree() in case of error
More comments and DBUG_ASSERT() for underflow()
storage/maria/ma_open.c:
Make keyinfo->underflow_block_length smaller for packed keys. This has to be done as for long packed keys, underflow() otherwise generates a key page where all keys didn't fit.
storage/maria/ma_page.c:
New DBUG_ASSERT()
storage/maria/ma_write.c:
Fixed comment
storage/maria/maria_def.h:
We have to have space for at least 3 keys on a key page.
(Otherwise the underflow() code doesn't work for packed keys, even when we have an underflow() for an empty key page)
