Some ut_a(!rec_offs_any_null_extern()) assertion failures are indicating
genuine BLOB bugs, others are bogus failures when rolling back incomplete
transactions at crash recovery. This needs more work, and until I get a
chance to work on it, other testing must not be disrupted by this.
SYNTAX TRIGGERS IN ANY WAY
Table with triggers which were using deprecated (5.0-only) syntax became
unavailable for any DML and DDL after upgrade to 5.1 version of server.
Attempt to execute any statement on such a table resulted in parsing
error reported. Since this included DROP TRIGGER and DROP TABLE
statements (actually, the latter was allowed but was not functioning
properly for such tables) it was impossible to fix the problem without
manual operations on .TRG and .TRN files in data directory.
The problem was that failure to parse trigger body (due to 5.0-only
syntax) when opening trigger file for a table prevented the table
from being open. This made all operations on the table impossible
(except DROP TABLE which due to peculiarity in its implementation
dropped the table but left trigger files around).
This patch solves this problem by silencing error which occurs when
we parse trigger body during table open. Error message is preserved
for the future use and table is marked as having a broken trigger.
We also try to analyze parse tree to recover trigger name, which
will be needed in order to drop the broken trigger. DML statements
which invoke triggers on the table marked as having broken trigger
are prohibited and emit saved error message. The same happens for
DDL which change triggers except DROP TRIGGER and DROP TABLE which
try their best to do what was requested. Table becomes no longer
marked as having broken trigger when last such trigger is dropped.
mysql-test/r/trigger-compat.result:
Add results for test case for bug#45235
mysql-test/t/trigger-compat.test:
Add test case for bug#45235.
sql/sp_head.cc:
Added protection against MEM_ROOT double restoring to
sp_head::restore_thd_mem_root() method. Since this
method can be sometimes called twice during parsing
of stored routine (the first time during normal flow
of parsing, and the second time when a syntax error
is detected) we need to shortcut execution of the
method to avoid damaging MEM_ROOT by the second
consecutive call to this method.
sql/sql_trigger.cc:
Added error handler Deprecated_trigger_syntax_handler to
catch non-OOM errors during parsing of trigger body.
Added handling of parse errors into method
Table_triggers_list::check_n_load().
sql/sql_trigger.h:
Added new members to handle broken triggers and error messages.
THE EVENT STATUS.
Any ALTER EVENT statement on a disabled event enabled it back
(unless this ALTER EVENT statement explicitly disabled the event).
The problem was that during processing of an ALTER EVENT statement
value of status field was overwritten unconditionally even if new
value was not specified explicitly. As a consequence this field
was set to default value for status which corresponds to ENABLE.
The solution is to check if status field was explicitly specified in
ALTER EVENT statement before assigning new value to status field.
mysql-test/r/events_bugs.result:
test's result for Bug#11764334 was added.
mysql-test/t/events_bugs.test:
new test for Bug#11764334 was added.
sql/event_db_repository.cc:
mysql_event_fill_row() was modified: set value for status field
in events tables only in case if statement CREATE EVENT
is being processed or if this value was set in ALTER EVENT
statement.
Event_db_repository::create_event was modified: removed redundant
setting of status field after return from call to mysql_event_fill_row().
sql/event_parse_data.h:
Event_parse_data structure was modified: added flag
status_changed that is set to true if status's value
was changed in ALTER EVENT statement.
sql/sql_yacc.yy:
Set flag status_changed if status was set in ALTER EVENT
statement.
SEEMS TO BE 'LEAKING' INTO THE SCHEMA NAME SPACE)
and bug#12428824 (Parser stack overflow and crash in sp_add_used_routine
with obscure query).
The first problem was that attempts to call a stored function by
its fully qualified name ended up with unwarranted error "ERROR 1305
(42000): FUNCTION someMixedCaseDb.my_function_name does not exist"
if this function belonged to a schema that had uppercase letters in
its name AND --lower_case_table_names was equal to either 1 or 2.
The second problem was that 5.5 version of MySQL server might have
crashed when a user tried to call stored function with too long name
or too long database name (i.e if a function and database name combined
occupied more than 2*3*64 bytes in utf8). This issue didn't affect
versions of server < 5.5.
The first problem was caused by the fact that in cases when a stored
function was called by its fully qualified name we didn't lowercase
name of its schema before performing look up of the function in
mysql.proc table even although lower_case_table_names mode was on.
As result we were unable to find this function since during its
creation we store lowercased version of schema name in the system
table in this mode and field for schema name uses binary collation.
Calls to stored functions were unaffected by this problem since for
them schema name is converted to lowercase as necessary.
The reason for the second bug was that MySQL Server didn't check length
of function name and database name before proceeding with execution of
stored function. As a consequence too long database name or function
name caused buffer overruns in places where the code assumes that their
length is within fixed limits, like mdl_key_init() in 5.5.
Again this issue didn't affect calls to stored procedures as for them
length of schema name and procedure name are properly checked.
This patch fixes both these bugs by adding calls to check_db_name()
and check_routine_name() to grammar rule which corresponds to a call
to a stored function. These functions ensure that length of database
name and function name for routine called is within standard limit.
Moreover call to check_db_name() handles conversion of database name
to lowercase if --lower_case_table_names mode is on.
Note that even although the second issue seems to be only reproducible
in 5.5 we still add code fixing it to 5.1 to be on the safe side (and
make code a bit more robust against possible future changes).
mysql-test/r/sp-error.result:
Added testcase results for bug#12428824.
mysql-test/r/sp.result:
Added testcase result for bug#11840395.
mysql-test/t/sp-error.test:
Added testcase for bug#12428824.
mysql-test/t/sp.test:
Added testcase for bug#11840395.
sql/sql_yacc.yy:
Modified 'function_call_generic' rule to call check_db_name() and
check_routine_name() in order to ensure that lengths of database name
and function name are within limits. check_db_name() is also responsible
for normalizing function's database name for lookup in cases when
lowercase_table_names mode is on.
If UNIV_DEBUG or UNIV_BLOB_LIGHT_DEBUG is enabled, add
!rec_offs_any_null_extern() assertions, ensuring that records do not
contain null pointers to externally stored columns in inappropriate
places.
btr_cur_optimistic_update(): Assert !rec_offs_any_null_extern().
Incomplete records must never be updated or deleted. This assertion
will cover also the pessimistic route.
row_build(): Assert !rec_offs_any_null_extern(). Search tuples must
never be built from incomplete index entries.
row_rec_to_index_entry(): Assert !rec_offs_any_null_extern() unless
ROW_COPY_DATA is requested. ROW_COPY_DATA is used for
multi-versioning, and therefore it might be valid to copy the most
recent (uncommitted) version while it contains a null pointer to
off-page columns.
row_vers_build_for_consistent_read(),
row_vers_build_for_semi_consistent_read(): Assert !rec_offs_any_null_extern()
on all versions except the most recent one.
trx_undo_prev_version_build(): Assert !rec_offs_any_null_extern() on
the previous version.
rb:682 approved by Sunny Bains
Problem: in case of wrong data insert into indexed GEOMETRY fields
(e.g. NULL value for a not NULL field) MyISAM reported
"ERROR 126 (HY000): Incorrect key file for table; try to repair it"
due to misuse of the key deletion function.
Fix: always use R-tree key functions for R-tree based indexes
and B-tree key functions for B-tree based indexes.
mysql-test/r/gis-rtree.result:
Bug#11764487: myisam corruption with insert ignore and invalid spatial data
- test result.
mysql-test/t/gis-rtree.test:
Bug#11764487: myisam corruption with insert ignore and invalid spatial data
- test case.
storage/myisam/mi_update.c:
Bug#11764487: myisam corruption with insert ignore and invalid spatial data
- handling update errors check for HA_ERR_NULL_IN_SPATIAL as well to be
consistent with mi_write();
- always use keyinfo->ck_delete()/ck_insert() instead of _mi_ck_delete()/_mi_ck_write()
to handle index properly, as it may be of B-tree or R-tree type.
storage/myisam/mi_write.c:
Bug#11764487: myisam corruption with insert ignore and invalid spatial data
- always use keyinfo->ck_delete() instead of _mi_ck_delete() to handle
index properly, as it may be of B-tree or R-tree type.
This is the 5.1 version of the fix.
Need to free the memory allocated by the option parsing code for empty
strings when resetting the pointer to NULL.
No test case needed, as the existing ones already cover this path.
row_build_index_entry(): In innodb_file_format=Barracuda
(ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED), a secondary index on a
full column can refer to a field that is stored off-page in the
clustered index record. Take that into account.
rb:692 approved by Jimmy Yang
Refactor the !rec_offs_any_extern relaxation in row_build().
trx_assert_active(trx_id): Assert that the given transaction is active.
(In the 5.1 built-in InnoDB, there is no trx->is_recovered field.)
trx_assert_recovered(trx_id): Assert that the given transaction is
active and has been recovered after a crash.
row_build(): Replace a bunch of code with an assertion that invokes
trx_assert_active() or trx_assert_recovered() and row_get_rec_trx_id().
row_get_trx_id_offset(): Make the function inlined. Remove the unused
parameter rec, and make all parameters const.
row_get_rec_trx_id(), row_get_rec_roll_ptr(): Make all parameters const.
rb:691 approved by Jimmy Yang
page_zip_dir_elems(): New function, refactored from page_zip_dir_size().
page_zip_dir_size(): Use page_zip_dir_elems()
page_zip_dir_start_offs(): New function: Gets an offset to the
compressed page trailer (the dense page directory), including deleted
records (the free list)
page_zip_dir_start_low(page_zip, n_dense): Constness-preserving
wrapper macro for page_zip_dir_start_offs().
page_zip_dir_start(page_zip): Constness-preserving
wrapper macro for page_zip_dir_start_offs().
page_zip_decompress_node_ptrs(), page_zip_decompress_clust(): Replace
a formula with a fully equivalent page_zip_dir_start_low() call.
page_zip_write_rec(), page_zip_parse_write_node_ptr(),
page_zip_write_node_ptr(), page_zip_write_trx_id_and_roll_ptr(),
page_zip_clear_rec(): Replace a formula with an almost equivalent
page_zip_dir_start() call.
It is OK to replace page_dir_get_n_heap(page) with
page_dir_get_n_heap(page_zip->data), because
ut_ad(page_zip_header_cmp(page_zip, page)) or
page_zip_validate(page_zip, page) asserts that the
page headers are identical.
rb:687 approved by Jimmy Yang
BOGUS "THE TABLE MYSQL.PROC IS MISSING,..."
There was a race condition between loading a stored routine
(function/procedure/trigger) specified by fully qualified name
SCHEMA_NAME.PROC_NAME and dropping the stored routine database.
The problem was that there is a window for race condition when one server
thread tries to load a stored routine being executed and the other thread
tries to drop the stored routine schema.
This condition race window exists in implementation of function
mysql_change_db() called by db_load_routine() during loading of stored
routine to cache. Function mysql_change_db() calls check_db_dir_existence()
that might failed because specified database was dropped during concurrent
execution of DROP SCHEMA statement. db_load_routine() calls mysql_change_db()
with flag 'force_switch' set to 'true' value so when referenced db is not found
then my_error() is not called and function mysql_change_db() returns ok.
This shadows information about schema opening error in db_load_routine().
Then db_load_routine() makes attempt to parse stored routine that is failed.
This makes to return error to sp_cache_routines_and_add_tables_aux() but since
during error generation a call to my_error wasn't made and hence
THD::main_da wasn't set we set the generic "mysql.proc table corrupt" error
when running sp_cache_routines_and_add_tables_aux().
The fix is to install an error handler inside db_load_routine() for
the mysql_op_change_db() call, and check later if the ER_BAD_DB_ERROR
was caught.
sql/sql_db.cc:
Added synchronization point "before_db_dir_check" to emulate a race condition during
processing of CALL/DROP SCHEMA.
approved by: Marko
rb://681
Coalescing of free buf_page_t descriptors can prove to be one severe
bottleneck in performance of compression. One such workload where it
hurts badly is DROP TABLE. This patch removes buf_page_t allocations
from buf_buddy and uses ut_malloc instead.
In order to further reduce overhead of colaescing we no longer attempt
to coalesce a block if the corresponding free_list is less than 16 in
size.
TO POSITION FIRST CAN CAUSE DATA TO BE CORRUPTED".
ALTER TABLE MODIFY/CHANGE ... FIRST did nothing except renaming
columns if new version of the table had exactly the same
structure as the old one (i.e. as result of such statement, names
of columns changed their order as specified but data in columns
didn't). The same thing happened for ALTER TABLE DROP COLUMN/ADD
COLUMN statements which were supposed to produce new version of
table with exactly the same structure as the old version of table.
I.e. in the latter case the result was the same as if old column
was renamed instead of being dropped and new column with default
as value being created.
Both these problems were caused by the fact that ALTER TABLE
implementation incorrectly interpreted both these situations as
simple renaming of columns and assumed that in-place ALTER TABLE
algorithm could have been used for them.
This patch fixes this problem by ensuring that in cases when some
column is moved to the first position or some column is dropped
the default ALTER TABLE algorithm involving table copying is
always used. This is achieved by detecting such situations in
mysql_prepare_alter_table() and setting Alter_info::change_level
to ALTER_TABLE_DATA_CHANGED for them.
mysql-test/r/alter_table.result:
Added test for bug #12652385 - "61493: REORDERING COLUMNS TO
POSITION FIRST CAN CAUSE DATA TO BE CORRUPTED".
mysql-test/t/alter_table.test:
Added test for bug #12652385 - "61493: REORDERING COLUMNS TO
POSITION FIRST CAN CAUSE DATA TO BE CORRUPTED".
sql/sql_table.cc:
Changed mysql_prepare_alter_table() to detect situations in
which we some column moved to the first position or some column
is dropped and ensure that such ALTER TABLE statements won't
be carried out using in-place algorithm. The latter could have
happened before this patch if new version of table had the same
structure as the old one (except the column names).
The buf_pool->zip_clean list is only needed for debugging, or for
recomputing buf_pool->page_hash when resizing the buffer pool. Buffer
pool resizing was never fully implemented. Remove the resizing code,
and define buf_pool->zip_clean only in debug builds.
buf_pool->zip_clean, buf_LRU_insert_zip_clean(): Enclose in
#if defined UNIV_DEBUG || UNIV_BUF_DEBUG.
buf_chunk_free(), buf_chunk_all_free(), buf_pool_shrink(),
buf_pool_page_hash_rebuild(), buf_pool_resize(): Remove (unreachable code).
rb:671 approved by Inaam Rana
There is an apparent problem with page_zip_clear_rec().
In btr_cur_optimistic_update() we do this:
page_cur_delete_rec(page_cursor, index, offsets, mtr);
...
rec = btr_cur_insert_if_possible(cursor, new_entry, 0/*n_ext*/, mtr);
ut_a(rec); /* <- We calculated above the insert would fit */
The problem is that page_cur_delete_rec() could fill the modification
log while doing page_zip_clear_rec(), requiring recompression for the
btr_cur_insert_if_possible(). In a pathological case, the data could
fail to recompress.
page_zip_clear_rec(): Leave the page modification log alone. Only
clear the necessary fields.
rb:673 approved by Jimmy Yang
Replace UNIV_BLOB_NULL_DEBUG with UNIV_DEBUG||UNIV_BLOB_LIGHT_DEBUG.
Fix known bogus failures.
btr_cur_optimistic_update(): If rec_offs_any_null_extern(), assert that
the current transaction is an incomplete transaction that is being
rolled back in crash recovery.
row_build(): If rec_offs_any_null_extern(), assert that the transaction
that last updated the record was recovered during crash recovery
(and will soon be rolled back).
btr_cur_compress_if_useful(), btr_compress(): Add the parameter ibool
adjust. If adjust=TRUE, adjust the cursor position after compressing
the page.
btr_lift_page_up(): Return a pointer to the father page.
BTR_KEEP_POS_FLAG: A new flag for btr_cur_pessimistic_update().
btr_cur_pessimistic_update(): If *big_rec != NULL and flags &
BTR_KEEP_POS_FLAG, keep the cursor positioned on the updated record.
Also, do not release the index tree x-lock if *big_rec != NULL.
btr_cur_mtr_commit_and_start(): Commits and restarts a
mini-transaction so that it will retain an x-lock on index->lock and
the page of the cursor. This is invoked when
btr_cur_pessimistic_update() returns *big_rec != NULL.
In all callers of btr_cur_pessimistic_update() that do not pass
BTR_KEEP_POS_FLAG, assert that *big_rec == NULL.
btr_cur_compress(): Unused function [in the built-in MySQL 5.1], remove.
page_rec_get_nth(): Return the nth record on the page (an inverse
function of page_rec_get_n_recs_before()). Refactored from
page_get_middle_rec().
page_get_middle_rec(): Invoke page_rec_get_nth().
page_cur_insert_rec_zip_reorg(): Make use of the page directory
shortcuts in page_rec_get_nth() instead of scanning the whole list of
records.
row_ins_clust_index_entry_by_modify(): Pass BTR_KEEP_POS_FLAG to
btr_cur_pessimistic_update().
row_ins_index_entry_low(): If row_ins_clust_index_entry_by_modify()
returns a big_rec, invoke btr_cur_mtr_commit_and_start() in order to
commit and start the mini-transaction without releasing the x-locks on
index->lock and the cursor page, and write the big_rec. Releasing the
page latch in mtr_commit() caused a race condition.
row_upd_clust_rec(): Pass BTR_KEEP_POS_FLAG to
btr_cur_pessimistic_update(). If it returns a big_rec, invoke
btr_cur_mtr_commit_and_start() in order to commit and start the
mini-transaction without releasing the x-locks on index->lock and the
cursor page, and write the big_rec. Releasing the page latch in
mtr_commit() caused a race condition.
sync_thread_add_level(): Add the parameter ibool relock. When TRUE,
bypass the latching order rules.
rw_lock_add_debug_info(): For nested X-lock requests, pass relock=TRUE
to sync_thread_add_level().
rb:678 approved by Jimmy Yang
FAIL IN EMBEDDED SERVER
FreeBSD 64 bit needs the FP_X_DNML to fpsetmask() to prevent exceptions from
propagating into mysql (as a threaded application).
However fpsetmask() itself is deprecated in favor of fedisableexcept().
1. Fixed the #ifdef to check for FP_X_DNML instead of i386.
2. Added a configure.in check for fedisableexcept() and, if present,
this function is called insted of the fpsetmask().
No need for new tests, as the existing tests cover this already.
Removed the affected tests from the experimental list.
