Replacing binlog_row_based_if_mixed with variable binlog_stmt_flags
holding several flags and adding member functions to manipulate the
flags.
Added code to generate a warning when an attempt to log an unsafe
statement to the binary log was made. The warning is both pushed to the
SHOW WARNINGS table and written to the error log. The prevent flooding
the error log, the warning is just written to the error log once per
open session.
The following type conversions was done:
- Changed byte to uchar
- Changed gptr to uchar*
- Change my_string to char *
- Change my_size_t to size_t
- Change size_s to size_t
Removed declaration of byte, gptr, my_string, my_size_t and size_s.
Following function parameter changes was done:
- All string functions in mysys/strings was changed to use size_t
instead of uint for string lengths.
- All read()/write() functions changed to use size_t (including vio).
- All protocoll functions changed to use size_t instead of uint
- Functions that used a pointer to a string length was changed to use size_t*
- Changed malloc(), free() and related functions from using gptr to use void *
as this requires fewer casts in the code and is more in line with how the
standard functions work.
- Added extra length argument to dirname_part() to return the length of the
created string.
- Changed (at least) following functions to take uchar* as argument:
- db_dump()
- my_net_write()
- net_write_command()
- net_store_data()
- DBUG_DUMP()
- decimal2bin() & bin2decimal()
- Changed my_compress() and my_uncompress() to use size_t. Changed one
argument to my_uncompress() from a pointer to a value as we only return
one value (makes function easier to use).
- Changed type of 'pack_data' argument to packfrm() to avoid casts.
- Changed in readfrm() and writefrom(), ha_discover and handler::discover()
the type for argument 'frmdata' to uchar** to avoid casts.
- Changed most Field functions to use uchar* instead of char* (reduced a lot of
casts).
- Changed field->val_xxx(xxx, new_ptr) to take const pointers.
Other changes:
- Removed a lot of not needed casts
- Added a few new cast required by other changes
- Added some cast to my_multi_malloc() arguments for safety (as string lengths
needs to be uint, not size_t).
- Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done
explicitely as this conflict was often hided by casting the function to
hash_get_key).
- Changed some buffers to memory regions to uchar* to avoid casts.
- Changed some string lengths from uint to size_t.
- Changed field->ptr to be uchar* instead of char*. This allowed us to
get rid of a lot of casts.
- Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar
- Include zlib.h in some files as we needed declaration of crc32()
- Changed MY_FILE_ERROR to be (size_t) -1.
- Changed many variables to hold the result of my_read() / my_write() to be
size_t. This was needed to properly detect errors (which are
returned as (size_t) -1).
- Removed some very old VMS code
- Changed packfrm()/unpackfrm() to not be depending on uint size
(portability fix)
- Removed windows specific code to restore cursor position as this
causes slowdown on windows and we should not mix read() and pread()
calls anyway as this is not thread safe. Updated function comment to
reflect this. Changed function that depended on original behavior of
my_pwrite() to itself restore the cursor position (one such case).
- Added some missing checking of return value of malloc().
- Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow.
- Changed type of table_def::m_size from my_size_t to ulong to reflect that
m_size is the number of elements in the array, not a string/memory
length.
- Moved THD::max_row_length() to table.cc (as it's not depending on THD).
Inlined max_row_length_blob() into this function.
- More function comments
- Fixed some compiler warnings when compiled without partitions.
- Removed setting of LEX_STRING() arguments in declaration (portability fix).
- Some trivial indentation/variable name changes.
- Some trivial code simplifications:
- Replaced some calls to alloc_root + memcpy to use
strmake_root()/strdup_root().
- Changed some calls from memdup() to strmake() (Safety fix)
- Simpler loops in client-simple.c
This pads the value of CHAR columns with spaces up to full column length (according to ANSI)
It's not makde part of oracle or ansi mode yet, as this would cause a notable behaviour change.
Added uuid_short(), a generator for increasing 'unique' longlong integers (8 bytes)
The MBR-prefixed function aliases (e.g., MBROVERLAPS()) were missing in 5.1, after
some code refactoring left them out. This simply adds them into the func_array list.
The problem was that some facilities (like CONVERT_TZ() function or
server HELP statement) may require implicit access to some tables in
'mysql' database. This access was done by ordinary means of adding
such tables to the list of tables the query is going to open.
However, if we issued LOCK TABLES before that, we would get "table
was not locked" error trying to open such implicit tables.
The solution is to treat certain tables as MySQL system tables, like
we already do for mysql.proc. Such tables may be opened for reading
at any moment regardless of any locks in effect. The cost of this is
that system table may be locked for writing only together with other
system tables, it is disallowed to lock system tables for writing and
have any other lock on any other table.
After this patch the following tables are treated as MySQL system
tables:
mysql.help_category
mysql.help_keyword
mysql.help_relation
mysql.help_topic
mysql.proc (it already was)
mysql.time_zone
mysql.time_zone_leap_second
mysql.time_zone_name
mysql.time_zone_transition
mysql.time_zone_transition_type
These tables are now opened with open_system_tables_for_read() and
closed with close_system_tables(), or one table may be opened with
open_system_table_for_update() and closed with close_thread_tables()
(the latter is used for mysql.proc table, which is updated as part of
normal MySQL server operation). These functions may be used when
some tables were opened and locked already.
NOTE: online update of time zone tables is not possible during
replication, because there's no time zone cache flush neither on LOCK
TABLES, nor on FLUSH TABLES, so the master may serve stale time zone
data from cache, while on slave updated data will be loaded from the
time zone tables.
Before this fix, a call to a User Defined Function (UDF) could,
under some circumstances, be interpreted as a call to a Stored function
instead. This occurred if a native function was invoked in the parameters
for the UDF, as in "select my_udf(abs(x))".
The root cause of this defect is the introduction, by the fix for Bug 21809,
of st_select_lex::udf_list, and it's usage in the parser in sql_yacc.yy
in the rule function_call_generic (in 5.1).
While the fix itself for Bug 21809 is correct in 5.0, the code change
merged into the 5.1 release created the issue, because the calls in 5.1 to :
- lex->current_select->udf_list.push_front(udf)
- lex->current_select->udf_list.pop()
are not balanced in case of native functions, causing the udf_list,
which is really a stack, to be out of sync with the internal stack
maintained by the bison parser.
Instead of moving the call to udf_list.pop(), which would have fixed the
symptom, this patch goes further and removes the need for udf_list.
This is motivated by two reasons:
a) Maintaining a stack in the MySQL code in sync with the stack maintained
internally in sql_yacc.cc (not .yy) is extremely dependent of the
implementation of yacc/bison, and extremely difficult to maintain.
It's also totally dependent of the structure of the grammar, and has a risk
to break with regression defects each time the grammar itself is changed.
b) The previous code did report construct like "foo(expr AS name)" as
syntax errors (ER_PARSER_ERROR), which is incorrect, and misleading.
The syntax is perfectly valid, as this expression is valid when "foo" is
a UDF. Whether this syntax is legal or not depends of the semantic of "foo".
With this change:
a) There is only one stack (in bison), and no List<udf_func> to maintain.
b) "foo(expr AS name)", when used incorrectly, is reported as semantic error:
- ER_WRONG_PARAMETERS_TO_NATIVE_FCT (for native functions)
- ER_WRONG_PARAMETERS_TO_STORED_FCT (for stored functions)
This is achieved by the changes implemented in item_create.cc
Before this change, the functions BENCHMARK, ENCODE, DECODE and FORMAT could
only accept a constant for some parameters.
After this change, this restriction has been removed. An implication is that
these functions can also be used in prepared statements.
The change consist of changing the following classes:
- Item_func_benchmark
- Item_func_encode
- Item_func_decode
- Item_func_format
to:
- only accept Item* in the constructor,
- and evaluate arguments during calls to val_xxx()
which fits the general design of all the other functions.
The 'TODO' items identified in item_create.cc during the work done for
Bug 21114 are addressed by this fix, as a natural consequence of aligning
the design.
In the 'func_str' test, a single very long test line involving an explain
extended select with many functions has been rewritten into multiple
separate tests, to improve maintainability.
The result of explain extended select decode(encode(...)) has changed,
since the encode and decode functions now print all their parameters.
Due to the complexity of this change, everything is documented in WL#3565
This patch is the third iteration, it takes into account the comments
received to date.
1) Fix for BUG#19630 "stored function inserting into two auto_increment breaks
statement-based binlog":
a stored function inserting into two such tables may fail to replicate
(inserting wrong data in the slave's copy of the second table) if the slave's
second table had an internal auto_increment counter different from master's.
Because the auto_increment value autogenerated by master for the 2nd table
does not go into binlog, only the first does, so the slave lacks information.
To fix this, if running in mixed binlogging mode, if the stored function or
trigger plans to update two different tables both having auto_increment
columns, we switch to row-based for the whole function.
We don't have a simple solution for statement-based binlogging mode, there
the bug remains and will be documented as a known problem.
Re-enabling rpl_switch_stm_row_mixed.
2) Fix for BUG#20630 "Mixed binlogging mode does not work with stored
functions, triggers, views", which was a documented limitation (in mixed
mode, we didn't detect that a stored function's execution needed row-based
binlogging (due to some UUID() call for example); same for
triggers, same for views (a view created from a SELECT UUID(), and doing
INSERT INTO sometable SELECT theview; would not replicate row-based).
This is implemented by, after parsing a routine's body, remembering in sp_head
that this routine needs row-based binlogging. Then when this routine is used,
the caller is marked to require row-based binlogging too.
Same for views: when we parse a view and detect that its SELECT needs
row-based binary logging, we mark the calling LEX as such.
3) Fix for BUG#20499 "mixed mode with temporary table breaks binlog":
a temporary table containing e.g. UUID has its changes not binlogged,
so any query updating a permanent table with data from the temporary table
will run wrongly on slave. Solution: in mixed mode we don't switch back
from row-based to statement-based when there exists temporary tables.
4) Attempt to test mysqlbinlog on a binlog generated by mysqlbinlog;
impossible due to BUG#11312 and BUG#20329, but test is in place for when
they are fixed.
invoker name
The bug was fixed similar to how context switch is handled in
Item_func_sp::execute_impl(): we store pointer to current
Name_resolution_context in Item_func_current_user class, and use
its Security_context in Item_func_current_user::fix_fields().
CONNECTION_ID() was implemented as a constant Item, i.e. an instance of
Item_static_int_func class holding value computed at creation time.
Since Items are created on parsing, and trigger statements are parsed
on table open, the first connection to open a particular table would
effectively set its own CONNECTION_ID() inside trigger statements for
that table.
Re-implement CONNECTION_ID() as a class derived from Item_int_func, and
compute connection_id on every call to fix_fields().
- detect the need for row-based binlogging not at execution stage but earlier at parsing stage; needed for example for CREATE TABLE SELECT UUID().
- more tests of this mixed mode.
libmysqld/Makefile.am:
sql/Makefile.am:
Adding new source files.
Adding new file into build process.
include/my_xml.h:
strings/xml.c:
Adding new XML parse flags to skip text normalization and
to use relative tag names. Adding enum for XML token types.
sql/lex.h:
Making parser aware of new SQL functions.
sqll/item_create.h, sql/item_create.cc:
Adding creators for ExtractValue and UpdateXML.
sql/item.h:
Adding new Item types: nodeset and nodeset comparator.
sql/item_xmlfunc.h
sql/item_xmlfunc.cc
Adding new classes implementing XPath functions.
mysql-test/t/xml.test, mysql-test/r/xml.result:
New files: adding test case
The table opening process now works the following way:
- Create common TABLE_SHARE object
- Read the .frm file and unpack it into the TABLE_SHARE object
- Create a TABLE object based on the information in the TABLE_SHARE
object and open a handler to the table object
Other noteworthy changes:
- In TABLE_SHARE the most common strings are now LEX_STRING's
- Better error message when table is not found
- Variable table_cache is now renamed 'table_open_cache'
- New variable 'table_definition_cache' that is the number of table defintions that will be cached
- strxnmov() calls are now fixed to avoid overflows
- strxnmov() will now always add one end \0 to result
- engine objects are now created with a TABLE_SHARE object instead of a TABLE object.
- After creating a field object one must call field->init(table) before using it
- For a busy system this change will give you:
- Less memory usage for table object
- Faster opening of tables (if it's has been in use or is in table definition cache)
- Allow you to cache many table definitions objects
- Faster drop of table
"Interleaved SPs execution is now binlogged properly, "SELECT spfunc()" is binlogged too.
The known remaining issue is binlogging/replication of "a routine is deleted while it is executed" scenario.
