It is now possible to kill query by query id. KILL syntax was extended to:
KILL [HARD | SOFT] [CONNECTION | QUERY [ID query_id]] [thread_id | USER user_name]
Added QUERY_ID column to INFORMATION_SCHEMA.PROCESSLIST.
Fixed tests affected by this change:
- added PROCESSLIST.QUERY_ID column
- ID is now keyword and is quoted in SHOW CREATE TABLE output
- PFS statement digest is calculated basing on token id
(not token text). Token id has shifted for keywords residing
after ID in keywords array.
- YYPARSE_PARAM and YYLEX_PARAM are removed in Bison 3.0. Deprecated
since Bison 1.875 in favor of %lex-param, %parse-param.
- %parse-param adds an argument to yyerror() as well, updated
MYSQLerror() accordingly.
- %parse-param allows to declare proper type for argument. That's
what 99% of this patch is about.
LOAD DATA CAN CAUSE SQL INJECTION
Problem:
=======
A long SET expression in LOAD DATA is incorrectly truncated
when written to the binary log.
Analysis:
========
LOAD DATA statements are reconstructed once again before
they are written to the binary log. When SET clauses are
specified as part of LOAD DATA statement, these SET clause
user command strings need to be stored as it is inorder to
reconstruct the original user command. At present these
strings are stored as part of SET clause item tree's
top most Item node's name itself which is incorrect. As an
Item::name can be of MAX_ALIAS_NAME (256) size. Hence the
name will get truncated to "255".
Because of this the rewritten LOAD DATA statement will be
terminated incorrectly. When this statment is read back by
the mysqlbinlog tool it reads a starting single quote and
continuos to read till it finds an ending quote. Hence any
statement written post ending quote will be considered as
a new statement.
Fix:
===
As name field has length restriction the string value
should not be stored in Item::name. A new String list is
maintained to store the SET expression values and this list
is read during reconstrution.
sql/sql_lex.cc:
Clear the load data set string list during each query
execution.
sql/sql_lex.h:
Added a new String list to store the load data operation's
SET clause user command strings.
sql/sql_load.cc:
Read the SET clause user command strings from load data
set string list.
sql/sql_yacc.yy:
Store the SET caluse user command string as part of load
data set string list.
Bug#13116514 - CREATE LOGFILE GROUP INITIAL_SIZE & UNDO_BUFFER_SIZE FAILS
Fixing parser to accept the syntax: to give a size with suffix 'M', eg. undo_buffer_size=10M (M for mega bytes), in 'create logfile group' command.
Change of user interface to be more logical and more in line with expectations
to work similar to old-style replication.
User can now explicitly choose in CHANGE MASTER whether binlog position is
taken into account (master_gtid_pos=current_pos) or not (master_gtid_pos=
slave_pos) when slave connects to master.
@@gtid_pos is replaced by three separate variables @@gtid_slave_pos (can
be set by user, replicated GTIDs only), @@gtid_binlog_pos (read only), and
@@gtid_current_pos (a combination of the two, most recent GTID within each
domain). mysql.rpl_slave_state is renamed to mysql.gtid_slave_pos to match.
This fixes MDEV-4474.
Implement START SLAVE UNTIL master_gtid_pos = "<GTID position>".
Add test cases, including a test showing how to use this to promote
a new master among a set of slaves.
MDEV-4489 "Replication of big5, cp932, gbk, sjis strings makes wrong values on slave"
has been fixed.
Problem:
String constants of some Asian charsets (big5,cp932,gbk,sjis)
can have backslash '\' (0x5C) in the second byte of multi-byte characters.
Replicating of such constants using the standard '\'-escaping is dangerous.
Therefore, constants of these charsets are replicated using hex notation:
INSERT INTO t1 (a) VALUES (0x815C);
However, 0xHHHH constants do not work well in some cases,
because they can behave as strings and as numbers, depending on context
(for example, depending on the data type of the column in an INSERT statement).
This SQL script was not replicated correctly with statement-based replication:
SET NAMES gbk;
PREPARE STMT FROM 'INSERT INTO t1 (a) VALUES (?)';
SET @a = '1';
EXECUTE STMT USING @a;
The INSERT statement was replicated as:
INSERT INTO t1 (a) VALUES (0x31);
'1' was correctly converted to the number 1 on master.
But the 0x31 constant was treated as number 49 on slave.
Fix:
1. Binary log now uses X'HHHH' instead of 0xHHHH constants.
2. The X'HHHH' constants now work always as strings, in all contexts.
This is the SQL standard compliant behaviour.
After the fix, the above statement is replicated as:
INSERT INTO t1 (a) VALUES (X'31');
X'31' is treated as string '1' on slave, and is correctly converted to 1.
modified:
@ mysql-test/r/ctype_cp932_binlog_stm.result
@ mysql-test/r/select.result
@ mysql-test/r/select_jcl6.result
@ mysql-test/r/select_pkeycache.result
@ mysql-test/r/user_var-binlog.result
@ mysql-test/r/varbinary.result
@ mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.result
@ mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
@ mysql-test/suite/rpl/r/rpl_charset_sjis.result
@ mysql-test/suite/rpl/r/rpl_mdev382.result
@ mysql-test/suite/rpl/t/rpl_charset_sjis.test
@ mysql-test/t/ctype_cp932_binlog_stm.test
@ mysql-test/t/select.test
@ mysql-test/t/varbinary.test
Adding and updating tests
@ sql/item.cc
@ sql/item.h
@ sql/sql_yacc.yy
@ sql/sql_lex.cc
Splitting the implementations of X'HH' and 0xHH constants into two
separate classes. Fixing the parser to distinguish the two syntaxes.
@ sql/log_event.cc
Using X'HH' instead of 0xHH for binary logging for string constants
of the "dangerous" charsets.
@ sql/sql_string.h
Adding a helped method String::append_hex().
Merge of 10.0-mdev26 feature tree into 10.0-base.
Global transaction ID is prepended to each event group in the binlog.
Slave connect can request to start from GTID position instead of specifying
file name/offset of master binlog. This facilitates easy switch to a new
master.
Slave GTID state is stored in a table mysql.rpl_slave_state, which can be
InnoDB to get crash-safe slave state.
GTID includes a replication domain ID, allowing to keep track of distinct
positions for each of multiple masters.
!TABLES->NEXT_NAME_RESOLUTION_TABLE) || !TAB
Problem:
The context info of select query gets corrupted when a query
with group_concat having order by is present in an order by
clause of the select query. As a result, server crashes with
an assert.
Analysis:
While parsing order by for group_concat, it is presumed that
it is always present before the actual order by for the
select query.
As a result, parser uses select->order_list to populate the
order by items of group_concat and creates a select->gorder_list
to which select->order_list is copied onto. Once this is done,
it empties the select->order_list.
In the case presented in the bugpage, as order by is already
parsed when group_concat's order by is encountered, parser
presumes that it is the second order by in the select query
and creates fake_lex_unit which results in the change of
context info.
Solution:
Make group_concat's order by parsing independent of the select
sql/item_sum.cc:
Change the argument as, select->gorder_list is not pointer anymore
sql/item_sum.h:
Change the argument as, select->gorder_list is not pointer anymore
sql/mysql_priv.h:
Parsing for group_concat's order by is made independent.
As a result, add_order_to_list cannot be used anymore.
sql/sql_lex.cc:
Parsing for group_concat's order by is made independent.
As a result, add_order_to_list cannot be used anymore.
sql/sql_lex.h:
Parsing for group_concat's order by is made independent.
As a result, add_order_to_list cannot be used anymore.
sql/sql_yacc.yy:
Make group_concat's order by parsing independent of the select
queries order by.
Syntax modified to allow statements:
ALTER TABLE ADD/DROP COLUMN
ALTER TABLE ADD/DROP INDEX
ALTER TABLE ADD/DROP FOREIGN KEY
ALTER TABLE ADD/DROP PARTITION
ALTER TABLE CHANGE COLUMN
ALTER TABLE MODIFY COLUMN
DROP INDEX
to have IF (NOT) EXISTS options.
Appropriate implementations added to mysql_alter_table().
per-file comments:
mysql-test/r/alter_table.result
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
test result updated.
mysql-test/r/fulltext.result
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
mysql-test/r/partition.result
test result updated.
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
mysql-test/t/alter_table.test
tests added.
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
mysql-test/t/fulltext.test
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
tests added.
mysql-test/t/partition.test
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
tests added.
sql/field.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
create_if_not_exists field added.
sql/field.h
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
create_if_not_exists field added.
sql/partition_info.h
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
has_unique_name made public.
sql/sp_head.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
sql/sql_class.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
create_if_not_exists inited.
sql/sql_class.h
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
create_if_not_exists inited.
sql/sql_lex.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
check_exists inited.
sql/sql_lex.h
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
check_exists inited.
sql/sql_parse.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
check_exists inited.
sql/sql_table.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
handle_if_exists_options() added.
it's called in mysql_alter_table().
sql/sql_trigger.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
check_exists instead of drop_if_exists.
sql/sql_view.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
check_exists instead of drop_if_exists.
sql/sql_yacc.yy
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
sintax modified.
Replace CHANGE MASTER TO ... master_gtid_pos='xxx' with a new system
variable @@global.gtid_pos.
This is more logical; @@gtid_pos is global, not per-master, and it is not
affected by RESET SLAVE.
Also rename master_gtid_pos=AUTO to master_use_gtid=1, which again is more
logical.
Post push fix:
setup_ref_array() now uses n_sum_items to determine size of ref_pointer_array.
The problem was that n_sum_items kept growing, it wasn't reset for each query.
A similar memory leak was fixed with the patch for:
Bug 14683676 ENDLESS MEMORY CONSUMPTION IN SETUP_REF_ARRAY WITH MAX IN SUBQUERY
sql/sql_yacc.yy:
Reset parsing_place when we're done parsing SHOW commands,
to prevent Item::Item incrementing select_n_having_items
(which is also used in setup_ref_array())
STRAIGHT_JOIN couldn't be combined with NATURAL or USING(),
INNER JOIN not with NATURAL (MDEV-4271, MySQL Bug #35268)
Separate rules existed for "natural" (non-outer) joins and
for STRAIGHT_JOIN, with the only difference code wise being
that with STRAIGHT_JOIN the "straight" property of the right
side table was set before calling the appropriate add_...()
function.
The "natural_join" parser rule has now been extended to also
accept STRAIGHT_JOIN, and the rule result value is set to
1 for straight joins, 0 otherwise, so acting as a "straight"
flag that can directly be assigned to the "straight" property
of the right side table.
The rule parsing NATURAL JOIN was hard coded to accept just
this keyword combination, without support for either
STRAIGHT_JOIN or the optional INNER.
The "natural_join" rule has now been split up in an inner
"inner_join" rule that matches the JOIN, INNER JOIN and
STRAIGHT_JOIN cases while "natural_join" also matches
CROSS JOIN.
The NATURAL rule has been changed to accept "inner_join"
instead of just JOIN, so now NATURAL STRAIGHT_JOIN and
NATURAL INNER JOIN also work as expected.
As a side effect the removal of the duplciated rules
for STRAIGHT_JOIN handling has reduced the shift/reduce
conflict count by one.
mysql-test/r/join.result:
Added new test cases
mysql-test/t/join.test:
Added new test cases
sql/sql_yacc.yy:
The "natural_join" parser rule was extended to also accept STRAIGHT_JOIN
NATURAL STRAIGHT_JOIN and NATURAL INNER JOIN also now work as expected
mysql-test/r/keywords.result:
Test that option works as table/column/variable
mysql-test/suite/funcs_1/r/storedproc.result:
OPTION is now a valid identifier
mysql-test/suite/funcs_1/t/storedproc.test:
OPTION is now a valid identifier
mysql-test/t/keywords.test:
Test that option works as table/column/variable
sql/sql_yacc.yy:
OPTION is now a valid identifier
Due to an internal change in the server code in between 5.1 and 5.5
(wl#2649) the hash function used in KEY partitioning changed
for numeric and date/time columns (from binary hash calculation
to character based hash calculation).
Also enum/set changed from latin1 ci based hash calculation to
binary hash between 5.1 and 5.5. (bug#11759782).
These changes makes KEY [sub]partitioned tables on any of
the affected column types incompatible with 5.5 and above,
since the calculation of partition id differs.
Also since InnoDB asserts that a deleted row was previously
read (positioned), the server asserts on delete of a row that
is in the wrong partition.
The solution for this situation is:
1) The partitioning engine will check that delete/update will go to the
partition the row was read from and give an error otherwise, consisting
of the rows partitioning fields. This will avoid asserts in InnoDB and
also alert the user that there is a misplaced row. A detailed error
message will be given, including an entry to the error log consisting
of both table name, partition and row content (PK if exists, otherwise
all partitioning columns).
2) A new optional syntax for KEY () partitioning in 5.5 is allowed:
[SUB]PARTITION BY KEY [ALGORITHM = N] (list_of_cols)
Where N = 1 uses the same hashing as 5.1 (Numeric/date/time fields uses
binary hashing, ENUM/SET uses charset hashing) N = 2 uses the same
hashing as 5.5 (Numeric/date/time fields uses charset hashing,
ENUM/SET uses binary hashing). If not set on CREATE/ALTER it will
default to 2.
This new syntax should probably be ignored by NDB.
3) Since there is a demand for avoiding scanning through the full
table, during upgrade the ALTER TABLE t PARTITION BY ... command is
considered a no-op (only .frm change) if everything except ALGORITHM
is the same and ALGORITHM was not set before, which allows manually
upgrading such table by something like:
ALTER TABLE t PARTITION BY KEY ALGORITHM = 1 () or
ALTER TABLE t PARTITION BY KEY ALGORITHM = 2 ()
4) Enhanced partitioning with CHECK/REPAIR to also check for/repair
misplaced rows. (Also works for ALTER TABLE t CHECK/REPAIR PARTITION)
CHECK FOR UPGRADE:
If the .frm version is < 5.5.3
and uses KEY [sub]partitioning
and an affected column type
then it will fail with an message:
KEY () partitioning changed, please run:
ALTER TABLE `test`.`t1` PARTITION BY KEY ALGORITHM = 1 (a)
PARTITIONS 12
(i.e. current partitioning clause, with the addition of
ALGORITHM = 1)
CHECK without FOR UPGRADE:
if MEDIUM (default) or EXTENDED options are given:
Scan all rows and verify that it is in the correct partition.
Fail for the first misplaced row.
REPAIR:
if default or EXTENDED (i.e. not QUICK/USE_FRM):
Scan all rows and every misplaced row is moved into its correct
partitions.
5) Updated mysqlcheck (called by mysql_upgrade) to handle the
new output from CHECK FOR UPGRADE, to run the ALTER statement
instead of running REPAIR.
This will allow mysql_upgrade (or CHECK TABLE t FOR UPGRADE) to upgrade
a KEY [sub]partitioned table that has any affected field type
and a .frm version < 5.5.3 to ALGORITHM = 1 without rebuild.
Also notice that if the .frm has a version of >= 5.5.3 and ALGORITHM
is not set, it is not possible to know if it consists of rows from
5.1 or 5.5! In these cases I suggest that the user does:
(optional)
LOCK TABLE t WRITE;
SHOW CREATE TABLE t;
(verify that it has no ALGORITHM = N, and to be safe, I would suggest
backing up the .frm file, to be used if one need to change to another
ALGORITHM = N, without needing to rebuild/repair)
ALTER TABLE t <old partitioning clause, but with ALGORITHM = N>;
which should set the ALGORITHM to N (if the table has rows from
5.1 I would suggest N = 1, otherwise N = 2)
CHECK TABLE t;
(here one could use the backed up .frm instead and change to a new N
and run CHECK again and see if it passes)
and if there are misplaced rows:
REPAIR TABLE t;
(optional)
UNLOCK TABLES;
