Commit graph

2685 commits

Author SHA1 Message Date
Eugene Kosov
a82e42fd13 NFC: refactor Field::is_equal() and related stuff
Make Field::is_equal() const and return bool as it's a naturally fitting
type for it. Also it's agrument was narrowed to Column_definition.

InnoDB can change type of some columns by itself. InnoDB-specific code used to
reside in Field_xxx:is_equal() methods. Now engine-specific stuff was
moved to a virtual methods of handler::can_convert{string,varstring,blob,geom}.
These methods are called by Field::can_be_converted_by_engine() which is a
double dispatch pattern.

Some InnoDB-specific code still resides in compare_keys_but_name(). It should
be moved from here someday to handler::compare_key_parts(...) or similar.

IS_EQUAL_WITH_REINTERPRET_COMPATIBLE_CHARSET
IS_EQUAL_WITH_REINTERPRET_COMPATIBLE_CHARSET_BUT_COLLATE: both was removed

IS_EQUAL_NO, IS_EQUAL_YES are not needed now and should be removed
along with deprecated handler::check_if_incompatible_data().

HA_EXTENDED_TYPES_CONVERSION: was removed as such logic is not needed now by
server code.

ALTER_COLUMN_EQUAL_PACK_LENGTH: was renamed to a more generic
ALTER_COLUMN_TYPE_CHANGE_BY_ENGINE
2019-06-22 14:09:12 +03:00
Oleksandr Byelkin
f66d1850ac Merge branch '10.3' into 10.4 2019-06-14 22:10:50 +02:00
Oleksandr Byelkin
4a3d51c76c Merge branch '10.2' into 10.3 2019-06-14 07:36:47 +02:00
Marko Mäkelä
4bbd8be482 Merge 10.1 into 10.2 2019-06-12 10:30:01 +03:00
Sergei Golubchik
27fcdb161c MDEV-16249 CHECKSUM TABLE for a spider table is not parallel and saves all data in memory in the spider head by default (#1328)
followup for be5c432a42

ha_partition::calculate_checksum() has to invoke calculate_checksum()
for partitions unconditionally, not under (HA_HAS_OLD_CHECKSUM | HA_HAS_NEW_CHECKSUM).
Because the server uses ::info() to ask for a live checksum, while
calculate_checksum() must, precisely, calculate it the slow way,
also for tables that don't have the live checksum at all.

Also, fix the compilation on Windows (ha_checksum/ulonglong type mix).
2019-06-11 18:42:45 +02:00
Kentoku SHIBA
be5c432a42
MDEV-16249 CHECKSUM TABLE for a spider table is not parallel and saves all data in memory in the spider head by default (#1328)
add checksum_null for setting null value of checksum
2019-06-11 00:25:08 +09:00
Alexander Barkov
5a19908b95 MDEV-19653 Add class Sql_cmd_create_table 2019-05-31 16:22:53 +04:00
Marko Mäkelä
cf77951fb6 Merge 10.3 into 10.4 2019-05-22 08:42:31 +03:00
Eugene Kosov
6473641b9a MDEV-18512 using DATETIME(6) as row_start/row_end crashes server
Disallow DATETIME for SYSTEM VERSIONING tables.
2019-05-20 15:19:01 +04:00
Oleksandr Byelkin
c07325f932 Merge branch '10.3' into 10.4 2019-05-19 20:55:37 +02:00
Sergei Golubchik
d199591cf2 generalize the error message 2019-05-18 20:34:03 +02:00
Eugene Kosov
73de06c48f make method const
Closes #677
2019-05-17 13:53:22 +02:00
Eugene Kosov
3d649c6e37 MDEV-15408 Confusing error message upon ER_VERS_FIELD_WRONG_TYPE while omitting UNSIGNED in BIGINT
Improve diagnostics. Try to guess what type user tried to type.
2019-05-17 13:53:22 +02:00
Marko Mäkelä
73e038520f Merge 10.2 into 10.3 2019-05-14 18:10:06 +03:00
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Sujatha
43bbf88dcb MDEV-19158: MariaDB 10.2.22 is writing duplicate entries into binary log
Problem:
========
We have a Master/Master Setup on two servers, but are only writing to one of
those servers (so it is essentially Master/Slave) We upgraded from 10.1.* to
10.2.22 last week and starting with the upgrade, we are getting duplicate key
errors on the slave. BINLOG=mixed.

Analysis:
=========
This issue happens with LOCK TABLES and binlog_format=MIXED combination. When an
UNSAFE statement is encountered in 'MIXED' mode, it is logged in the form of
'ROW' format. For all the tables that are part of LOCK TABLES list their table maps
are written into the binary log. For each table in the list a check is
done to see if 'check_table_binlog_row_based_done' flag is set or not. If it is not set
a check process is initiated to see if table qualifies for row based binary
logging or not and 'check_table_binlog_row_based_done' is set. This flag will be
cleared at the time of closing thread tables.

But there can be special cases where the LOCK TABLES contains more number of
tables but the unsafe query is actually using subset of tables from LOCK TABLES
list.

For example: LOCK TABLES locks t1,t2,t3 but the unsafe statement makes use of
only two tables t1,t3. In this case the 'check_table_binlog_row_based_done' flag
is enabled for table 't2' while writing table map, but 'close_thread_tables'
function call will not reset this flag. Since the flag is not cleared for table
't2' even a safe statement which used t2 will be logged in the form of row based
format.

This leads to an assert on debug builds and causes duplicate entries in release
builds. In release builds a statement is logged in the form of both ROW and
STATEMENT format. This causes the slave to fail with duplicate key error.

Fix:
===
During 'close_thread_tables' when LOCK TABLE modes are active "ha_reset" is done
for all the tables which were part of current statement. As mentioned in the
example 'ha_reset' is called for tables 't1' and 't3'. This will clear the
'check_table_binlog_row_based_done' flag. At this point add a check for the rest
of the tables to see if 'check_table_binlog_row_based_done' is enabled or not.
If enabled clear the flag.
2019-05-14 16:06:55 +05:30
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Vicențiu Ciorbaru
f177f125d4 Merge branch '5.5' into 10.1 2019-05-11 19:15:57 +03:00
Vicențiu Ciorbaru
15f1e03d46 Follow-up to changing FSF address
Some places didn't match the previous rules, making the Floor
address wrong.

Additional sed rules:

sed -i -e 's/Place.*Suite .*, Boston/Street, Fifth Floor, Boston/g'
sed -i -e 's/Suite .*, Boston/Fifth Floor, Boston/g'
2019-05-11 18:30:45 +03:00
Sergei Golubchik
ffb83ba650 cleanup: move checksum code to handler class
make live checksum to be returned in handler::info(),
and slow table-scan checksum to be calculated in handler::checksum().

part of
MDEV-16249 CHECKSUM TABLE for a spider table is not parallel and saves all data in memory in the spider head by default
2019-05-07 18:40:36 +02:00
Marko Mäkelä
b132b8895e Merge 10.3 into 10.4 2019-05-05 10:23:14 +03:00
Sachin
27980b0f83 MDEV-19365 Assertion failure in LONG Unique after 10.3 merge
If handler->inited is RND use cloned handler for long unique duplicate search.
2019-05-05 10:19:22 +03:00
Marko Mäkelä
4d59f45260 Merge 10.2 into 10.3 2019-04-27 20:41:31 +03:00
Oleksandr Byelkin
4e01bc8c96 MDEV-16240: Assertion `0' failed in row_sel_convert_mysql_key_to_innobase
Set table in row ID position mode before using this function.
2019-04-25 18:02:31 +02:00
Marko Mäkelä
e6bdf77e4b Merge 10.3 into 10.4
In is_eits_usable(), we disable an assertion that fails due to
MDEV-19334.
2019-04-25 16:05:20 +03:00
Sergey Vojtovich
b7fd7ce286 Moved normal transaction xid to implicit_xid
Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)
2019-04-25 15:06:40 +04:00
Sergey Vojtovich
a168cfb396 Move XID_state::xa_state handing inside xa.cc
Let xid_cache_insert()/xid_cache_delete() handle xa_state.

Let session tracker use is_explicit_XA() rather than xa_state != XA_NOTR.

Fixed open_tables() to refuse data access in XA_ROLLBACK_ONLY state.

Removed dead code from THD::cleanup(). It was supposed to be a reminder,
but it got messed up over time.

spider_internal_start_trx() is called either with XA_NOTR or XA_ACTIVE,
which is guarded by server callers. Thus is_explicit_XA() is acceptable
replacement for XA_ACTIVE check (which was likely wrong anyway).

Setting xa_state to XA_PREPARED in spider_internal_xa_prepare() isn't
meaningful, as this value is never accessed later. It can't be accessed
by current thread and it can't be recovered either. It can only be
accessed by spider internally, which never happens.

Make spider_xa_lock()/spider_xa_unlock() static.

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)
2019-04-25 15:06:40 +04:00
Sergey Vojtovich
f189f34ed4 Move XID_STATE::rm_error to XID_cache_element
XID_STATE::rm_error is never used by internal 2PC, it is intended to be
used by explicit XA only.

Also removed redundant xid reset from THD::init_for_queries(). Must've
been done already either by THD::transaction constructor or by
THD::cleanup().

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)
2019-04-25 15:06:40 +04:00
Sergey Vojtovich
07140f171d Just move, no code changes otherwise.
Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)
2019-04-25 15:06:40 +04:00
Marko Mäkelä
acf6f92aa9 Merge 10.2 into 10.3 2019-04-25 09:05:52 +03:00
Marko Mäkelä
bc145193c1 Merge 10.1 into 10.2 2019-04-25 09:04:09 +03:00
Marko Mäkelä
bfb0726fc2 Merge 5.5 into 10.1 2019-04-24 12:03:11 +03:00
Thirunarayanan Balathandayuthapani
d5da8ae04d MDEV-15772 Potential list overrun during XA recovery
InnoDB could return the same list again and again if the buffer
passed to trx_recover_for_mysql() is smaller than the number of
transactions that InnoDB recovered in XA PREPARE state.

We introduce the transaction state TRX_PREPARED_RECOVERED, which
is like TRX_PREPARED, but will be set during trx_recover_for_mysql()
so that each transaction will only be returned once.

Because init_server_components() is invoking ha_recover() twice,
we must reset the state of the transactions back to TRX_PREPARED
after returning the complete list, so that repeated traversals
will see the complete list again, instead of seeing an empty list.
Without this tweak, the test main.tc_heuristic_recover would hang
in MariaDB 10.1.
2019-04-24 11:46:14 +03:00
Eugene Kosov
3a3d5ba235 MDEV-13301 Optimize DROP INDEX, ADD INDEX into RENAME INDEX
Just rename index in data dictionary and in InnoDB cache when it's possible.
Introduce ALTER_INDEX_RENAME for that purpose so that engines can optimize
such operation.

Unused code between macro MYSQL_RENAME_INDEX was removed.

compare_keys_but_name(): compare index definitions except for index names

Alter_inplace_info::rename_keys:
ha_innobase_inplace_ctx::rename_keys: vector of rename indexes

fill_alter_inplace_info():: fills Alter_inplace_info::rename_keys
2019-04-03 18:36:33 +02:00
Sachin
ba7d33a898 MDEV-18820 Assertion `lock_table_has(trx, index->table, LOCK_IX)' failed in lock_rec_insert_check_and_lock upon INSERT into table with blob key
Don't Ignore Any error during index lookup, And throw duplicate key error
only if error is HA_ERR_FOUND_DUPP_KEY
2019-04-03 12:53:58 +05:30
Marko Mäkelä
5c3ff5cb93 Merge 10.3 into 10.4 2019-04-02 11:04:54 +03:00
Nikita Malyavin
e6230e844c MDEV-15951 system versioning by trx id doesn't work with partitioning
Fix partitioning for trx_id-versioned tables.
`partition by hash`, `range` and others now work.
`partition by system_time` is forbidden.
Currently we cannot use row_start and row_end in `partition by`, because
insertion of versioned field is done by engine's handler, as well as
row_start/row_end's value set up, which is a transaction id -- so it's
also forbidden.

The drawback is that it's now impossible to use `partition by key()`
without parameters for such tables, because it references row_start and
row_end implicitly.

* add handler::vers_can_native()
* drop Table_scope_and_contents_source_st::vers_native()
* drop partition_element::find_engine_flag as unused
* forbid versioning partitioning for trx_id as not supported
* adopt vers tests for trx_id partitioning
* forbid any row_end referencing in `partition by` clauses,
  including implicit `by key()`
2019-03-29 12:51:19 +01:00
Marko Mäkelä
514b305dfb Merge 10.3 into 10.4
The MDEV-17262 commit 26432e49d3
was skipped. In Galera 4, the implementation would seem to require
changes to the streaming replication.

In the tests archive.rnd_pos main.profiling, disable_ps_protocol
for SHOW STATUS and SHOW PROFILE commands until MDEV-18974
has been fixed.
2019-03-20 10:41:32 +02:00
Sergei Golubchik
f38c352172 post-merge: gcc 8 warnings 2019-03-17 13:06:56 +01:00
sachinsetia1001@gmail.com
8995f33c0b MDEV-18889 Long unique on virtual fields crashes server
Use table->record[0] for ha_index_read_map so that vfield gets automatically
be updated.
2019-03-17 13:45:57 +05:30
Marko Mäkelä
2a791c53ad Merge 10.3 into 10.4 2019-03-06 09:00:52 +02:00
Julius Goryavsky
723ffdb32e MDEV-9519: After-merge fix for 10.3 2019-03-05 14:11:42 +01:00
Marko Mäkelä
a2fc36989e Merge 10.2 into 10.3 2019-03-04 17:01:00 +02:00
Teemu Ollakka
a8ff4f243d MDEV-18631 Fix streaming replication with wsrep_gtid_mode=ON
With wsrep_gtid_mode=ON, the appropriate commit hooks were not
called in all cases for applied streaming transactions.

As a fix, removed all special handling of commit order critical
section from Wsrep_high_priority_service and Wsrep_storage_service.
Now commit order critical section is always entered in ha_commit_trans().

Check for wsrep_run_commit_hook is now done in handler.cc, log.cc.
This makes it explicit that the transaction is an active wsrep
transaction which must go through commit hooks.
2019-03-04 09:00:25 +02:00
Jan Lindström
f65f40bb35 Merge remote-tracking branch 'origin/10.1' into 10.2 2019-02-28 13:08:11 +02:00
Jan Lindström
5a87e3ee87 Revert offending part of MDEV-9519: Data corruption will happen on the Galera cluster size change
This will allow test binlog.binlog_stm_binlog to pass more often.
Note that this is not a real fix to that test failure.
2019-02-28 09:29:19 +02:00
Sergei Golubchik
b10340998f MDEV-18748 REPLACE doesn't work with unique blobs on MyISAM table
on long unique conflict, set table->file->dup_ref for
engines that support it
2019-02-27 23:27:43 -05:00
Sergei Golubchik
304f0084ef MDEV-18720 Assertion `inited==NONE' failed in ha_index_init upon update on versioned table with key on blob
* update system versioning fields before generaled columns
* don't presume that ha_write_row() means INSERT. It could still be UPDATE
* use the correct handler in check_duplicate_long_entry_key()
2019-02-27 23:27:43 -05:00
Sergei Golubchik
387b690eab cleanup: cosmetic fixes 2019-02-27 23:15:28 -05:00
Julius Goryavsky
50b3632fa4 MDEV-9519: Data corruption will happen on the Galera cluster size change
If we have a 2+ node cluster which is replicating from an async master
and the binlog_format is set to STATEMENT and multi-row inserts are executed
on a table with an auto_increment column such that values are automatically
generated by MySQL, then the server node generates wrong auto_increment
values, which are different from what was generated on the async master.

In the title of the MDEV-9519 it was proposed to ban start slave on a Galera
if master binlog_format = statement and wsrep_auto_increment_control = 1,
but the problem can be solved without such a restriction.

The causes and fixes:

1. We need to improve processing of changing the auto-increment values
after changing the cluster size.

2. If wsrep auto_increment_control switched on during operation of
the node, then we should immediately update the auto_increment_increment
and auto_increment_offset global variables, without waiting of the next
invocation of the wsrep_view_handler_cb() callback. In the current version
these variables retain its initial values if wsrep_auto_increment_control
is switched on during operation of the node, which leads to inconsistent
results on the different nodes in some scenarios.

3. If wsrep auto_increment_control switched off during operation of the node,
then we must return the original values of the auto_increment_increment and
auto_increment_offset global variables, as the user has set. To make this
possible, we need to add a "shadow copies" of these variables (which stores
the latest values set by the user).

https://jira.mariadb.org/browse/MDEV-9519
2019-02-26 08:09:04 +02:00