Commit graph

185083 commits

Author SHA1 Message Date
Marko Mäkelä
2bd7f32980 MDEV-16188: Remove redundant !this ||
Fix clang warning: 'this' pointer cannot be null in well-defined C++ code;
pointer may be assumed to always convert to true

The only caller of TABLE::best_range_rowid_filter_for_partial_join()
already seems to be assuming that s->table != NULL.
2019-02-18 11:44:20 +02:00
mkaruza
54ffc4996a MDEV-18588 Segfault during SST on joiner with bin-log, no bin-log-index
When node is JOINER and bin-log is enabled but bin-log-index is not set in configuration, we use NULL pointer which causes segfault. 
Fixed by checking for NULL pointer before using variable.
2019-02-18 11:05:40 +02:00
Oleksandr Byelkin
5b43dbb33e rempve relaxed chack made by galera 2019-02-18 09:51:08 +01:00
Oleksandr Byelkin
221d462cc7 fix for windows 2019-02-18 09:42:46 +01:00
Galina Shalygina
7a77b221f1 MDEV-7486: Condition pushdown from HAVING into WHERE
Condition can be pushed from the HAVING clause into the WHERE clause
if it depends only on the fields that are used in the GROUP BY list
or depends on the fields that are equal to grouping fields.
Aggregate functions can't be pushed down.

How the pushdown is performed on the example:

SELECT t1.a,MAX(t1.b)
FROM t1
GROUP BY t1.a
HAVING (t1.a>2) AND (MAX(c)>12);

=>

SELECT t1.a,MAX(t1.b)
FROM t1
WHERE (t1.a>2)
GROUP BY t1.a
HAVING (MAX(c)>12);

The implementation scheme:

1. Extract the most restrictive condition cond from the HAVING clause of
   the select that depends only on the fields that are used in the GROUP BY
   list of the select (directly or indirectly through equalities)
2. Save cond as a condition that can be pushed into the WHERE clause
   of the select
3. Remove cond from the HAVING clause if it is possible

The optimization is implemented in the function
st_select_lex::pushdown_from_having_into_where().

New test file having_cond_pushdown.test is created.
2019-02-17 23:38:44 -08:00
Vladislav Vaintroub
3220227794 MDEV-18578 build aws sdk for all plugins to use 2019-02-18 07:52:13 +01:00
Teemu Ollakka
3be9982f6a MDEV-18571 Fix wsrep PS protocol crash
Calls to wsrep_after_statement() were missing on PS protocol
codepath. Added calls after mysqld_stmt_execute() and
mysqld_stmt_bulk_execute().
2019-02-18 08:26:20 +02:00
Teemu Ollakka
b4c75f685b MDEV-18480 Backwards compatibility in log_view()
Galera versions below 4.x do not generate unique sequence number
for view events. Take this into account when writing the SE checkpoint
to avoid debug assertion in InnoDB.
2019-02-18 08:26:20 +02:00
Daniele Sciascia
9b8fc089bd MDEV-18198 Fix MTR test galera_sr.galera_sr_table_contents
* Created new binlog-header file
* Fixed warning on SELECT INTO DUMPFILE
* Re-recorded the test result
2019-02-18 08:25:57 +02:00
Aleksey Midenkov
a4cd91c526 MDEV-18603 Debug option to disable InnoDB monitor threads
If --debug=d,innodb_skip_monitors is specified, skip the creation
of srv_error_monitor_thread and srv_monitor_thread. In this way,
interactive debugging sessions will not be interfered by messages
about a mutex or rw_lock being held for a long time.

Closes #1192
2019-02-18 08:19:22 +02:00
Marko Mäkelä
790b6f5ae2 MDEV-18598: Wrong results after instant integer conversions
Field_str::is_equal(): Do not allow instant conversions between
BIT (which is stored big-endian) and integer types (which can
be stored big-endian or little-endian, depending on storage engine).

row_sel_field_store_in_mysql_format_func(): Properly extend
narrower integer and DATA_FIXBINARY values to the current format.
DATA_FIXBINARY was incorrectly padded with 0x20 instead of 0.
2019-02-17 01:05:31 +02:00
Marko Mäkelä
e9e47889c8 Merge 10.3 into 10.4 2019-02-16 12:34:41 +02:00
Marko Mäkelä
df51dc28f5 Fix tests for innodb_checksum_algorithm=strict_crc32
In tests that directly write InnoDB data file pages,
compute the innodb_checksum_algorithm=crc32 checksums,
instead of writing the 0xdeadbeef value used by
innodb_checksum_algorithm=none. In this way, these tests
will not cause failures when executing
./mtr --mysqld=--loose-innodb-checksum-algorithm=strict_crc32
2019-02-16 12:06:52 +02:00
Sergei Golubchik
28f18aa7a6 Merge branch '10.4' into bb-10.4-mdev16188 2019-02-15 20:31:58 +01:00
Igor Babaev
a44f2c3ee8 Merge branch '10.4' into bb-10.4-mdev16188 2019-02-15 07:48:18 -08:00
Alexander Barkov
62c0ac2da6 A cleanup for MDEV-13916 Enforce check constraint on JSON type
1. Renaming Type_handler_json to Type_handler_json_longtext
   There will be other JSON handlers soon, e.g. Type_handler_json_varchar.

2. Making the code more symmetric for data types:

- Adding a new virtual method
  Type_handler::Column_definition_validate_check_constraint()
- Moving JSON-specific code from sql_yacc.yy to
  Type_handler_json_longtext::Column_definition_validate_check_constraint()

3. Adding new files sql_type_json.cc and sql_type_json.h
   and moving Type_handler+JSON related code into these files.
2019-02-15 19:46:58 +04:00
Vladislav Vaintroub
568dd5293c MDEV-18592 Warning if temp-file (with "#sql-" prefix) is missing during prepare 2019-02-15 16:15:26 +01:00
Marko Mäkelä
e1af460146 MDEV-18579 Assertion !ctx->online || num_fts_index == 0
instant_alter_column_possible(): Add the other MDEV-17459 work-around
condition. The existence of fulltext indexes only prevents instant
DROP COLUMN or changing the order of columns. Other forms of instant
ALTER TABLE are no problem.

Before commit 4e7ee166a9 that merged
the MDEV-18295 fix from 10.3, the work-around of MDEV-17459 in
instant_alter_column_possible() was categorically refusing any
ALGORITHM=INSTANT if any FULLTEXT INDEX was present. After that commit,
a related condition was only present in prepare_inplace_alter_table_dict()
but not in the other callers of instant_alter_column_possible().
2019-02-15 13:56:04 +02:00
Marko Mäkelä
2b921845ac Merge 10.3 into 10.4 2019-02-15 12:30:43 +02:00
Jan Lindström
dcaabf07fd MDEV-18109: Galera 4: run galera_sr test suite
Fix some of the galera_sr suite test failures.
Remove tests that are not going to be run because required
feature is not supported.

	modified:   mysql-test/suite/galera_sr/disabled.def
	deleted:    mysql-test/suite/galera_sr/r/GCF-574.result
	modified:   mysql-test/suite/galera_sr/r/galera_sr_cc_slave.result
	modified:   mysql-test/suite/galera_sr/r/galera_sr_kill_all_norecovery.result
	modified:   mysql-test/suite/galera_sr/r/galera_sr_load_data.result
	deleted:    mysql-test/suite/galera_sr/r/galera_sr_sbr.result
	modified:   mysql-test/suite/galera_sr/r/mysql-wsrep-features#148.result
	deleted:    mysql-test/suite/galera_sr/r/mysql-wsrep-features#29.result
	deleted:    mysql-test/suite/galera_sr/t/GCF-574.test
	modified:   mysql-test/suite/galera_sr/t/galera_sr_cc_slave.test
	modified:   mysql-test/suite/galera_sr/t/galera_sr_kill_all_norecovery.cnf
	modified:   mysql-test/suite/galera_sr/t/galera_sr_kill_all_norecovery.test
	modified:   mysql-test/suite/galera_sr/t/galera_sr_load_data.test
	deleted:    mysql-test/suite/galera_sr/t/galera_sr_sbr.test
	modified:   mysql-test/suite/galera_sr/t/mysql-wsrep-features#148.test
	deleted:    mysql-test/suite/galera_sr/t/mysql-wsrep-features#29.test
2019-02-15 09:33:49 +02:00
Igor Babaev
294b8c426f Merge branch '10.4' into bb-10.4-mdev16188 2019-02-14 22:42:56 -08:00
Marko Mäkelä
e17fc72940 MDEV-15564: Fix bool/ibool type mismatch
dtype_is_string_type(), dtype_is_binary_string_type(),
dtype_is_non_binary_string_type(): Define as inline functions that
return bool, not ibool.
2019-02-15 08:15:46 +02:00
Marko Mäkelä
10c05d4ae9 MDEV-15564: Fix the non-debug build 2019-02-15 08:13:50 +02:00
Igor Babaev
98d55b1366 Merge branch '10.4' into bb-10.4-mdev16188 2019-02-14 22:07:33 -08:00
Igor Babaev
ccce4d3be9 MDEV-16188 Post review fixes
Also adjusted some test files.
2019-02-14 15:23:23 -08:00
Vladislav Vaintroub
282ba973e7 MDEV-18549 Failing assertion: opt_no_lock during mariabackup --backup
The assertion happens under BACKUP STAGE BLOCK_COMMIT, when a DDL on a
temporary table (#sql-xxx) is found.

Apparently, assumption that all DDLs are blocked under FTWRL does not
hold for BACKUP STAGE, and temporary tables can still have ALTERs

The fix is to relax the assertion, and only check for opt_no_lock if
backup is *really* inconsistent, i.e either optimized DDL or CREATE/RENAME
are done on the tables that were not skipped during backup.
2019-02-14 20:18:34 +01:00
Marko Mäkelä
ea0be9e2d6 MDEV-15564 Avoid table rebuild in ALTER TABLE on collation or charset changes
Allow ALGORITHM=INSTANT (or avoid touching any data)
when changing the collation, or in some cases, the character set,
of a non-indexed CHAR or VARCHAR column. There is no penalty
for subsequent DDL or DML operations, and compatibility with
older MariaDB versions will be unaffected.

Character sets may be changed when the old encoding is compatible
with the new one. For example, changing from ASCII to anything
ASCII-based, or from 3-byte to 4-byte UTF-8 can sometimes be
performed instantly.

This is joint work with Eugene Kosov.
The test cases as well as ALTER_CONVERT_TO, charsets_are_compatible(),
Type_handler::Charsets_are_compatible() are his work.
The Field_str::is_equal(), Field_varstring::is_equal() and
the InnoDB changes were mostly rewritten by me due to conflicts
with MDEV-15563.

Limitations:

Changes of indexed columns will still require
ALGORITHM=COPY. We should allow ALGORITHM=NOCOPY and allow
the indexes to be rebuilt inside the storage engine,
without copying the entire table.

Instant column size changes (in bytes) are not supported by
all storage engines.

Instant CHAR column changes are only allowed for InnoDB
ROW_FORMAT=REDUNDANT. We could allow this for InnoDB
when the CHAR internally uses a variable-length encoding,
say, when converting from 3-byte UTF-8 to 4-byte UTF-8.

Instant VARCHAR column changes are allowed for InnoDB
ROW_FORMAT=REDUNDANT, and for others only if the size
in bytes does not change from 128..255 bytes to more
than 256 bytes.

Inside InnoDB, this slightly changes the way how MDEV-15563
works and fixes the result of the innodb.instant_alter_extend test.
We change the way how ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT
is handled. All column extension, type changes and renaming
now go through a common route, except when ctx->is_instant()
is in effect, for example, instant ADD or DROP COLUMN has
been initiated. Only in that case we will go through
innobase_instant_try() and rewrite all column metadata.

get_type(field, prtype, mtype, len): Convert a SQL data type into
InnoDB column metadata.

innobase_rename_column_try(): Remove the update of SYS_COLUMNS.

innobase_rename_or_enlarge_column_try(): New function,
replacing part of innobase_rename_column_try() and all of
innobase_enlarge_column_try(). Also changes column types.

innobase_rename_or_enlarge_columns_cache(): Also change
the column type.
2019-02-14 20:17:14 +02:00
Sergei Golubchik
e5701d8363 cleanup: Account_options
move account options from LEX to Account_options structure
namely, mqh and ssl_*

Also, use LEX_CSTRING for ssl_*/x509_* strings and move
setting of ACL_USER::account_locked where it belongs
2019-02-14 17:30:22 +01:00
Robert Bindar
6c8ce999f8 MDEV-13095 Implement User Account locking
Add server support for user account locking.
This patch extends the ALTER/CREATE USER statements for
denying a user's subsequent login attempts:
  ALTER USER
    user [, user2] ACCOUNT [LOCK | UNLOCK]
  CREATE USER
    user [, user2] ACCOUNT [LOCK | UNLOCK]
The SHOW CREATE USER statement was updated to display the
locking state of an user.

Closes #1006
2019-02-14 17:30:22 +01:00
Sergei Golubchik
d89cdfc229 bugfix: mysql_fix_privilege_tables table_schema=database()
fix mysql_fix_privilege_tables to look for the `user` table in the
correct schema when deciding whether to convert it to `global_priv` table

make related tests a bit more verbose
2019-02-14 17:30:22 +01:00
Sergei Golubchik
84cbd69c9e cleanup: reformat 2019-02-14 17:30:22 +01:00
Sergei Golubchik
c0745e3730 bugfix: CREATE...SELECT lost COMMENT and VERSIONING 2019-02-14 17:30:22 +01:00
Sergei Golubchik
1d8b5524f4 cleanup: remove THD::query_start_timeval()
Too easy to abuse. One tends to use it even when
microseconds are not needed and will be ignored.
2019-02-14 17:30:22 +01:00
Jan Lindström
c568e25379
Merge pull request #1185 from codership/10.4-wsrep_schema_cleanup
Cleanup wsrep_schema and remove all references to wsrep_thd_pool
2019-02-14 14:30:13 +02:00
Jan Lindström
677a1e7c52
Merge pull request #1183 from codership/10.4-wsrep_debug
WSREP debug log levels support
2019-02-14 14:29:01 +02:00
Oleksandr Byelkin
6ffbfb92ed Try to fix windows compiler warnings 2019-02-14 12:53:49 +01:00
Vladislav Vaintroub
40b4f9c907 MDEV-18575 remove innodb-encrypt-log parameter from mariabackup
The variable is obsolete.
In mariabackup --backup, encryption plugin loading code sets the value
this parameter to the same as in server.

In mariabackup --prepare, no new redo  log is generated,
and xtrabackup_logfile is removed after it anyway.
2019-02-14 11:54:34 +01:00
Oleksandr Byelkin
56a8acd2e9 fix problem of last / 2019-02-14 11:06:15 +01:00
Daniele Sciascia
047754a728 Cleanup wsrep_schema and remove all references to wsrep_thd_pool
* Removed all references related to wsrep_thd_pool (which was removed)

* Removed unused declarations in wsrep_schema.h

* The following would result invalid reads in
  Wsrep_schema::replay_transaction():
  ```
  frag_table->field[4]->val_str(&buf);

  Wsrep_schema_impl::end_index_scan(frag_table);
  Wsrep_schema_impl::finish_stmt(thd);
  ret= wsrep_apply_events(thd, rli, buf.c_ptr_safe(), buf.length());
  ```

  because `buf` was accessed after closing the table. The fix is to
  perform storage reads using a different THD.

* In Wsrep_schema::recover_sr_transactions(), cluster_table was opened
  for write, however it is only read here. And frag_table was opened
  for read, wereas write is potentially needed.
  Also, avoid copy caused by String::c_ptr() to zero terminate the c
  string, use c_ptr_quick instead.
2019-02-14 09:55:14 +01:00
Igor Babaev
e1de23b8d5 MDEV-16188 Introduced the notion of adjusted filter gain.
Due to inconsistent usage of different cost models to calculate
the cost of ref accesses we have to make the calculation of the
gain promising by usage a range filter more complex.
2019-02-14 00:17:20 -08:00
Igor Babaev
7d9f45e072 Merge branch '10.4' into bb-10.4-mdev17096 2019-02-13 14:59:34 -08:00
Marko Mäkelä
a081a998a6 MDEV-15563: Fix cmake -DPLUGIN_PERFSCHEMA=NO
Commit 22feb179ae broke the build
with performance_schema disabled.

dict_col_t::same_charset(): An auxiliary function to
dict_col_t::same_format(). Determine if two non-binary string
columns have the same character set.
2019-02-13 22:11:40 +02:00
Alexey Botchkov
e8b6c15010 connect.xml.result fixed. 2019-02-13 23:26:23 +04:00
Daniel Black
ce0678f6cb hash (storage): hp_hashnr is local
Remove 'register' quantifer from args too.
2019-02-13 23:15:55 +04:00
Daniel Black
7fa67e391f heap: remove NEW_HASH_FUNCTION
This was added in 29fa72526a (in 2000) and nothing defines
NEW_HASH_FUNCTION. A low collision is the wrong tradeoff,
it should be focused on throughput.

d405bee058 shows the existing function
is pretty cpu intensive and needs to be improved as well.
2019-02-13 23:15:55 +04:00
mkaruza
3e64e7f24c WSREP debug log levels support
Global variable wsrep_debug now can be used to filter wsrep-lib messages based on debug level provided.
Type of wsrep_debug is now set to be unsigned int, so tests and configuration files changed accordingly.
2019-02-13 18:47:27 +01:00
Monty
0f48949439 MDEV-13916 Enforce check constraint on JSON type
When creating a field of type JSON, it will be automatically
converted to TEXT with CHECK (json_valid(`a`)), if there wasn't any
previous check for the column.

Additional things:
- Added two bug fixes that was found while testing JSON. These bug
  fixes has also been pushed to 10.3 (with a test case), but as they
  where minimal and needed to get this task done and tested, the fixes
  are repeated here.
  - CREATE TABLE ... SELECT drops constraints for columns that
    are both in the create and select part.
  - If one has both a default expression and check constraint for a
    column, one can get the error "Expression for field `a` is refering
    to uninitialized field `a`.
- Removed some duplicate MYSQL_PLUGIN_IMPORT symbols
2019-02-13 19:40:26 +02:00
Igor Babaev
62fad4e8e9 MDEV-17096 Pushdown of simple derived tables to storage engines
Fixing failures of federated test on 32-bit platforms
2019-02-13 08:55:38 -08:00
Monty
438811b4b2 Fixed two bugs related to column level constraints
- CREATE TABLE ... SELECT drops constraints for columns that
  are both in the create and select part.
  - Fixed by copying the constraint in
    Column_definiton::redefine_stage1_common()
- If one has both a default expression and check constraint for a
  column, one can get the error "Expression for field `a` is refering
  to uninitialized field `a`.
  - Fixed by ignoring default expressions for current column when checking
    for CHECK constraint
2019-02-13 18:21:19 +02:00
Marko Mäkelä
22feb179ae MDEV-15563: Instant ROW_FORMAT=REDUNDANT column extension
This was developed by Aleksey Midenkov based on my design.

In the original InnoDB storage format (that was retroactively named
ROW_FORMAT=REDUNDANT in MySQL 5.0.3), the length of each index field
is stored explicitly.

Because of this, we can and now will allow instant conversion from
VARCHAR to CHAR or VARBINARY to BINARY of equal or greater size,
as well as instant conversion of TINYINT to SMALLINT to MEDIUMINT
to INT to BIGINT (while not changing between signed and unsigned).

Theoretically, we could allow changing from an unsigned integer to
a bigger unsigned integer, as well as changing CHAR to VARCHAR, but
that would require additional metadata and conversions whenever
reading old records.

Field_str::is_equal(), Field_varstring::is_equal(), Field_num::is_equal():
Return the new result IS_EQUAL_PACK_LENGTH_EXT if the table advertises
HA_EXTENDED_TYPES_CONVERSION capability and we are considering the
above-mentioned conversions.

ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT: A new ALTER TABLE flag, similar
to ALTER_COLUMN_EQUAL_PACK_LENGTH but requiring conversions when
reading the data. The Field::is_equal() result IS_EQUAL_PACK_LENGTH_EXT
will map to this flag.

dtype_get_fixed_size_low(): For BINARY, CHAR and integer columns
in ROW_FORMAT=REDUNDANT, return 0 (variable length) from now on.

dtype_get_sql_null_size(): Keep returning the current size for
BINARY, CHAR and integer columns, so that in ROW_FORMAT=REDUNDANT
it will remain possible to update in place between NULL and NOT NULL
values.

btr_index_rec_validate(): Relax a CHECK TABLE length check for
ROW_FORMAT=REDUNDANT tables.

btr_cur_instant_init_low(): No longer trust fixed_len
for ROW_FORMAT=REDUNDANT tables.

We cannot rely on fixed_len anymore because the record can have shorter
length from before instant extension. Note that importing such tablespace
into earlier MariaDB versions produces ER_TABLE_SCHEMA_MISMATCH when
using a .cfg file.
2019-02-13 17:39:05 +02:00