Commit graph

184856 commits

Author SHA1 Message Date
Igor Babaev
62fad4e8e9 MDEV-17096 Pushdown of simple derived tables to storage engines
Fixing failures of federated test on 32-bit platforms
2019-02-13 08:55:38 -08:00
Monty
438811b4b2 Fixed two bugs related to column level constraints
- CREATE TABLE ... SELECT drops constraints for columns that
  are both in the create and select part.
  - Fixed by copying the constraint in
    Column_definiton::redefine_stage1_common()
- If one has both a default expression and check constraint for a
  column, one can get the error "Expression for field `a` is refering
  to uninitialized field `a`.
  - Fixed by ignoring default expressions for current column when checking
    for CHECK constraint
2019-02-13 18:21:19 +02:00
Marko Mäkelä
22feb179ae MDEV-15563: Instant ROW_FORMAT=REDUNDANT column extension
This was developed by Aleksey Midenkov based on my design.

In the original InnoDB storage format (that was retroactively named
ROW_FORMAT=REDUNDANT in MySQL 5.0.3), the length of each index field
is stored explicitly.

Because of this, we can and now will allow instant conversion from
VARCHAR to CHAR or VARBINARY to BINARY of equal or greater size,
as well as instant conversion of TINYINT to SMALLINT to MEDIUMINT
to INT to BIGINT (while not changing between signed and unsigned).

Theoretically, we could allow changing from an unsigned integer to
a bigger unsigned integer, as well as changing CHAR to VARCHAR, but
that would require additional metadata and conversions whenever
reading old records.

Field_str::is_equal(), Field_varstring::is_equal(), Field_num::is_equal():
Return the new result IS_EQUAL_PACK_LENGTH_EXT if the table advertises
HA_EXTENDED_TYPES_CONVERSION capability and we are considering the
above-mentioned conversions.

ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT: A new ALTER TABLE flag, similar
to ALTER_COLUMN_EQUAL_PACK_LENGTH but requiring conversions when
reading the data. The Field::is_equal() result IS_EQUAL_PACK_LENGTH_EXT
will map to this flag.

dtype_get_fixed_size_low(): For BINARY, CHAR and integer columns
in ROW_FORMAT=REDUNDANT, return 0 (variable length) from now on.

dtype_get_sql_null_size(): Keep returning the current size for
BINARY, CHAR and integer columns, so that in ROW_FORMAT=REDUNDANT
it will remain possible to update in place between NULL and NOT NULL
values.

btr_index_rec_validate(): Relax a CHECK TABLE length check for
ROW_FORMAT=REDUNDANT tables.

btr_cur_instant_init_low(): No longer trust fixed_len
for ROW_FORMAT=REDUNDANT tables.

We cannot rely on fixed_len anymore because the record can have shorter
length from before instant extension. Note that importing such tablespace
into earlier MariaDB versions produces ER_TABLE_SCHEMA_MISMATCH when
using a .cfg file.
2019-02-13 17:39:05 +02:00
Marko Mäkelä
0ae3ea7919 MDEV-15563: Instant VARCHAR extension for ROW_FORMAT=REDUNDANT
In the original InnoDB storage format (which was retroactively named
ROW_FORMAT=REDUNDANT in MySQL 5.0.3), the length of each index field
is stored explicitly. Thus, we can and from now on will allow arbitrary
extension of VARBINARY and VARCHAR columns when the table is in
ROW_FORMAT=REDUNDANT.

ha_innobase::open(): Advertise a new HA_EXTENDED_TYPES_CONVERSION
capability for ROW_FORMAT=REDUNDANT tables.

Field_varstring::is_equal(): If the HA_EXTENDED_TYPES_CONVERSION
capability is advertised for the table, return IS_EQUAL_PACK_LENGTH
for any length extension.
2019-02-13 16:42:03 +02:00
Daniel Black
44898d28f0 my_close: Don't retry on close
According to close(2) "Retrying the close() after a failure return is
the wrong thing to do"

Even the EINTR case its maybe closed. Take the prudent approach here
an risk leaking one file descriptor rather than closing one that is
nolonger ours.
2019-02-13 18:28:12 +04:00
Marko Mäkelä
ad17875c0d MDEV-15563: Allow instant VARCHAR extension from <128 bytes
For up to 127 bytes length, InnoDB would use 1 byte for length, and
that byte would always be less than 128. If the maximum length is
longer than 255 bytes, InnoDB would use a variable-length encoding
for the length, using 1 byte for lengths up to 127 bytes, and
2 bytes for longer lengths.

Thus, 1-byte lengths are always compatible when the maximum size
changes from less than 128 bytes to anything longer.

Field_varstring::is_equal(): Return IS_EQUAL_PACK_LENGTH also when
converting from VARCHAR less than 128 bytes to any longer VARCHAR.
2019-02-13 15:46:52 +02:00
Marko Mäkelä
8ef4105a89 MDEV-6111: Adjust a result 2019-02-13 15:46:04 +02:00
Sergey Vojtovich
8a3a332bc0 Cleanup Item_func_sp::fix_fields()
No need to call list.empty(): first one is called by List constructor,
second one doesn't make sense as the object is destroyed immediately
afterwards.
2019-02-13 17:40:03 +04:00
Daniel Black
a3ccad0f21 Aggregate functions: only create list if args exist
Also always empty list rather than only if there is no error.
2019-02-13 17:35:51 +04:00
Jan Lindström
c2c637c583
Merge pull request from codership/10.4-load-data-splitting
10.4 make wsrep_load_data_splitting use streaming replication
2019-02-13 13:29:51 +02:00
Jan Lindström
6476126cba MDEV-18564: Change wsrep_load_data_splitting off by default
Variable wsrep_load_data_splitting is deprecated and should be off
by default.
2019-02-13 13:19:37 +02:00
Vladislav Vaintroub
d28dab7658 Fix compilation on old gcc 2019-02-13 12:15:14 +01:00
Jan Lindström
91451739f2
Merge pull request from grooverdan/10.4-friendlier-wsrep-message
cmake-wsrep: friendly error message about missing wsrep_api.h
2019-02-13 09:27:43 +02:00
Igor Babaev
953ca199fb Merge branch '10.4' into bb-10.4-mdev17096 2019-02-12 23:19:43 -08:00
Igor Babaev
27c3abde30 MDEV-17096 Pushdown of simple derived tables to storage engines
MDEV-17631 select_handler for a full query pushdown

Added comments and file headers for files introduced in these tasks.
2019-02-12 22:56:24 -08:00
Varun Gupta
be8709eb7b MDEV-6111 Optimizer Trace
This task involves the implementation for the optimizer trace.

This feature produces a trace for any SELECT/UPDATE/DELETE/,
which contains information about decisions taken by the optimizer during
the optimization phase (choice of table access method, various costs,
transformations, etc). This feature would help to tell why some decisions were
taken by the optimizer and why some were rejected.

Trace is session-local, controlled by the @@optimizer_trace variable.
To enable optimizer trace we need to write:
   set @@optimizer_trace variable= 'enabled=on';

To display the trace one can run:
   SELECT trace FROM INFORMATION_SCHEMA.OPTIMIZER_TRACE;

This task also involves:
    MDEV-18489: Limit the memory used by the optimizer trace
    introduces a switch optimizer_trace_max_mem_size which limits
    the memory used by the optimizer trace. This was implemented by
    Sergei Petrunia.
2019-02-13 11:52:36 +05:30
Varun Gupta
6b979416e0 Extending the API for json_writer by introdcing
classes for Json_writer_object and Json_writer_array.
These classes will be used for the implementation
of the optimizer trace.
2019-02-13 11:52:35 +05:30
Daniel Black
4f5c65367a cmake-wsrep: friendly error message about missing wsrep_api.h
Without this commit the error message is:

$ cmake . -DWITH_WSREP=ON
-- Running cmake version 3.12.1
-- MariaDB 10.4.3
CMake Error at cmake/wsrep.cmake:36 (FILE):
  FILE STRINGS file
  ".../mariadb-server-10.4/wsrep-lib/wsrep-API/v26/wsrep_api.h"
  cannot be read.
Call Stack (most recent call first):
  CMakeLists.txt:147 (INCLUDE)

Having 'git config cmake.update-submodules yes' nicely updates the
build without error.

$ cmake . -DWITH_WSREP=ON
-- Running cmake version 3.12.1
-- MariaDB 10.4.3
Submodule path 'wsrep-lib': checked out 'e7d72ae7f6a6995a21d743389426a963429a1fff'
Submodule path 'wsrep-lib/wsrep-API/v26': checked out '75a5f452f2ba07b0f4a3a9a94825fccc71b27398'
2019-02-13 11:36:03 +11:00
Igor Babaev
17d00d9a94 MDEV-17096 Pushdown of simple derived tables to storage engines
Made the setting of the system variable federated_pushdown at
the default connection.
2019-02-12 14:00:48 -08:00
Igor Babaev
58b950737c MDEV-17096 Pushdown of simple derived tables to storage engines
Added plugin system variable federated_pushdown.
2019-02-12 13:11:32 -08:00
Sergey Vojtovich
4d5f85a3ec Bootstrap cleanups
Remove unused key_thread_bootstrap.
No more global bootstrap_error: it is return value of bootstrap() instead.
2019-02-12 23:07:51 +04:00
Sergei Golubchik
ffa4677c09 MDEV-12834 mysql_secure_installation should ask about unix_socket authentication
Optionally, restore the default root authentication, that is
mysql_native_password USING 'invalid' OR unix_socket
2019-02-12 19:14:37 +01:00
Sergei Golubchik
4e4df7b46d MDEV-12834 mysql_secure_installation should ask about unix_socket authentication
recognize new unix_socket protected accounts, and don't
scare the user into setting the password for them
2019-02-12 19:14:37 +01:00
Sergei Golubchik
b9f3f06857 MDEV-12484 Enable unix socket authentication by default
Change the default authentication for root@localhost to

  IDENTIFIED VIA mysql_native_password USING 'invalid' OR unix_socket

which provides secure passwordless login, while still allowing
SET PASSWORD to work as expected.

Also create a second all-privilege account for the user that owns
datadir (and thus has full access to the data anyway).

Compile unix_socket plugin statically into the server.
2019-02-12 19:14:37 +01:00
Sergei Golubchik
f07b76fcfd cleanup: remove reduntant variable 2019-02-12 19:14:37 +01:00
Jan Lindström
7dfbb66fcb
Merge pull request from grooverdan/10.4-disable-wsrep-allo
cmake/wsrep: allow disabling -DWITH_WSREP=OFF
2019-02-12 15:29:12 +02:00
Teemu Ollakka
f20dfeecc6 Recorded galera_sr_load_data_splitting, galera_sr_load_data 2019-02-12 10:37:05 +02:00
Teemu Ollakka
f06a0b5338 Implement wsrep_load_data_splitting with streaming replication
If wsrep_load_data_splitting is configured, change streaming replication
parameters internally to match the original behavior, i.e. replicate
on every 10000 rows. After load data is over, restore original
streaming replication settings.

Removed redundant wsrep_tc_log_commit().
2019-02-12 10:37:05 +02:00
Alexander Barkov
ce6505f890 MDEV-18447 Assertion `!is_zero_datetime()' failed in Timestamp_or_zero_datetime::tv 2019-02-12 11:59:17 +04:00
Daniel Black
b2dd88f095 cmake/wsrep: allow disabling -DWITH_WSREP=OFF 2019-02-12 15:59:05 +11:00
Marko Mäkelä
9f56dd7382 Merge 10.3 into 10.4 2019-02-11 17:55:25 +02:00
Jan Lindström
3a269a8b52 Record galera_var_load_data_splitting result after deprecating
warning.
2019-02-11 14:59:59 +02:00
Marko Mäkelä
4e7ee166a9 MDEV-18295 IMPORT TABLESPACE fails with instant-altered tables
When importing a tablespace, we must initialize dummy DEFAULT NULL
values for any instantly added columns in order to avoid a debug
assertion failure when PageConverter::update_records() invokes
rec_get_offsets(). Finally, when the operation completes, we must
evict and reload the table definition, so that the correct
default values for instantly added columns will be loaded.

ha_innobase::discard_or_import_tablespace(): On successful
IMPORT TABLESPACE, evict and reload the table definition,
so that btr_cur_instant_init() will load the correct metadata.

PageConverter::update_index_page(): Fill in dummy DEFAULT NULL values
for instantly added columns. These will be replaced upon the
completion of the operation by evicting and reloading the metadata.

row_discard_tablespace(): Invoke dict_table_t::remove_instant().
After DISCARD TABLESPACE, the table is no longer in "instant ALTER"
format, because there is no data file attached.
2019-02-11 14:42:48 +02:00
Jan Lindström
cbfbb70dd2
Merge pull request from codership/10.4-rsync_fix
MDEV-18178, mariabackup script timestamp change, galera suite fixed & re-recorded tests
2019-02-11 13:46:26 +02:00
Jan Lindström
c4f3998365
Merge pull request from codership/10.4-fix-wsrep-toi-end
Fixed use of uninitialized value in wsrep_TOI_end()
2019-02-11 10:43:55 +02:00
Teemu Ollakka
f0513de525 Fixed use of uninitialized value in wsrep_TOI_end()
The if/else was a remnant from error voting code which was not
included in the release.
2019-02-11 10:10:55 +02:00
Marko Mäkelä
76c34a74a8 Fix the 32-bit results for max_rowid_filter_size 2019-02-11 09:24:31 +02:00
Igor Babaev
cd00d03fe2 MDEV-16188 Fixed the code of ha_partition::multi_range_read_info()
The code was rewritten in the same way as the code of
ha_partition::multi_range_read_info_const() had been rewritten
earlier.

The fix allowed to run spider.partition_mrr.
2019-02-10 21:15:48 -08:00
Galina Shalygina
3955d2a153 MDEV-18413: Find constraint correlated indexes
Find indexes of one table which parts participate in one constraint.
These indexes are called constraint correlated.

New methods: TABLE::find_constraint_correlated_indexes() and
virtual method check_index_dependence() were added.
For each index it's own constraint correlated index map was created
where all indexes that are constraint correlated with the current are
marked.

The results of this task are used for MDEV-16188 (Use in-memory
PK filters built from range index scans).
2019-02-10 22:36:46 +03:00
Igor Babaev
d11be23933 MDEV-17096 Pushdown of simple derived tables to storage engines
Resolved the problem of forming a proper query string for FEDERATEDX.
Added test cases.

Cleanup of extra spaces.
2019-02-09 22:54:26 -08:00
Igor Babaev
15fe81c571 MDEV-16188 Post merge fixes: trying to get rid of Windows warnings. 2019-02-09 10:54:26 -08:00
Igor Babaev
9fe1e83df0 MDEV-16188 Post merge fixes: more for TokuDB 2019-02-08 12:32:31 -08:00
Marko Mäkelä
3c305d3f19 MDEV-15528: Introduce MLOG_INIT_FREE_PAGE
This is only a placeholder that allows an implementation later
during the development of MariaDB, so that downgrade to an earlier
version (with this code) will be possible.
We want to be able to zero out freed pages to reduce write amplification,
and to scrub old data. Zeroing out the pages is optional, not mandatory
for correctness. After all, the MLOG_INIT_FREE_PAGE record can only be
emitted for pages that are marked free in the allocation bitmap page.
2019-02-08 19:16:32 +02:00
Marko Mäkelä
ea25bdc135 Do not write MLOG_IBUF_BITMAP_INIT
Use ibuf_bitmap_page_init() only during recovery.

fsp_fill_free_list(): Initialize the FIL_PAGE_TYPE using MLOG_2BYTES.
The page contents will already have been zeroed out by
MLOG_INIT_FILE_PAGE2.

ibuf_bitmap_init_apply(): Replaces ibuf_parse_bitmap_init().
2019-02-08 19:10:33 +02:00
Marko Mäkelä
b8e867e869 MDEV-18219 Assertion index->n_core_null_bytes <= ... after DROP COLUMN
rec_init_offsets(): Relax a debug assertion that is too strict
during the execution of btr_cur_instant_init_low().
2019-02-08 18:46:03 +02:00
Teemu Ollakka
58de2bf30d MDEV-18481 Use mlog_memset() to clear wsrep checkpoint 2019-02-08 14:17:23 +00:00
mkaruza
131a4680c7 Fixed and re-recorded tests for galera
Re-recorded: galera.galera_gcache_recover_manytrx
Fixed: galera_3nodes.galera_evs_suspect_timeout
2019-02-08 14:26:56 +01:00
mkaruza
accc7f6029 MDEV-18178 Galera test failure on galera_sst_rsync2
* Donor node will now provide binlog-index argument to wsrep_sst_rsync script if binlog is used.
* Write correct path and binlog file names into joiner binlog-index file
2019-02-08 11:18:18 +01:00
mkaruza
26dcf102e3 Extend mariabackup archive timestamp in SST script
During parallel execution mariabackup script can fail when trying to create archive because of non unique name.  Extending archive timestamp with nanosecond.
2019-02-08 11:15:15 +01:00
Igor Babaev
651347b6c1 MDEV-16188 Post merge fixes fot TokuDB 2019-02-08 01:07:27 -08:00