Commit graph

176364 commits

Author SHA1 Message Date
Marko Mäkelä
743ac7c2d0 MDEV-12061 Allow innodb_log_files_in_group=1
The InnoDB redo log consists of a list of files that logically form
a bigger file, as if the individual files were concatenated together.

The first file will always be written on redo log checkpoint, because
the two checkpoint pages are at the start of the single logical
redo log file.

There is no technical reason why InnoDB requires at least 2 files
to exist. Let us reduce the minimum number to 1. In that way,
restoring from backups will become easier, since InnoDB can directly
deal with a single backed-up redo log file.
2017-02-15 08:07:20 +02:00
Sergei Golubchik
3d85292afd sporadic failures of rpl.rpl_binlog_errors
the test restarts the server, giving it 60 seconds to shutdown
and then killing it mercilessly.

make sure the server closes all MyISAM tables before shutdown,
as we cannot reliably expect it to make the deadline.
2017-02-14 20:43:41 +01:00
Sergei Golubchik
96d097a7fa sporadic failures of main.default_debug
wait for the first connection to reach the debug sync point
before letting the second connection do its stuff
2017-02-14 20:43:41 +01:00
Sergei Golubchik
41f42dff06 cleanup: mtr startup warning
options --log-slow-admin-statements, --log-queries-not-using-indexes and --log-slow-slave-statements have no effect if --log_slow_queries is not set

1. s/--log_slow_queries/--log-slow-queries/
2. disable log-slow-admin-statement/etc in mytr when doing mysqld --help
2017-02-14 20:43:41 +01:00
Sergei Golubchik
6f6d0531dc MDEV-11439 No data type JSON, but CAST(something AS JSON) pretends to work
Use JSON_COMPACT(X) instead of CAST(X AS JSON).
Bonus - X is validated to be a valid JSON.
Fix a typo in the error message.
2017-02-14 20:43:41 +01:00
Sergei Golubchik
e0fa2ce40f bugfix: uninitialized variable
fixes crashes on win32-debug
2017-02-14 20:43:41 +01:00
Vladislav Vaintroub
8877f1c325 MDEV-12056 mysql_config outputs non-existing mysqlclient
Change output of mysql_config --libs to have -lmariadb instead of
-lmysqlclient

Static linking is not yet supported, because static library has different
base name ( libmariadbclient.a static vs libmariadb.so shared)
2017-02-14 18:07:42 +01:00
Alexey Botchkov
f76d5fefb8 MDEV-11439 No data type JSON, but CAST(something AS JSON) pretends to
work.

        json_detailed() fixed
2017-02-14 17:51:03 +04:00
Marko Mäkelä
2bf07556e8 Post-push fix for MDEV-12057.
Fix a test result for the narrower type of innodb_fast_shutdown.
2017-02-14 10:53:17 +02:00
Marko Mäkelä
1b4b4f6887 MDEV-12057 Embedded server shutdown hangs in InnoDB
Ever since MDEV-5800 enabled indexed virtual columns for InnoDB,
the InnoDB shutdown relied on close_connections() that would set
thd->killed for the InnoDB purge threads. Alas, the embedded server
shutdown is not invoking close_connections(), and thus InnoDB purge
threads fail to initiate shutdown, causing a hang.

innodb_inited: Remove. Use srv_was_started instead.

innobase_fast_shutdown: Remove. Use srv_fast_shutdown instead.

srv_running: Renamed from thd_destructor_myvar, and made global.
The value NULL means that shutdown was requested or the purge threads
should not be running because of innodb_read_only_mode=1.

innobase_init(): Set srv_was_started after ensuring that srv_running
was initialized. (In innodb_read_only mode, the purge threads are not
started and we do not care if srv_running==NULL.)

innobase_start_or_create_for_mysql(): Do not set srv_was_started.
Let it be set by the only caller innobase_init().

srv_purge_should_exit(): Check also srv_was_started and srv_running
when evaluating thd->killed.
2017-02-14 10:12:38 +02:00
Vicențiu Ciorbaru
d731ce21a7 MDEV-11170: MariaDB 10.2 cannot start on MySQL 5.7 datadir
PART 2 of the fix adds the logic of not using password column, unless it
exists. If password column is missing we attempt to use plugin &&
authentication_string columns.
2017-02-14 07:46:58 +02:00
Vicențiu Ciorbaru
5ab93737be MDEV-11170: MariaDB 10.2 cannot start on MySQL 5.7 datadir
PART 1 of the fix requires a bit of refactoring to not use hard-coded
field indices any more. Create classes that express the grant tables structure,
without exposing the underlying field indices.

Most of the code is converted to use these classes, except parts which
are not directly affected by the MDEV-11170. These however are TODO
items for subsequent refactoring.
2017-02-14 07:46:58 +02:00
Vicențiu Ciorbaru
dc90e24978 Define a helper class to allow for saving sql_mode using RAII
On construction the Sql_mode_save class stores the current THD's
sql_mode. On destruction, the THD's mode is restored.
2017-02-14 07:46:58 +02:00
Vicențiu Ciorbaru
bdca350f59 Allow READ_RECORD to make use of RAII and free itself on destruction 2017-02-14 07:46:58 +02:00
Vicențiu Ciorbaru
5bf338435a MDEV-11746: Wrong result upon using FIRST_VALUE with a window frame
The same approach is needed for LAST_VALUE, otherwise the LAST_VALUE sum
functions are not cleared correctly. Now LAST_VALUE behaves as NTH_VALUE
with 0 offset, only that the frame that it is examining is the bottom bound,
not the top bound.
2017-02-14 07:46:58 +02:00
Vicențiu Ciorbaru
57341852b5 MDEV-11746: Wrong result upon using FIRST_VALUE with a window frame
Reimplement FIRST_VALUE to act as NTH_VALUE with 0 offset. The previous
implementation was flawed when the window frame would remove values.
2017-02-14 07:46:58 +02:00
Vicențiu Ciorbaru
f675eab7dc MDEV-10122: MariaDB does not support group functions in some contexts where MySQL does
The problematic queries involve unions. For unions we have an
optimization where we skip the ORDER BY clause in a query from one side
of the union if it will be performed later due to UNION.
EX:
(SELECT a from t1 ORDER BY a) ORDER BY b;
The first ordering by a is not necessary and it gets removed.

The problem is that we still need to resolve the Items before removing the
ORDER BY list from the
SELECT_LEX structure. During this final resolve step however, we forgot to
allow SET functions within the ORDER BY clause. This caused us to return
an "Invalid use of group function" error during the checking performed
by fix_fields in Item_sum::init_sum_func_check.
2017-02-14 07:46:58 +02:00
Vicențiu Ciorbaru
fdfdea40f1 MDEV-11170: MariaDB 10.2 cannot start on MySQL 5.7 datadir:
Fatal error: mysql.user table is damaged or in unsupported 3.20 format

The problem stems from MySQL 5.7.6. According to MySQL documentation:
In MySQL 5.7.6, the Password column was removed and all credentials are
stored in the authentication_string column.

If opening a MySQL 5.7.6 (and up) datadir with MariaDB 10.2, the user table
appears corrupted. In order to fix this, the server must be started with
--skip-grant-tables and then a subsequent mysql_upgrade command must be
issued.

This patch updates the mysql_upgrade command to also add the removed
Password column. The password column is necessary, otherwise
the mysql_upgrade script fails due to the Event_scheduler not being able
to start, as it can't find Event_priv in the table where it ought to be.
MySQL's version has column position 28 (0 index) vs our datadir version
expects position 29.
2017-02-14 07:46:58 +02:00
Sergei Golubchik
e1f0f0dd06 Revert "MDEV-11439 No data type JSON, but CAST(something AS JSON) pretends to work"
This reverts commit 1f372cf1de.
Wasn't supposed to be pushed just yet.
2017-02-13 18:37:06 +01:00
Sergei Golubchik
1f372cf1de MDEV-11439 No data type JSON, but CAST(something AS JSON) pretends to work
remove CAST(... AS JSON) from the grammar for 10.2.4
2017-02-13 18:19:02 +01:00
Sergei Golubchik
87075e7f87 MDEV-11704 InnoDB: Failing assertion: dfield_is_null(dfield2) || dfield2->data
relax innodb assertion, because Field_blob::store() clearly says
that a data pointer can be zero if the length is zero.
2017-02-13 18:12:16 +01:00
Sergei Golubchik
239790d92f MDEV-11604 Assertion `!check_datetime_range(ltime)' failed in TIME_to_longlong_datetime_packed 2017-02-13 18:12:16 +01:00
Sergei Golubchik
1afb11074e MDEV-11582 InnoDB: Failing assertion: !((field)->vcol_info && !(field)->stored_in_db())
change the parser not to allow SERIAL as a normal data type.
make a special rule for it, where it could be used for define
fields, but not generated fields, not return type of a stored function, etc.
2017-02-13 18:12:15 +01:00
Sergei Golubchik
ca503e830b mtr: make sphinx skipping a bit less verbose 2017-02-13 18:12:15 +01:00
Sergei Golubchik
0df39e6037 bugfix: uninit variable 2017-02-13 18:12:15 +01:00
Sergei Golubchik
8d99166c69 MDEV-11640 gcol.gcol_select_myisam fails in buildbot on Power
JOIN_CACHE's were initialized in  check_join_cache_usage()
from make_join_readinfo(). After that make_join_readinfo() was looking
whether it's possible to use keyread. Later, after make_join_readinfo(),
optimizer decided whether to use filesort. And even later, at the
execution time, from join_read_first(), keyread was actually enabled.

The problem is, that if a query uses a vcol, base columns that it
depends on are automatically added to the read_set - because they're
needed to calculate the vcol. But if we're doing keyread, vcol is taken
from the index, not calculated, and base columns do not need to  be
in the read set (even should not be - as they aren't getting values).

The bug was that JOIN_CACHE used read_set with base columns,
they were not read because of keyread, so it was caching garbage.

So read_set is only known after the keyread was decided. And after the
filesort was decided, as filesort doesn't use keyread. But
check_join_cache_usage() needs to be done in make_join_readinfo(),
as the code below depends on these checks,

Fix: keep JOIN_CACHE checks where they were, but move initialization
down to the very end of JOIN::optimize_inner. If keyread was enabled,
update the read_set to include only columns that are part of the index.
Copy the keyread logic from join_read_first() to happen at optimize time.
2017-02-13 18:12:14 +01:00
Sergei Golubchik
0e5230e12d support keyread in READ_RECORD
make init_read_record() to detect enabled keyread
and use index_* access methods, not rnd_*

this makes MariaDB to use keyread a lot more often than before
2017-02-13 18:12:05 +01:00
Sergei Golubchik
1913daf42c bugfix: disable ICP in InnoDB for indexes on virtual columns
because it doesn't work, vcols are never calculated for ICP
2017-02-13 18:12:05 +01:00
Sergei Golubchik
01dd355635 cleanup: make a couple of tests more robust
with --sorted_result
2017-02-13 18:12:05 +01:00
Sergei Golubchik
cff144a8ea cleanup: handler::key_read
* rename to "keyread" (to avoid conflicts with tokudb),
* change from bool to uint and store the keyread index number there
* provide a bool accessor to check if keyread is enabled
2017-02-13 18:12:05 +01:00
Sergei Golubchik
dafb507e3e find_all_keys: add an assert, remove current_thd
Filesort temporarily changes read_set to be tmp_set and marks only
fields needed for filesort. Add an assert to ensure that it doesn't
overwrite the old value of tmp_set, that is that read_set was *not*
already tmp_set when filesort was invoked.

Fix sql_update.cc that was was doing exactly that - changing read_set to
tmp_set, configuring tmp_set for keyread, and then invoking filesort.
2017-02-13 18:12:05 +01:00
Sergei Golubchik
e46c42217f cleanup: TABLE::mark_columns_used_by_index()
mark_columns_used_by_index used to do
reset + mark_columns_used_by_index_no_reset + start keyread + set bitmaps

Now prepare_for_keyread does that, while mark_columns_used_by_index
does only reset + mark_columns_used_by_index_no_reset,
just as its name suggests.
2017-02-13 18:12:05 +01:00
Sergei Golubchik
460ff39871 bugfix: don't calculate vcols if doing keyread
old code didn't calculate vcols that were part of keyread,
but calculated other vcols. It was wrong - there was no guarantee
that vcol's base columns were part of keyread.

Technically it's possible for the vcol not be a part of keyread,
but all its base columns being part of keyread. But currently the
optimizer doesn't do that, keyread is only used if it covers
all columns used in the query.

This fixes crashes of vcol.vcol_trigger_sp_innodb
2017-02-13 18:12:05 +01:00
Sergei Golubchik
3cae225b0f cleanup: remove TABLE::add_read_columns_used_by_index
TABLE::add_read_columns_used_by_index() is conceptually wrong,
it *adds* columns used by index to the bitmap, without clearing
it first. But it also enables keyread, meaning that *only* columns
from the index will be read. It is supposed to be used to
add columns used by an index to a bitmap that already has columns
of a primary key - for engines where a primary key is part of every
index.

The correct fix is to change mark_columns_used_by_index() to
take into account extended keys.

this reverts 1d0acc7754 and cf97cbd1db
2017-02-13 18:12:05 +01:00
Sergei Golubchik
9fa6589f64 bugfix: TABLE::mark_columns_used_by_index_no_reset
it should not mark base columns that a vcol depends on, because
keyread (on a vcol) will not read them
2017-02-13 18:12:05 +01:00
Sergei Golubchik
4dd7e11332 cleanup: mark_columns_used_by_index_no_reset in handler::get_auto_increment
use table->mark_columns_used_by_index, don't copy it
2017-02-13 18:12:05 +01:00
Sergei Golubchik
bf8f70a47c cleanup: mark_columns_used_by_index_no_reset in opt_range.cc
use table->mark_columns_used_by_index, don't copy it
2017-02-13 18:12:05 +01:00
Sergei Golubchik
0254f1a6e0 bugfix: TABLE::mark_columns_used_by_index
Do *not* modify write_set.
keyread only affects what columns are *read*, UPDATE statement can
*write* into columns that aren't part of the keyread.
2017-02-13 18:12:04 +01:00
Sergei Golubchik
5d7607f340 cleanup: style fixes, sql_join_cache.cc 2017-02-13 18:12:04 +01:00
Sergei Golubchik
8246b0ac50 cleanup: TABLE::non_determinstic_insert
move the check where it make sense, remove incorrect comment
2017-02-13 18:12:04 +01:00
Sergei Golubchik
6c4144a468 InnoDB: suppress posix_fallocate() failure errors when EINVAL
EINVAL means that the filesystem doesn't support posix_fallocate().

There were two places where this error was issued, one checked for
EINVAL, the other did not. This commit fixed the other place
to also check for EINVAL.

Also, remove the space after the REFMAN to get the valid url
with no space in the middle.

Also don't say "Make sure the file system supports this function."
when posix_fallocate() fails, because this message is only shown
when the filesystem does support this function.
2017-02-13 18:12:04 +01:00
Sergei Golubchik
ef0db6bfd1 fix rpl_binlog_errors test not to leave dbug enabled 2017-02-13 18:12:04 +01:00
Sergei Golubchik
d2f84ab978 MDEV-10352 Server crashes in Field::set_default on CREATE TABLE
fix Item_default_value not to pretend being const_item
if the field's default_value expression isn't parsed yet
2017-02-13 18:12:04 +01:00
Sergei Golubchik
b6a3917b05 MDEV-11750 Assertion `vfield' failed in TABLE::update_virtual_fields after crash recovery on corrupted MyISAM table
Adjust the length of the BIT field
(same as in _mi_put_key_in_record())
2017-02-13 18:12:04 +01:00
Sergei Golubchik
29ed440d44 MDEV-11836 vcol.vcol_keys_myisam fails in buildbot and outside
move TABLE::key_read into handler. Because in index merge and DS-MRR
there can be many handlers per table, and some of them use
key read while others don't. "keyread" is really per handler,
not per TABLE property.
2017-02-13 18:12:04 +01:00
Sergei Golubchik
4cf4b61b24 oqgraph: remove redundant update_virtual_fields() calls 2017-02-13 18:12:04 +01:00
Sergei Golubchik
eda2ebefba MDEV-11784 View is created with invalid definition which causes ERROR 1241 (21000): Operand should contain 1 column(s)
set the correct print precedence for IN subqueries
2017-02-13 18:12:04 +01:00
Sergei Golubchik
cf00393378 Race condition in DEFAULT() with expressions
Item_default_value::calculate was updating table->s->default_values,
but it is supposed to be read-only
2017-02-13 18:12:04 +01:00
Sergei Golubchik
cd4dd2b62d MDEV-10201 Bad results for CREATE TABLE t1 (a INT DEFAULT b, b INT DEFAULT 4)
Optionally do table->update_default_fields() even for INSERT
that supposedly provides values for all column. Because these
"values" might be DEFAULT, which would need table->update_default_fields()
at the end.

Also set Item_default_value::used_tables() from the default expression.
Non-zero used_field() means that mysql_insert() will initialize all
fields to their default values (with restore_record()) even if
all columns are later provided with values. Because default expressions
may refer to other columns and they must be initialized.
2017-02-13 18:12:04 +01:00
Marko Mäkelä
588eca31e3 Post-fix for MDEV-12050 Remove unused InnoDB Memcached hooks
Remove also the field trx_t::read_write that was only used by
the Memcached hooks.
2017-02-13 11:40:19 +02:00