Commit graph

21082 commits

Author SHA1 Message Date
Marko Mäkelä
dde2ca4aa1 Merge 10.3 into 10.4 2018-11-19 20:22:33 +02:00
Marko Mäkelä
fd58bb71e2 Merge 10.2 into 10.3 2018-11-19 18:45:53 +02:00
Marko Mäkelä
e669e705a1 Fix the Windows build
Commit ff88e4bb8a broke the Windows build.
Clean up one more redundant #include to fix it.
2018-11-19 13:13:39 +02:00
Marko Mäkelä
ff88e4bb8a Remove many redundant #include from InnoDB 2018-11-19 11:42:14 +02:00
Marko Mäkelä
cb5bca721b MDEV-17765 lock_discard_page() may fail to discard locks for SPATIAL INDEX
lock_discard_page(): Traverse and discard the B-tree record locks
only if they exist. Else, discard the R-tree (spatial) index locks.
2018-11-19 11:40:10 +02:00
Marko Mäkelä
f037b91098 MDEV-17726: Fix compiler warning
Compare against the correct constant 0. NOT_KILLED belongs to
enum killed_state, while THD_IS_NOT_KILLED is what we want.
2018-11-19 11:11:53 +02:00
Marko Mäkelä
7debbd7859 Fix a compilation error
ib_counter_t::ib_counter_element_t: Avoid sizeof on a std::atomic type,
because it causes errors on some version of the Microsoft compiler.
2018-11-18 15:50:43 +02:00
Marko Mäkelä
075820ab23 MDEV-17750: Fix the Windows build
Remove some redundant declarations of global variables. These declarations
can refer to undefined type names, because they occur before any #include
statement for the type definitions. On GNU/Linux, uint and ulong seem to
be defined in <sys/types.h> due to _DEFAULT_SOURCE and __USE_MISC
becoming defined in <features.h>.
2018-11-18 15:46:53 +02:00
Marko Mäkelä
f92d223fe2 MDEV-17750: Remove dict_index_get_sys_col_pos()
dict_index_t::db_trx_id(): Return the position of DB_TRX_ID.
Only valid for the clustered index.

dict_index_t::db_roll_ptr(): Return the position of DB_ROLL_PTR.
Only valid for the clustered index.

dict_index_get_sys_col_pos(): Remove. This was performing unnecessarily
complex computations, which only made sense for DB_ROW_ID, which would
exist either as the first field in the clustered index or as the last
field in a secondary index (only when a DB_ROW_ID column is materialised).

row_sel_store_row_id_to_prebuilt(): Remove, and replace with simpler code.

row_upd_index_entry_sys_field(): Remove.

btr_cur_log_sys(): Replaces row_upd_write_sys_vals_to_log().

btr_cur_write_sys(): Write DB_TRX_ID,DB_ROLL_PTR to a data tuple.
2018-11-16 20:37:06 +02:00
Marko Mäkelä
eea0c3c3e7 MDEV-17750: Remove unnecessary rec_get_offsets() in IMPORT TABLESPACE
row_import_set_sys_max_row_id(): Change the return type to void,
and access the first column (DB_ROW_ID) directly.
2018-11-16 20:37:00 +02:00
Marko Mäkelä
3773bc594d MDEV-17750: Avoid some rec_get_offsets() for accessing FTS_DOC_ID
fts_get_max_doc_id(): Remove totally unnecessary call to rec_get_offsets().
In FTS_DOC_ID_INDEX, the FTS_DOC_ID is always stored in the first 8 bytes
of the record.

fts_get_doc_id_from_row(): Get the offsets from the caller.

row_ins_foreign_check_on_constraint(): Invoke rec_get_offsets()
for the call.
2018-11-16 20:36:47 +02:00
Marko Mäkelä
0cbf578ac8 Add dict_index_t member functions for determining index type 2018-11-16 20:36:47 +02:00
Monty
37d6d3b661 Max transid was not stored directly after Aria recovery
This caused ma_test_recovery.pl to fail

Other things:
- Fixed bug where "ma_test_recovert.pl --abort-on-error" didn't abort
  on error
2018-11-16 16:54:33 +02:00
Marko Mäkelä
16d43150ae MDEV-17721 Corrupted data dictionary after instant ADD COLUMN
dict_index_t::reconstruct_fields(): Correctly permute the fields
of the columns. The code was totally wrong in MDEV-15562.
It would only work when columns are added last or dropped,
but not when columns are permuted.
2018-11-16 16:35:56 +02:00
Marko Mäkelä
f85012246c MDEV-17735 Assertion failure in row_parse_int() on first ADD/DROP COLUMN
row_ins_clust_index_entry_low(): Do not attempt to read an AUTO_INCREMENT
column value from a metadata record, because it does not make any sense.
Moreover, the field offset would be off by one in case the AUTO_INCREMENT
column is not part of the PRIMARY KEY, because the MDEV-15562 metadata
record would contain an extra field at index->first_user_field().

On MariaDB Server 10.3 after MDEV-11369, we would unnecessarily read
a dummy AUTO_INCREMENT value from the metadata record, but that value
would always be written as NULL or 0, so there is no problem.
2018-11-16 13:28:37 +02:00
Eugene Kosov
89337d510e MDEV-16580 Remove unused monitor counters from InnoDB
Remove one totally dead monitor.
2018-11-16 11:52:44 +03:00
Marko Mäkelä
ab812c1089 MDEV-17726: A better fix
THD::close_temporary_tables(): Revert the change.

ha_innobase::delete_table(): Move the work-around inside
a debug assertion, and check thd_kill_level() instead of thd_killed(),
because the latter would not hold for KILL_CONNECTION.
2018-11-16 10:42:45 +02:00
Alexander Barkov
b9a9055793 MDEV-17712 Remove C_TIME_FUZZY_DATES, C_TIME_DATETIME_ONLY, C_TIME_TIME_ONLY 2018-11-14 16:00:38 +04:00
Marko Mäkelä
f718477714 os_aio_validate_skip(): Fix a data race 2018-11-14 09:41:37 +02:00
Marko Mäkelä
59b87e75d0 Fix a comment 2018-11-12 18:06:41 +02:00
Marko Mäkelä
fde5386d16 Add dict_table_t::not_redundant() 2018-11-12 17:01:56 +02:00
Marko Mäkelä
4b773ca298 MDEV-13564 clean-up: Replace memset() with init() 2018-11-12 10:16:28 +02:00
Marko Mäkelä
3ea7de9a78 MDEV-15562 post-push cleanup
dict_instant_t::non_pk_col_map: Change the type to uint16_t
The persistent storage for this metadata is only 16 bits.

dict_table_t::serialise_columns(): Define inline.
2018-11-09 18:00:07 +02:00
Oleksandr Byelkin
f5bcf788e7 Merge branch '10.2-connect' into 10.2 2018-11-09 10:32:10 +01:00
Marko Mäkelä
c761b43451 Merge 10.3 into 10.4 2018-11-08 10:19:55 +02:00
Alexander Barkov
07e4853c23 MDEV-17563 Different results using table or view when comparing values of time type
MDEV-17625 Different warnings when comparing a garbage to DATETIME vs TIME

- Splitting processes of data type conversion (to TIME/DATE,DATETIME)
  and warning generation.
  Warning are now only get collected during conversion (in an "int" variable),
  and are pushed in the very end of conversion (not in parallel).
  Warnings generated by the low level routines str_to_xxx() and number_to_xxx()
  can now be changed at the end, when TIME_FUZZY_DATES is applied,
  from "Invalid value" to "Truncated invalid value".

  Now "Illegal value" is issued only when the low level routine returned
  an error and TIME_FUZZY_DATES was not set. Otherwise, if the low level
  routine returned "false" (success), or if NULL was converted to a zero
  datetime by TIME_FUZZY_DATES, then "Truncated illegal value"
  is issued. This gives better warnings.

- Methods Type_handler::Item_get_date() and
  Type_handler::Item_func_hybrid_field_type_get_date() now only
  convert and collect warning information, but do not push warnings.

- Changing the return data type for Type_handler::Item_get_date()
  and Type_handler::Item_func_hybrid_field_type_get_date() from
  "bool" to "void". The conversion result (success vs error) can be
  checked by testing ltime->time_type. MYSQL_TIME_{NONE|ERROR}
  mean mean error, other values mean success.

- Adding new wrapper methods Type_handler::Item_get_date_with_warn() and
  Type_handler::Item_func_hybrid_field_type_get_date_with_warn()
  to do conversion followed by raising warnings, and changing
  the code to call new Type_handler::***_with_warn() methods.

- Adding a helper class Temporal::Status, a wrapper
  for MYSQL_TIME_STATUS with automatic initialization.

- Adding a helper class Temporal::Warn, to collect warnings
  but without actually raising them. Moving a part of ErrConv
  into a separate class ErrBuff, and deriving both Temporal::Warn
  and ErrConv from ErrBuff. The ErrBuff part of Temporal::Warn
  is used to collect textual representation of the input data.

- Adding a helper class Temporal::Warn_push. It's used
  to collect warning information during conversion, and
  automatically pushes warnings to the diagnostics area
  on its destructor time (in case of non-zero warning).

- Moving more code from various functions inside class Temporal.

- Adding more Temporal_hybrid constructors and
  protected Temporal methods make_from_xxx(),
  which convert and only collect warning information, but do not
  actually raise warnings.

- Now the low level functions  str_to_datetime() and str_to_time()
  always set status->warning if the return value is "true" (error).

- Now the low level functions number_to_time() and number_to_datetime()
  set the "*was_cut" argument if the return value is "true" (error).

- Adding a few DBUG_ASSERTs to make sure that str_to_xxx() and
  number_to_xxx() always set warnings on error.

- Adding new warning flags MYSQL_TIME_WARN_EDOM and MYSQL_TIME_WARN_ZERO_DATE
  for the code symmetry. Before this change there was a special
  code path for (rc==true && was_cut==0) which was treated by
  Field_temporal::store_invalid_with_warning as "zero date violation".
  Now was_cut==0 always means that there are no any error/warnings/notes
  to be raised, not matter what rc is.

- Using new Temporal_hybrid constructors in combination with
  Temporal::Warn_push inside str_to_datetime_with_warn(),
  double_to_datetime_with_warn(), int_to_datetime_with_warn(),
  Field::get_date(), Item::get_date_from_string(), and a few other places.

- Removing methods Dec_ptr::to_datetime_with_warn(),
  Year::to_time_with_warn(), my_decimal::to_datetime_with_warn(),
  Dec_ptr::to_datetime_with_warn().
  Fixing Sec6::to_time() and Sec6::to_datetime() to
  convert and only collect warnings, without raising warnings.
  Now warning raising functionality resides in Temporal::Warn_push.

- Adding classes Longlong_hybrid_null and Double_null, to
  return both value and the "IS NULL" flag. Adding methods
  Item::to_double_null(), to_longlong_hybrid_null(),
  Item_func_hybrid_field_type::to_longlong_hybrid_null_op(),
  Item_func_hybrid_field_type::to_double_null_op().
  Removing separate classes VInt and VInt_op, as they
  have been replaced by a single class Longlong_hybrid_null.

- Adding a helper method Temporal::type_name_by_timestamp_type(),
  moving a part of make_truncated_value_warning() into it,
  and reusing in Temporal::Warn::push_conversion_warnings().

- Removing Item::make_zero_date() and
  Item_func_hybrid_field_type::make_zero_mysql_time().
  They provided duplicate functionality.
  Now this code resides in Temporal::make_fuzzy_date().
  The latter is now called for all Item types when data type
  conversion (to DATE/TIME/DATETIME) is involved, including
  Item_field and Item_direct_view_ref.
  This fixes MDEV-17563: Item_direct_view_ref now correctly converts
  NULL to a zero date when TIME_FUZZY_DATES says so.
2018-11-08 09:31:46 +04:00
Marko Mäkelä
862af4d255 Merge 10.2 into 10.3 2018-11-07 13:11:04 +02:00
Marko Mäkelä
89f948c766 Merge 10.1 into 10.2 2018-11-07 08:17:47 +02:00
Marko Mäkelä
59c82dde09 Merge 10.0 into 10.1 2018-11-07 08:08:45 +02:00
Marko Mäkelä
074c684099 Merge 10.3 into 10.4 2018-11-06 16:24:16 +02:00
Sergei Petrunia
7dfcb87107 Disable rocksdb.com_rpc_tx test
It tests SQL-level feature that is available in FB/MySQL and not in MariaDB
2018-11-06 17:23:39 +03:00
Anel Husakovic
9c026273a9 Add implementation in .h and delete unneccessery printing 2018-11-06 12:07:26 +00:00
Jan Lindström
ef40018535 MDEV-17230: encryption_key_id from alter is ignored by encryption threads
Background: Used encryption key_id is stored to encryption metadata
i.e. crypt_data that is stored on page 0 of the tablespace of the
table. crypt_data is created only if implicit encryption/not encryption
is requested i.e. ENCRYPTED=[YES|NO] table option is used
fil_create_new_single_table_tablespace on fil0fil.cc.

Later if encryption is enabled all tables that use default encryption
mode (i.e. no encryption table option is set) are encrypted with
default encryption key_id that is 1. See fil_crypt_start_encrypting_space on
fil0crypt.cc.

ha_innobase::check_table_options()
	If default encryption is used and encryption is disabled, you may
        not use nondefault encryption_key_id as it is not stored anywhere.
2018-11-06 10:22:25 +02:00
Marko Mäkelä
df563e0c03 Merge 10.2 into 10.3
main.derived_cond_pushdown: Move all 10.3 tests to the end,
trim trailing white space, and add an "End of 10.3 tests" marker.
Add --sorted_result to tests where the ordering is not deterministic.

main.win_percentile: Add --sorted_result to tests where the
ordering is no longer deterministic.
2018-11-06 09:40:39 +02:00
Marko Mäkelä
32062cc61c Merge 10.1 into 10.2 2018-11-06 08:41:48 +02:00
Marko Mäkelä
bae21bfb5d Merge 10.0 into 10.1 2018-11-05 17:50:41 +02:00
Marko Mäkelä
db55b39fb2 Revert some InnoDB/XtraDB changes
The relevant InnoDB/XtraDB fixes up to 5.6.42 had already
been applied to MariaDB in commit 30c3d6db32.

Revert some changes that appeared in
the merge commit 87d852f102.
2018-11-05 16:47:14 +02:00
Marko Mäkelä
d63e198061 Merge 10.0 into 10.1 2018-11-05 12:15:17 +02:00
Eugene Kosov
03977e8273 MDEV-13671 InnoDB should use case-insensitive column name comparisons like the rest of the server
Problem affects INPLACE ALTER rename columns.

innobase_rename_column_try(): some strcmp() was replaced with my_strcasecmp(),
queries to update data dictionary was updated to not match column name case.
2018-11-05 11:59:59 +02:00
Marko Mäkelä
8a346f31b9 MDEV-17073 INSERT…ON DUPLICATE KEY UPDATE became more deadlock-prone
thd_rpl_stmt_based(): A new predicate to check if statement-based
replication is active. (This can also hold when replication is not
in use, but binlog is.)

que_thr_stop(), row_ins_duplicate_error_in_clust(),
row_ins_sec_index_entry_low(), row_ins(): On a duplicate key error,
only lock all index records when statement-based replication is in use.
2018-11-02 18:52:39 +02:00
Marko Mäkelä
cfa047069e Remove an unused declaration 2018-11-02 14:30:08 +02:00
Marko Mäkelä
f0cb21ea2e Remove dead code is_thd_killed() 2018-11-02 12:42:01 +02:00
Marko Mäkelä
a33c0e3f34 Minor clean-up for MDEV-17038
ha_innobase::check_if_supported_inplace_alter(): Remove a redundant
condition and defer some computations.

If INNOBASE_ALTER_VERSIONED_REBUILD is set, innobase_need_rebuild()
will necessarily hold. Therefore, it is not necessary to assign
need_rebuild at the start of the function.
2018-11-01 17:30:11 +02:00
Sergei Golubchik
dd6e74c62a MDEV-16774 SET PASSWORD and ALTER USER with slightly different results
set both `password` and `authentication_string` columns in `mysql`.`user`
table for now.

Suppress the "password was ignored" warning if the password is
the same as the authentication string
2018-11-01 15:48:15 +01:00
Aleksey Midenkov
4acfc6ecd9 MDEV-17038 ALTER TABLE CHANGE COLUMN c1 c1 bigint NOT NULL - generates error if table uses SYSTEM VERSIONING
* Fine-grained inplace skipping by INNOBASE_ALTER_VERSIONED_REBUILD;
* Fixed column WITHOUT SYSTEM VERSIONING + ADD COLUMN;
* Fixed instant field change (MDEV-16330);
* Revisited test versioning.online;
* Merged the test versioning.trx_id_versioning_attribute_persistence
  to versioning.online;
* Renamed some versioning functions:
** change_fields_versioning_cache() -> vers_change_fields_cache()
** change_field_versioning_try() -> vers_change_field_try()
Skip condition moved out of func.

Closes tempesta-tech/mariadb#414
Closes tempesta-tech/mariadb#540
Related to tempesta-tech/mariadb#281
2018-11-01 16:42:06 +02:00
Marko Mäkelä
abcd09c95a mtr_t::start(): Remove unused parameters
The parameters bool sync=true, bool read_only=false of mtr_t::start()
were added in
eca5b0fc17
(MySQL 5.7.3).

The parameter read_only was never used anywhere.
The parameter sync was only copied around, and would be returned
by the unused function mtr_t::is_async().

We do not need this dead code in MariaDB.
2018-11-01 10:48:56 +02:00
Marko Mäkelä
d355be8877 Remove dead code in dict_build_table_def_step()
At the start of the function, we already dereferenced node,
so the checks for NULL are unnecessary and redundant.
2018-11-01 10:40:14 +02:00
Sergei Golubchik
c0d75a6d26 spider.show_system_tables: make the result stable 2018-10-31 16:06:16 +01:00
Sergei Golubchik
7c40996cc8 MDEV-12321 authentication plugin: SET PASSWORD support
Support SET PASSWORD for authentication plugins.

Authentication plugin API is extended with two optional methods:
* hash_password() is used to compute a password hash (or digest)
  from the plain-text password. This digest will be stored in mysql.user
  table
* preprocess_hash() is used to convert this digest into some memory
  representation that can be later used to authenticate a user.
  Build-in plugins convert the hash from hexadecimal or base64 to binary,
  to avoid doing it on every authentication attempt.

Note a change in behavior: when loading privileges (on startup or on
FLUSH PRIVILEGES) an account with an unknown plugin was loaded with a
warning (e.g. "Plugin 'foo' is not loaded"). But such an account could
not be used for authentication until the plugin is installed. Now an
account like that will not be loaded at all (with a warning, still).
Indeed, without plugin's preprocess_hash() method the server cannot know
how to load an account. Thus, if a new authentication plugin is
installed run-time, one might need FLUSH PRIVILEGES to activate all
existing accounts that were using this new plugin.
2018-10-31 16:06:16 +01:00
Sergei Golubchik
76151f3cbc Use mysql.user.authentication_string for password
Don't distinguish between a "password hash" and "authentication string"
anymore. Now both are stored in mysql.user.authentication_string, both
are handled identically internally. A "password hash" is just how some
particular plugins interpret authentication string.

Set mysql.user.plugin even if there is no password. The server will use
mysql_native_password plugin in these cases, let's make it expicit.

Remove LEX_USER::pwhash.
2018-10-31 16:06:16 +01:00