Commit graph

182367 commits

Author SHA1 Message Date
Daniel Black
c28be510d1 Power8: use C implementation of crc32 instead of ASM
Compiles to same vector code, just a bit simplier.
vec_crc32.c is now identical to upstream
(https://github.com/antonblanchard/crc32-vpmsum/).

C code by Rogerio Alves <rogealve@br.ibm.com>

This implemention has been tested on big endian too.

Signed-off-by: Daniel Black <daniel@linux.ibm.com>
2018-05-07 09:26:12 +10:00
Daniel Black
891f202193 tests: extend func_math crc32 to a larger size to hit crc32-vpmsum accelerated functions
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2018-05-07 09:25:30 +10:00
Olivier Bertrand
0138220ff6 - Fix MDEV-15735 CONNECT [filamtxt.cpp:429]: Suspicious condition
modified:   storage/connect/filamtxt.cpp

- Fix compiler warnings
  modified:   storage/connect/domdoc.cpp
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/json.h
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/tabext.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h

- Miscelleanous from 10.3
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/mycat.cc
  modified:   storage/connect/user_connect.cc
2018-05-07 00:27:13 +02:00
Michael Widenius
a0bc3b7eee Change read_to_buffer to use ulong instead of uint
This is mostly to document that read_to_buffer can read more than 65K.
Also changed merge_buffers to return bool instead of int
2018-05-07 00:07:33 +03:00
Michael Widenius
062a3176e7 Remove mem_alloc_error()
As thd->alloc() and new automatically calls my_error(ER_OUTOFMEORY)
there is no reason to call mem_alloc_error()

Other things:
- Fixed bug in mysql_unpack_partition() where lex.part_info was
  changed even if it would be a null pointer
2018-05-07 00:07:33 +03:00
Michael Widenius
70c1110a29 Optimize performance schema likely/unlikely
Performance schema likely/unlikely assume that performance schema
is enabled by default, which causes a performance degradation for
default installations that doesn't have performance schema enabled.

Fixed by changing the likely/unlikely in PS to assume it's
not enabled. This can be changed by compiling with
-DPSI_ON_BY_DEFAULT

Other changes:
- Added psi_likely/psi_unlikely that is depending on
  PSI_ON_BY_DEFAULT. psi_likely() is assumed to be true
  if PS is enabled.
- Added likely/unlikely to some PS interface code.
- Moved pfs_enabled to mysys (was initialized but not used before)
- Added "if (pfs_likely(pfs_enabled))" around calls to PS to avoid
  an extra call if PS is not enabled.
- Moved checking flag_global_instrumention before other flags
  to speed up the case when PS is not enabled.
2018-05-07 00:07:33 +03:00
Michael Widenius
9d6dc39ad9 Add checking of correct likely/unlikely
To use:
- Compile with -DUSE_MY_LIKELY
- Change (with replace) all likely/unlikely to my_likely/my_/unlikely
  replace likely my_likely unlikely my_unlikely -- *c *h
- Start mysqld with -T
- run some test
- When mysqld has shut down cleanely, report will be on stderr
2018-05-07 00:07:32 +03:00
Monty
30ebc3ee9e Add likely/unlikely to speed up execution
Added to:
- if (error)
- Lex
- sql_yacc.yy and sql_yacc_ora.yy
- In header files to alloc() calls
- Added thd argument to thd_net_is_killed()
2018-05-07 00:07:32 +03:00
Oleksandr Byelkin
a22a339f8e MDEV-13024: Server crashes in my_store_ptr upon DELETE from sequence in multi-table format
It is test only (fix was done by Monty in ha_sequence::open by allocating ref)
2018-05-06 22:21:55 +02:00
Varun Gupta
724ab9a1cb MDEV-16057: Using optimization Splitting with Group BY we see an unnecessary attached condition
t1.pk IS NOT NULL where pk is a PRIMARY KEY

For equalites in the WHERE clause we create a keyuse array that contains the set of all equalities.
For each KEYUSE inside the keyuse array we have a field "null_rejecting"
which tells that the equality will not hold if either the left or right
hand side of the equality is NULL.
If the equality is NULL rejecting then we accordingly add a NOT NULL condition for the field present in
the item val(present in the KEYUSE struct) when we are doing ref access.

For the optimization of splitting with GROUP BY we always set the null_rejecting to TRUE and we are doing ref access on
the GROUP by field. This does create a problem when the equality is NOT NULL rejecting. This happens in this case as
in the equality we have the right hand side as t1.pk where pk is a PRIMARY KEY , hence it is NOT NULLABLE. So we should have
null rejecting set to FALSE for such a case.
2018-05-06 23:05:37 +05:30
Monty
0bfd45f634 Fix for MDEV-15812 Assert in SEQUENCE when forcing STATEMEMT format
The bug was the we copied the lock type to the underlying engine even when
external_lock failed.
2018-05-06 19:39:48 +03:00
Monty
529c1a3b6c Fixed compiler warnings in spider 2018-05-06 16:22:38 +03:00
Sergei Golubchik
b4e8ad5080 update test results 2018-05-06 14:59:15 +02:00
Alexander Barkov
4891d514b6 MDEV-16095 Oracle-style placeholder inside GROUP BY..WITH ROLLUP breaks replication 2018-05-06 16:10:49 +04:00
Sergei Golubchik
f214d36512 ASAN error in is_stat_table()
don't memcmp beyond the first argument's end

Also:
  use my_strcasecmp(table_alias_charset), like elsewhere, not memcmp
2018-05-05 14:02:33 +02:00
Sergei Golubchik
9989c26bc9 Merge branch '10.0' into 10.1 2018-05-05 14:01:59 +02:00
Marko Mäkelä
39d248fa55 MDEV-16092 Crash in encryption.create_or_replace
If the tablespace is dropped or truncated after the
space->is_stopping() check in fil_crypt_get_page_throttle_func(),
we would proceed to request the page, and eventually report a fatal
error.

buf_page_get_gen(): Do not retry reading if mode==BUF_GET_POSSIBLY_FREED.

lock_rec_block_validate(): Be prepared for a NULL return value when
invoking buf_page_get_gen() with mode=BUF_GET_POSSIBLY_FREED.
2018-05-04 22:44:33 +03:00
Marko Mäkelä
b20039dd80 Merge 10.2 into 10.3 2018-05-04 22:02:05 +03:00
Sergei Golubchik
3c07ed141c Merge branch '5.5' into 10.0 2018-05-04 17:35:09 +02:00
Sergei Golubchik
04b1e61d69 compiler warning
storage/connect/tabjson.cpp:198:10: warning: converting to non-pointer type ‘int’ from NULL [-Wconversion-null]
   return NULL;
2018-05-04 17:35:01 +02:00
Marko Mäkelä
d257c425f2 MDEV-15764 InnoDB may write uninitialized garbage to redo log
log_sys_init(), log_buffer_extend(): Add TRASH_ALLOC() instrumentation

log_write_up_to(): Correctly calculate the byte offset.
2018-05-04 15:57:06 +03:00
Sergei Golubchik
1d58d184c2 protocol: verify that number of rows is correct 2018-05-04 14:40:19 +02:00
Marko Mäkelä
3b50cd2492 Make a test independent of the build options
The number of records in INFORMATION_SCHEMA.COLUMNS depends on the
build options, and could easily change when features are added.
We are not interested in the number of rows returned. The test was
originally added because of problem 15 reported in MDEV-13900
(testing for MDEV-11369 instant ADD COLUMN). The issue was an
assertion failure ut_ad(!rec_is_default_row(rec, index))
in lock_clust_rec_cons_read_sees(), because the 'default row' record
was not being properly ignored by the b-tree cursor.
2018-05-04 14:44:58 +03:00
Elena Stepanova
d36034b6d8 Follow up for 9a84980668 - adjustments to storage_engine tests 2018-05-03 20:54:23 +03:00
Jacob Mathew
ca174051d8 MDEV-14019: Spider + binlog_format = ROW => CRASH
The fix for this bug was automatically merged from 10.2.  However, that fix
was incomplete in 10.3.  This commit is for the additional changes that are
necessary in 10.3.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.
2018-05-03 09:49:03 -07:00
Vladislav Vaintroub
1cb7c4bfc0 MDEV-16084 Calling exit() from a signal handler is unsafe.
Call _exit() from signal handler. main() can just do return.
2018-05-03 16:14:54 +01:00
Monty
57c3dd991b MDEV-15106 Unexpected ER_WRONG_INSERT_INTO_SEQUENCE upon INSERT with multiple locks on sequences
Fixed by removing the check of single lock in sequence insert and let MDL
code handle deadlock detection
2018-05-03 17:49:39 +03:00
Daniel Bartholomew
a411910dd6 bump the VERSION 2018-05-03 10:26:00 -04:00
Marko Mäkelä
6c43068d63 MDEV-15060 Assertion in row_log_table_apply_op after instant ADD when the table is emptied during subsequent ALTER TABLE
During an online table rebuild, a table could be emptied and converted
from 'instant ADD' format to plain (pre-10.3) format. All online_log
records for rebuilding the table must be written and parsed in the
format of the table that existed at the start of the operation.

row_log_t::n_core_fields: A new field for recording index->n_core_fields
when online ALTER is initiated in row_log_allocate().

row_log_t::is_instant(): Determine if the log is in the instant format.
Only invoked by the row_log_table_ family of functions.

dict_index_t::get_n_nullable(): Remove is_instant() debug assertions.
Because a table can be converted to non-instant format during a
table-rebuilding ALTER TABLE, these assertions would be bogus when
executing row_log_table_apply().

rec_init_offsets_temp(): Add the parameter n_core for passing the
original index->n_core_fields.

rec_init_offsets_temp(): Add a 3-parameter variant.

rec_init_offsets_comp_ordinary(): Add the parameter n_core for
passing the index->n_core_fields.
2018-05-03 15:17:16 +03:00
Marko Mäkelä
b432d4ad66 MDEV-16080 Crash in online table rebuild with concurrent DELETE of many BLOBs
Merge a fix from MySQL 8.0.11.

row_log_table_apply_delete(): Remove the unused parameter thr.
(This removal was not part of the Oracle fix.)
2018-05-03 11:42:19 +03:00
Mayank Prasad
f47eac2882 Bug #25928471: ONLINE ALTER AND CONCURRENT DELETE ON TABLE WITH MANY TEXT COLUMNS CAUSES CRASH
Issue:
------
Prefix for externally stored columns were being stored in online_log when a
table is altered and alter causes table to be rebuilt. Space in online_log is
limited and if length of prefix of externally stored columns is very big, then
it is being written to online log without making sure if it fits. This leads to
memory corruption.

Fix:
----
After fix for Bug#16544143, there is no need to store prefixes of externally
stored columnd in online_log. Thus remove the code which stores column prefixes
for externally stored columns. Also, before writing anything on online_log,
make sure it fits to available memory to avoid memory corruption.

Read RB page for more details.

Reviewed-by: Annamalai Gurusami <annamalai.gurusami@oracle.com>
RB: 18239
2018-05-03 11:41:21 +03:00
Marko Mäkelä
01843d1910 Follow-up to MDEV-12266: Do not display table->space
MDEV-12266 changed dict_table_t::space to a pointer.
Displaying pointer values in error messages would be even more
meaningless than displaying numeric tablespace identifiers.

row_create_table_for_mysql(): Do not display table->space when
deleting the file fails. We cannot dereference table->space here,
because fil_delete_tablespace() would have freed the object.

fil_wait_crypt_bg_threads(): Do not display table->space. We could
display table->space_id here, but it should not really add any value,
because the table reference-counts have no direct connection to files
or tablespaces.
2018-05-03 10:26:18 +03:00
Marko Mäkelä
73a10cbcc5 MDEV-16065 Assertion failed in btr_pcur_restore_position_func on UPDATE
btr_pcur_store_position(): Assert that the 'default row' record never
is the only record in a page. (If that would happen, an empty
root page would be re-created in the non-instant format, not containing
the special record.) When the cursor is positioned on the page infimum,
never use the 'default row' as the BTR_PCUR_BEFORE reference.
(This is additional cleanup, not fixing the bug.)

rec_copy_prefix_to_buf(): When converting a record prefix to
the non-instant-add format, copy the original number of null flags.
Rename the variable instant_len to instant_omit, and introduce a
few more variables to make the code easiser to read.

Note: In purge, rec_copy_prefix_to_buf() is also used for storing the
persistent cursor position on a 'default row' record. The stored record
reference will be garbage, but row_search_on_row_ref() will do special
handling to reposition the cursor on the 'default row', based on
ref->info_bits.

innodb.dml_purge: Also cover the 'default row'.
2018-05-02 15:44:52 +03:00
Thirunarayanan Balathandayuthapani
dcc5de66f4 MDEV-16071 Server crashed in innobase_build_col_map / prepare_inplace_alter_table_dict or Assertion `tuple' failed in dtuple_get_nth_field upon altering table with virtual column
- Virtual column should be considered during innobase_build_col_map() to find out
whether the field changed from NULL to NOT NULL.
2018-05-02 16:42:00 +05:30
Thirunarayanan Balathandayuthapani
8a941bad7d MDEV-14168 Unconditionally allow ALGORITHM=INPLACE for setting a column NOT NULL
- Added two new test case for it.
2018-05-02 13:55:34 +05:30
Jacob Mathew
da3c5c3c9a MDEV-15698: Spider ignores syntax errors in connection string in COMMENT field
When a comma separator is missing between COMMENT fields, Spider ignores the
parameter values that are beyond the last expected parameter value.  There are
also some error messages that Spider does generate on COMMENT fields that are
incorrectly formed.

I have introduced additional infrastructure in Spider to fix these problems.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Cherry-Picked:
  Commit c10da98 on branch bb-10.3-MDEV-15698
2018-05-01 18:47:04 -07:00
Jacob Mathew
a7852e3ec8 MDEV-15698: Spider ignores syntax errors in connection string in COMMENT field
When a comma separator is missing between COMMENT fields, Spider ignores the
parameter values that are beyond the last expected parameter value.  There are
also some error messages that Spider does generate on COMMENT fields that are
incorrectly formed.

I have introduced additional infrastructure in Spider to fix these problems.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Merged From:
  Branch bb-10.3-MDEV-15698
2018-05-01 16:45:03 -07:00
Jacob Mathew
72f0efac67 MDEV-15697: Remote user used by Spider needs SUPER privilege
The remote users need the SUPER privilege because by default Spider sends a
'SET SQL_LOG_OFF' statement to the data nodes.  This is controlled by the
spider_internal_sql_log_off configuration setting on the Spider node, which
can only be set to 0 or 1, with a default value of 1.

I have fixed the problem by changing this configuration setting so that if it
is NOT SET, which is the most likely case, the Spider node DOES NOT SEND the
'SET SQL_LOG_OFF' statement to the data nodes.  However if the
spider_internal_sql_log_off setting IS EXPLICITLY SET to either 0 or 1, then
the Spider node DOES SEND the 'SET SQL_LOG_OFF' statement, requiring a remote
user with the SUPER privilege.  The Spider documentation will be updated to
reflect this change.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.
2018-05-01 14:14:06 -07:00
Marko Mäkelä
2b27ac8282 Fix many -Wunused-parameter
Remove unused InnoDB function parameters and functions.

i_s_sys_virtual_fill_table(): Do not allocate heap memory.

mtr_is_block_fix(): Replace with mtr_memo_contains().

mtr_is_page_fix(): Replace with mtr_memo_contains_page().
2018-05-01 16:52:19 +03:00
Sergei Golubchik
42fac32413 Merge branch '5.5' into 10.0 2018-05-01 11:47:43 +02:00
Daniel Black
87b0434e0f MDEV-12645: mysql_install_db --no-test (postfix
mysql_test_db.sql is in the srcdir
2018-05-01 11:35:13 +04:00
Jacob Mathew
8fdeb079b9 MDEV-15712: If remote server used by Spider table is unavailable, some operations hang for a long time
When an attempt to connect to the remote server fails, Spider retries to
connect to the remote server 1000 times or until the connection attempt
succeeds.  This is perceived as a hang if the remote server remains
unavailable.

I have introduced changes in Spider's table status handler to fix this problem.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Cherry-Picked:
  Commit 6ee6933 on branch bb-10.3-MDEV-15712
2018-04-30 19:44:02 -07:00
Jacob Mathew
bf92784d38 MDEV-15712: If remote server used by Spider table is unavailable, some operations hang for a long time
When an attempt to connect to the remote server fails, Spider retries to
connect to the remote server 1000 times or until the connection attempt
succeeds.  This is perceived as a hang if the remote server remains
unavailable.

I have introduced changes in Spider's table status handler to fix this problem.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Merged From:
  Branch bb-10.3-MDEV-15712.
2018-04-30 19:07:44 -07:00
Sergei Golubchik
fab383aac0 Use after free in authentication 2018-05-01 00:30:17 +02:00
Sergey Vojtovich
68cbabbfb0 MDEV-15938 - TINYTEXT CHARACTER SET utf8 COMPRESSED truncates data
Unexpected data truncation may occur when storing data to compressed blob
column having multi byte variable length character sets.

The reason was incorrect number of characters limit was enforced for
blobs.
2018-04-30 19:34:08 +04:00
Sergey Vojtovich
9a84980668 MDEV-12645 - mysql_install_db: no install test db option
Added --skip-test-db option to mysql_install_db. If specified, no test
database created and relevant grants issued.

Removed --skip-auth-anonymous-user option of mysql_install_db. Now it is
covered by --skip-test-db.

Dropped some Debian patches that did the same.

Removed unused make_win_bin_dist.1, make_win_bin_dist and
mysql_install_db.pl.in.
2018-04-30 19:34:08 +04:00
Marko Mäkelä
8bbcc0d505 MDEV-12218: Put back mariabackup --innodb-flush-method
Implement innodb_flush_method as an enum parameter in Mariabackup,
instead of ignoring the option and hard-wiring it to a default value.

xb0xb.h: Remove. Only xtrabackup.cc refers to the enum parameters.

innodb_flush_method_names[], innodb_flush_method_typelib[]:
Define as non-static, so that mariabackup can share the definitions.

srv_file_flush_method: Change the type to ulong, to match the
assignment in init_one_value() and handle_options() in mariabackup.
2018-04-30 18:22:52 +03:00
Marko Mäkelä
9801715cb0 Use compile_time_assert() in InnoDB
Replace most use of #error. Some checks were impossible to
evaluate in the preprocessor due to the use of named
integer constants or enumerations.
2018-04-30 18:22:52 +03:00
Marko Mäkelä
8cd55ae2a9 Fix WITH_INNODB_EXTRA_DEBUG
The checks that used to be enabled by the flags
UNIV_AHI_DEBUG, UNIV_DDL_DEBUG, UNIV_DEBUG_FILE_ACCESSES
were already enabled in debug builds. So, there is no point
in setting these.

Only UNIV_ZIP_DEBUG is set independently of the debug build.

Allow WITH_INNODB_EXTRA_DEBUG to be set for non-debug builds as well.
Currently it only implies UNIV_ZIP_DEBUG, that is, extra validation
for operations on ROW_FORMAT=COMPRESSED tables.

page_zip_validate_low(): Allow the code to be built on non-debug server.
buf_LRU_block_remove_hashed(): Allow the code to be built without
WITH_INNODB_AHI.
2018-04-30 18:19:54 +03:00
Sergei Golubchik
a52c46e069 Bug#25471090: MYSQL USE AFTER FREE
a better fix
2018-04-30 15:49:19 +02:00