Commit graph

178192 commits

Author SHA1 Message Date
Marko Mäkelä
dd75087993 Fix a compilation warning 2017-06-30 15:03:01 +03:00
Sergei Golubchik
d3cc15eb82 Merge branch '10.0' into 10.1 2017-06-30 13:28:39 +02:00
Sergei Golubchik
b503b1c0ed MDEV-11646 main.myisam, maria.maria, main.mix2_myisam, main.myisampack, main.mrr_icp_extra fail in buildbot with valgrind (Syscall param pwrite64(buf) points to uninitialised byte(s))
If the table has a varchar column and a forced fixed for format
(as in varchar.inc), Field_varstring::store() will only store the
actual number of bytes, not padded, in the record[0].

That is, on inserts a part of record[0] can be uninitialized.

Fix: initialize record[0] when a TABLE is created, it doesn't matter
what kind of garbage can be in this unused/invisible part of the
record, as long as it's not some random memory contents
(that can contain sensitive data).
2017-06-30 13:28:28 +02:00
Marko Mäkelä
a1e51e7f47 Mariabackup: Test file cleanup 2017-06-30 11:35:28 +03:00
Marko Mäkelä
4fe89773d8 Mariabackup: Clean up xtrabackup options
Fix the binding of databases_file. It was incorrectly mapped
to OPT_XTRA_TABLES_FILE.

Remove some unused options and variables.
2017-06-30 11:35:28 +03:00
Sergei Golubchik
273e0f663e MDEV-13179 main.errors fails in buildbot and outside with wrong errno 1290 2017-06-30 10:28:01 +02:00
Marko Mäkelä
61847b9d80 Tablespace: Add iterator, const_iterator, begin(), end() 2017-06-30 09:31:19 +03:00
Marko Mäkelä
5ad4645779 Adjust a test for the changed innodb_log_file_size limits 2017-06-30 09:25:39 +03:00
Igor Babaev
9f0c1c0cf6 Fixed the bug mdev-13193.
When an equality that can be pushed into a materialized derived
table / view is extracted from multiple equalities and their
operands are cloned then if they have some pointers to Item_equal
objects those pointers must be set to NULL in the clones. Anyway
they are not valid in the pushed predicates.
2017-06-29 20:50:26 -07:00
Sergei Golubchik
92928bcdd9 Merge branch '5.5' into 10.0 2017-06-29 23:32:18 +02:00
Marko Mäkelä
84e4e4506f Reduce the granularity of innodb_log_file_size
In Mariabackup, we would want the backed-up redo log file size to be
a multiple of 512 bytes, or OS_FILE_LOG_BLOCK_SIZE. However, at startup,
InnoDB would be picky, requiring the file size to be a multiple of
innodb_page_size.

Furthermore, InnoDB would require the parameter to be a multiple of
one megabyte, while the minimum granularity is 512 bytes. Because
the data-file-oriented fil_io() API is being used for writing the
InnoDB redo log, writes will for now require innodb_log_file_size to
be a multiple of the maximum innodb_page_size (65536 bytes).

To complicate matters, InnoDB startup divided srv_log_file_size by
UNIV_PAGE_SIZE, so that initially, the unit was bytes, and later it
was innodb_page_size. We will simplify this and keep srv_log_file_size
in bytes at all times.

innobase_log_file_size: Remove. Remove some obsolete checks against
overflow on 32-bit systems. srv_log_file_size is always 64 bits, and
the maximum size 512GiB in multiples of innodb_page_size always fits
in ulint (which is 32 or 64 bits). 512GiB would be 8,388,608*64KiB or
134,217,728*4KiB.

log_init(): Remove the parameter file_size that was always passed as
srv_log_file_size.

log_set_capacity(): Add a parameter for passing the requested file size.

srv_log_file_size_requested: Declare static in srv0start.cc.

create_log_file(), create_log_files(),
innobase_start_or_create_for_mysql(): Invoke fil_node_create()
with srv_log_file_size expressed in multiples of innodb_page_size.

innobase_start_or_create_for_mysql(): Require the redo log file sizes
to be multiples of 512 bytes.
2017-06-29 23:15:05 +03:00
Marko Mäkelä
e903d458bb Clean up InnoDB shutdown
Tablespace::shutdown(): Clear m_path. This was moved from
Tablespace::~Tablespace().

LatchDebug::shutdown(): Remove a redundant condition.
2017-06-29 23:10:46 +03:00
Marko Mäkelä
591edccc93 Simplify access to the binlog offset in InnoDB
trx_sys_print_mysql_binlog_offset(): Use 64-bit arithmetics and ib::info().

TRX_SYS_MYSQL_LOG_OFFSET: Replaces TRX_SYS_MYSQL_LOG_OFFSET_HIGH,
TRX_SYS_MYSQL_LOG_OFFSET_LOW.

trx_sys_update_mysql_binlog_offset(): Remove the constant parameter
field=TRX_SYS_MYSQL_LOG_INFO. Use 64-bit arithmetics.
2017-06-29 23:03:39 +03:00
Marko Mäkelä
aea0e125d2 trx_sys_read_wsrep_checkpoint(): Return whether a checkpoint is present 2017-06-29 23:01:47 +03:00
Marko Mäkelä
cd623508df buf_read_ibuf_merge_pages(): Discard all entries for a missing tablespace
A similar change was contributed to Percona XtraBackup, but for some
reason, it is not present in Percona XtraDB. Since MDEV-9566
(MariaDB 10.1.23), that change is present in the MariaDB XtraDB.
2017-06-29 22:39:06 +03:00
Marko Mäkelä
68b5aeae4e Minor cleanup of InnoDB I/O routines
Change many function parameters from IORequest& to const IORequest&.

Remove an unused definition of ECANCELED.
2017-06-29 22:30:47 +03:00
Marko Mäkelä
859714e73d Simplify up InnoDB redo log system startup and shutdown
recv_sys_init(): Remove the parameter.

recv_sys_create(): Merge to recv_sys_init().

recv_sys_mem_free(): Merge to recv_sys_close().

log_mem_free(): Merge to log_shutdown().
2017-06-29 22:24:48 +03:00
Marko Mäkelä
8143ef1b7c trx_validate_state_before_free(): Add debug assertions 2017-06-29 22:20:34 +03:00
Marko Mäkelä
bb60a832ed Minor cleanup of InnoDB shutdown
os_thread_active(): Remove.

srv_shutdown_all_bg_threads(): Assert that high-level threads
have already exited. Do not sleep if os_thread_count=0.
2017-06-29 22:20:34 +03:00
Sergei Golubchik
4db6e1e4a5 uninitialized variable 2017-06-29 20:47:08 +02:00
Vicențiu Ciorbaru
5c89f23b45 Fix debug assert post MDEV-10306
tmp variable now points to str->ptr() buffer, not tmp_value buffer.
Comparing pointers otherwise can lead to false assertion errors as we
don't know where buffers are allocated in respect to each other.
2017-06-29 21:34:37 +03:00
Sachin Setiya
de3201df3f Fix MDEV-12758 2017-06-29 23:03:30 +05:30
Sachin Setiya
629c60973d MDEV-12758 wrep.pool_of_threads failed in buildbot with WSREP ...
Problem:- While running tests from wsrep suite , we include file have_wsrep.inc
or have_wsrep_enabled.inc , these file test wsrep plugin is ACTIVE or 'wsrep_on'
is ON. These select does not ensure that whether 'wsrep_ready' is ON , So that
we can process SQL queries. So sometimes we will get error like this
mysqltest: At line 81: query 'call mtr.check_testcase()' failed: 1047: WSREP has
not yet prepared node for application use  not ok

Solution:- In file have_wsrep.inc and have_wsrep_enabled.inc we will include
wait_until_ready.inc , which will wait untill 'wsrep_on' is on
2017-06-29 18:31:42 +05:30
Alexander Barkov
95979d2bae Removing duplicate code in mysql_execute_command: Adding prepare_db_action().
The code in mysql_execute_command() for SQLCOM_CREATE_DB, SQLCOM_DROP_DB
and SQLCOM_ALTER_DB contained around 20 lines of similar code.
Moving this code into a new function prepare_db_action().
2017-06-29 12:24:47 +04:00
Daniel Black
7c997f4b2e travis: force deb build not to use ccache
18a2b0a168 wasn't sufficient.
The path needed to be set so ccache wasn't picked up.
2017-06-29 10:59:40 +04:00
Igor Babaev
13221b1eb2 Fixed a failure of the test case for the bug mdev-13107 in --ps-protocol.
The fix was in the call the open_normal_and_derived_tables() from
the function mysql_test_select() and it was similar to those from
the patch for mdev-13107.

Added explicit PREPARE statements that failed in --ps-protocol.
2017-06-28 22:40:27 -07:00
Daniel Black
18a2b0a168 travis: disable ccache on deb build - ENOSPC 2017-06-29 08:57:52 +04:00
Daniel Black
9e0aa294aa travis: MDEV-13002 fixed - don't allow failures in MYSQL_TEST_SUITES=plugins
Revert "travis: allowed_failures MYSQL_TEST_SUITES=plugins (MDEV-13002)"

This reverts commit c4cbc7a880.
2017-06-29 08:57:52 +04:00
Daniel Black
9222d79b82 travis: packages for debian build - added fakeroot 2017-06-29 08:57:52 +04:00
Igor Babaev
e608023947 Fixed the bug mdev-13107 and some similar unreported bugs.
The problems were in the code of sql_show.cc. There the tables
could be opened in such a way that mysql_derived_init() never
worked for CTE tables. As a result they were not marked as
derived and mysql_handle_derived() were not called for derived
tables used in their specifications.
2017-06-28 11:39:33 -07:00
Vicențiu Ciorbaru
31ba0fa48d MDEV-12851: Case with window functions query crashes server
The "is null" function performs one operation which no other Item_func
does, which is to update used tables during fix_length_and_dec().

This however can not be performed before window functions have had a
chance to resolve their order by and partition by definitions, which
happens after the initial setup_fields call. Consequently, do not call
Item_func_isnull update_used_tables during fix_length_and_dec().

There was another issue detected once the crash was resolved.
Because window functions did not implement is_null() method, we would
end up returning bad results for "is null" and "is not null" functions.
Implemented is_null() method for Item_windowfunc.
2017-06-28 19:21:31 +03:00
Vicențiu Ciorbaru
23edc7c88f MDEV-13186: main.win failure post MDEV-12336
During statement preparation st_order::item gets set to a value in
ref_ptr_array. During statement execution we were overriding that value,
causing subsequent checks for window functions to return true.

Whenever we do any setting from ref_ptr_array, make sure to always
store the value in all_fields as well.

For function items containing window functions, as MDEV-12336 has
discovered, we don't need to create a separate Item_direct_ref or
Item_aggregate_ref as they will be computed directly from the top-level
item once the window function argument columns are computed.
2017-06-28 15:23:36 +03:00
Vicențiu Ciorbaru
9003869390 Simplify IO_CACHE by removing current_pos and end_pos as self-references
These self references were previously used to avoid having to check the
IO_CACHE's type. However, a benchmark shows that on x86 5930k stock,
the type comparison is marginally faster than the double pointer dereference.
For 40 billion my_b_tell calls, the difference is .1 seconds in favor of performing the
type check. (Basically there is no measurable difference)

To prevent bugs from copying the structure using the equals(=) operator,
and having to do the bookkeeping manually, remove these "convenience"
variables.
2017-06-28 15:23:36 +03:00
Marko Mäkelä
b3171607e3 Avoid InnoDB messages about recovery after creating redo logs
srv_log_files_created: A debug flag to ensure that InnoDB redo log
files can only be created once in the server lifetime, and that
after log files have been created, no crash recovery will take place.

recv_scan_log_recs(): Detect the special case where the log consists
of a sole MLOG_CHECKPOINT record, such as immediately after creating
the redo logs.

recv_recovery_from_checkpoint_start(): Skip the recovery message
if the redo log is logically empty.
2017-06-28 11:58:43 +03:00
Sergei Golubchik
a02ba9c1c9 Merge branch '5.5' into 10.0 2017-06-28 10:10:31 +02:00
Marko Mäkelä
3e1d0ff574 Fix a merge error in commit 8f643e2063
A merge error caused InnoDB bootstrap to fail when
innodb_undo_tablespaces was set to more than 2.
This was because of a bug that was introduced to
srv_undo_tablespaces_init() by the merge.

Furthermore, some adjustments for Oracle Bug#25551311 aka
Bug#23517560 changes were forgotten. We must minimize direct
references to srv_undo_tablespaces_open and use predicates
instead.

srv_undo_tablespaces_init(): Increment srv_undo_tablespaces_open
once, not twice, per loop iteration.

is_system_or_undo_tablespace(): Remove (unused function).

is_predefined_tablespace(): Invoke srv_is_undo_tablespace().
2017-06-27 21:23:12 +03:00
Vicențiu Ciorbaru
330792f3c3 Merge branch 'bb-10.1-vicentiu' into 10.1 2017-06-27 16:34:36 +03:00
Sergei Golubchik
d5cd334504 MDEV-13187 incorrect backslash parsing in clients
cover ANSI_QUOTES and NO_BACKSLASH_ESCAPES in mysqltest
2017-06-27 14:00:37 +02:00
Sergei Golubchik
39385ff7b2 MDEV-13187 incorrect backslash parsing in clients
don't do backslash escapes inside backticks
2017-06-27 13:25:50 +02:00
Marko Mäkelä
43c77bb937 Merge 10.2 into bb-10.2-ext 2017-06-27 08:13:59 +03:00
Marko Mäkelä
29624ea304 MDEV-13176 ALTER TABLE…CHANGE col col TIMESTAMP NOT NULL DEFAULT… fails
When it comes to DEFAULT values of columns, InnoDB is imposing both
unnecessary and insufficient conditions on whether ALGORITHM=INPLACE
should be allowed for ALTER TABLE.

When changing an existing column to NOT NULL, any NULL values in the
columns only get a special treatment if the column is changed to an
AUTO_INCREMENT column (which is not supported by ALGORITHM=INPLACE)
or the column type is TIMESTAMP. In all other cases, an error
must be reported for the failure to convert a NULL value to NOT NULL.

InnoDB was unnecessarily interested in whether the DEFAULT value
is not constant when altering other than TIMESTAMP columns. Also,
when changing a TIMESTAMP column to NOT NULL, InnoDB was performing
an insufficient check, and it was incorrectly allowing a constant
DEFAULT value while not being able to replace NULL values with that
constant value.

Furthermore, in ADD COLUMN, InnoDB is unnecessarily rejecting certain
nondeterministic DEFAULT expressions (depending on the session
parameters or the current time).
2017-06-27 07:39:42 +03:00
Elena Stepanova
3125ba8912 Updated list of unstable tests for 10.1.25 release 2017-06-27 03:48:04 +03:00
Igor Babaev
02655a91cf Fixed the bug mdev-13166.
This patch corrects the fix for mdev-12845.
2017-06-26 12:16:04 -07:00
Sergey Vojtovich
6d0aed42c5 MDEV-12070 - Introduce thd_query_safe() from MySQL 5.7
Merged relevant part of MySQL revision:
565d20b44f
2017-06-26 16:11:22 +04:00
Sergey Vojtovich
dd710e7552 MDEV-12882 - Assertion failed in MDL_context::upgrade_shared_lock
Relaxed assertion (in MySQL it was removed).
For "LOCK TABLES t1 WRITE CONCURRENT, t1 READ" upgrade lock to weakest
existing suitable lock, which is MDL_SHARED_NO_READ_WRITE.
2017-06-26 16:11:22 +04:00
Vicențiu Ciorbaru
32659db864 Fix merge error from 10.0
Item_func_sysconst behaves as a non-constant function during prepared
statements and view creation and constant otherwise. Current condition
implied the opposite.
2017-06-26 12:22:27 +03:00
Marko Mäkelä
0d69d313a1 Prevent interleaved error log output on InnoDB startup
buf_flush_page_cleaner_coordinator(): Signal the thread creator
that the error log output regarding setpriority() has been issued.

innobase_start_or_create_for_mysql(): Wait for
buf_flush_page_cleaner_coordinator() to completely start up.

This prevents sporadic failures of tests that search the server
error log for InnoDB redo log recovery messages.
2017-06-23 22:54:42 +03:00
Oleksandr Byelkin
b76b69cd5f MDEV-10880: Assertions keypart_map' or prebuilt->search_tuple->n_fields > 0' fail on DISTINCT and GROUP BY constant
add_group_and_distinct_keys() should take into account JOIN::simple_group.
2017-06-23 09:16:27 +02:00
Marko Mäkelä
0288fa619f Do allow writes for innodb_force_recovery=2 or 3
While the primary purpose of innodb_force_recovery is to allow
data to be rescued from an InnoDB instance that would crash due
to some data corruption, the settings 1, 2, or 3 are relatively
safe to use and there is no need to prevent write transactions
in these modes.

The setting innodb_force_recovery=4 and above can cause database
corruption. For those modes, we already set the flag
high_level_read_only to disable modifications, except DROP TABLE.

MODIFICATIONS_NOT_ALLOWED_MSG_FORCE_RECOVERY: Remove. There is no
need to spam the error log for each refused DML operation. It suffices
to return an error to the client. There will be messages at startup
if innodb_read_only or innodb_force_recovery are preventing writes.
2017-06-23 09:54:31 +03:00
Igor Babaev
9f3622191d Fixed the bug mdev-12845.
This patch fills in a serious flaw in the
code that supports condition pushdown into
materialized views / derived tables.

If a predicate happened to contain a reference
to a mergeable view / derived table and it does
not depended directly on the target materialized
view / derived table then the predicate was not
considered as a subject to pusdown to this view
/ derived table.
2017-06-22 22:06:03 -07:00