Commit graph

175981 commits

Author SHA1 Message Date
Olivier Bertrand
341a36021d - Fix MDEV-13925: Actually this fixes SELECT queries when
the WHERE clause have single quote.
  modified:   storage/connect/ha_connect.cc

- Use Windows VirtualAlloc and VirtualFree for the Sarea workspace
  modified:   storage/connect/global.h
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/plgdbutl.cpp
  modified:   storage/connect/plugutil.cpp
  modified:   storage/connect/user_connect.cc

- Change inihandl from c to c++.
  Because it now includes global.h that contains a bool function definition
  that make compile to fail on Linux.
  modified:   storage/connect/CMakeLists.txt
  removed:    storage/connect/inihandl.c
  added:      storage/connect/inihandl.cpp

- Fix MDEV-13860 CONNECT engine does not build with JDBC without ODBC.
  By including Sergei's patch in connect_assisted_discovery.
  modified:   storage/connect/ha_connect.cc
2017-11-03 15:24:48 +01:00
Olivier Bertrand
b7f352b278 - Update version number
modified:   storage/connect/ha_connect.cc

- Include MONGO in all Java enabled distributions
  Mongo will be enabled only for 10.2 and 10.3
  modified:   storage/connect/CMakeLists.txt

- Change JDBC_SUPPORT to JAVA_SUPPORT which also replaces MONGO_SUPPORT
  MONGO_SUPPORT is now just used to enable the MONGO table type
  modified:   storage/connect/filter.cpp
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h
  modified:   storage/connect/mongo.cpp
  modified:   storage/connect/mycat.cc
  modified:   storage/connect/plgdbutl.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h

- Move MakeSelector function from FILTER to mongo.cpp
  modified:   storage/connect/filter.cpp
  modified:   storage/connect/filter.h
  modified:   storage/connect/cmgoconn.cpp
  modified:   storage/connect/jmgoconn.cpp
  modified:   storage/connect/mongo.cpp

- Do mongo_init only on first use of the MongoDB C Driver
  This will permit to delay load the mongo lib on Windows
  modified:   storage/connect/cmgoconn.cpp
  modified:   storage/connect/cmgoconn.h
  modified:   storage/connect/ha_connect.cc

- Replace NEW_VAR by a test on MYSQL_VERSION_ID
  modified:   storage/connect/ha_connect.cc

- Suppress enable_mongo session variable
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/mycat.cc

- Make some function headers identical in .h and .cc file
  (replacing const char* by PCSZ)
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h

- Change a parameter type from uchar* to const uchar*
  (for ScanRecord and CheckRecord)
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h

- Changes on LIKE and NOT LIKE does not fix a bug yet
  modified:   storage/connect/ha_connect.cc

- Suppress PIVOT_SUPPORT (PIVOT type is unconditionnal)
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/mycat.cc

- Change the strz function from inline to static
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h

- export the JavaConn class and the MgoColumns and IsNum functions
  modified:   storage/connect/javaconn.h
  modified:   storage/connect/json.h
  modified:   storage/connect/mongo.h

- Fix MDEV-13924
  modified:   storage/connect/jdbconn.cpp

- Make a temporary fix for the compiler bug in CalculateArray
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/tabjson.cpp

- Typo
  modified:   storage/connect/jdbccat.h
  modified:   storage/connect/reldef.h
  modified:   storage/connect/tabext.h
  modified:   storage/connect/tabjmg.cpp
  modified:   storage/connect/tabxml.h
  modified:   storage/connect/valblk.h
  modified:   storage/connect/value.h
  modified:   storage/connect/xtable.h

- Fix a bug in MONGO tests by changing 'MONGO' to $TYPE
  modified:   storage/connect/mysql-test/connect/t/mongo_test.inc

- Record test results to reflect all changes
  modified:   storage/connect/mysql-test/connect/r/json_java_2.result
  modified:   storage/connect/mysql-test/connect/r/json_java_3.result
  modified:   storage/connect/mysql-test/connect/r/json_mongo_c.result
  modified:   storage/connect/mysql-test/connect/r/mongo_c.result
  modified:   storage/connect/mysql-test/connect/r/mongo_java_2.result
  modified:   storage/connect/mysql-test/connect/r/mongo_java_3.result
2017-10-17 12:16:32 +02:00
Olivier Bertrand
5aec754011 Fix MDEV-13924 modified: storage/connect/jdbconn.cpp 2017-10-11 12:18:15 +02:00
Olivier Bertrand
791e5fd488 Enable MONGO for the C driver. Modified: modified: storage/connect/CMakeLists.txt 2017-09-11 16:30:45 +02:00
Olivier Bertrand
0f4b540780 - Update version number
modified:   storage/connect/ha_connect.cc

- Regard columns with binary charset as string (was binary)
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/tabmysql.cpp
  modified:   storage/connect/tabutil.cpp

- Support length 0 for CHAR and VARCHAR
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/reldef.cpp
  modified:   storage/connect/value.cpp

- Add ACCEPT option for void columns in discovery
  modified:   storage/connect/tabjson.cpp

- Update some tests because of above change
  modified:   storage/connect/mysql-test/connect/r/json_java_2.result
  modified:   storage/connect/mysql-test/connect/r/json_java_3.result
  modified:   storage/connect/mysql-test/connect/r/json_mongo_c.result
  modified:   storage/connect/mysql-test/connect/r/mongo_c.result
  modified:   storage/connect/mysql-test/connect/r/mongo_java_2.result
  modified:   storage/connect/mysql-test/connect/r/mongo_java_3.result
  modified:   storage/connect/mysql-test/connect/r/odbc_oracle.result
  modified:   storage/connect/mysql-test/connect/r/updelx.result
  modified:   storage/connect/mysql-test/connect/t/mongo_test.inc
2017-09-05 15:58:46 +02:00
Olivier Bertrand
c23b832556 - Fix MongoDB C Driver adding for CMAKE.
Requires MongoDB C Driver version 1.7 now available
  modified:   storage/connect/CMakeLists.txt

- Add more trace to tbl_thread.test (to debug failure)
  modified:   storage/connect/mysql-test/connect/r/tbl_thread.result
  modified:   storage/connect/mysql-test/connect/t/tbl_thread.test
2017-09-02 16:07:46 +02:00
Olivier Bertrand
f6103105c0 Merging from MariaDB repository 2017-08-31 01:26:45 +02:00
Jan Lindström
01209de763 Merge remote-tracking branch 'origin/bb-10.1-jplindst' into 10.1 2017-08-29 20:30:18 +03:00
Olivier Bertrand
5c42d86aff - New distribution enabling or disabling the MONGO table type
modified:   storage/connect/CMakeLists.txt
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/mycat.cc

- Extend SRCDEF parameter processing to MYSQL
  modified:   storage/connect/tabext.cpp
  modified:   storage/connect/tabext.h
  modified:   storage/connect/tabmysql.cpp

- Typo
  modified:   storage/connect/mysql-test/connect/std_data/Mongo2.jar
  modified:   storage/connect/mysql-test/connect/std_data/Mongo3.jar
2017-08-29 18:20:01 +02:00
Jan Lindström
352d27ce36 MDEV-13557: Startup failure, unable to decrypt ibdata1
Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though
encryption is not enabled.

Problem was that we created encryption metadata (crypt_data) for
system tablespace even when no encryption was enabled and too early.
System tablespace can be encrypted only using key rotation.

Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables
require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION
contain row only if tablespace really has encryption metadata.

fil_crypt_set_thread_cnt: Send message to background encryption threads
if they exits when they are ready. This is required to find tablespaces
requiring key rotation if no other changes happen.

fil_crypt_find_space_to_rotate: Decrease the amount of time waiting
when nothing happens to better enable key rotation on startup.

fsp_header_init: Write encryption metadata to page 0 only if tablespace is
encrypted or encryption is disabled by table option.

i_s_dict_fill_tablespaces_encryption : Skip tablespaces that do not
contain encryption metadata. This is required to avoid too early
wait condition trigger in encrypted -> unencrypted state transfer.

open_or_create_data_files: Do not create encryption metadata
by default to system tablespace.
2017-08-29 14:23:34 +03:00
Vladislav Vaintroub
dda40b9304 AWS Key Management : Introduce "mock" variable, available in debug build.
If this variable is set, skip actual AWS calls, and fake/mock
both generation and encryption of the keys.

The advantage of having a mock mode is that more aws_key_management tests
can be enabled on buildbot.
2017-08-28 18:28:07 +00:00
Marko Mäkelä
11352d52cd Merge 10.0 into 10.1 2017-08-28 15:05:46 +03:00
Jan Lindström
bbfd53cd32 Add galera suite to default suites and disable failing test
cases.
2017-08-28 11:39:28 +03:00
Jan Lindström
61096ff214 MDEV-13591: InnoDB: Database page corruption on disk or a failed file read and assertion failure
Problem is that page 0 and its possible enrryption information
is not read for undo tablespaces.

fil_crypt_get_latest_key_version(): Do not send event to
encryption threads if event does not yet exists. Seen
on regression testing.

fil_read_first_page: Add new parameter does page belong to
undo tablespace and if it does, we do not read FSP_HEADER.

srv_undo_tablespace_open : Read first page of the tablespace
to get crypt_data if it exists and pass it to fil_space_create.

Tested using innodb_encryption with combinations with
innodb-undo-tablespaces.
2017-08-28 09:49:30 +03:00
Elena Stepanova
882f4566e5 Combinations with innodb-undo-tablespaces to use in test files 2017-08-28 09:49:30 +03:00
Olivier Bertrand
5b998bf953 - Fix MDEV-13621 JDBC UPDATE containing single or double quote chars produces wrong result
in ha_connect::GetStringOption
  modified:   storage/connect/ha_connect.cc

- Begin implement data type BINARY
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/myutil.cpp
  modified:   storage/connect/valblk.cpp
  modified:   storage/connect/valblk.h
  modified:   storage/connect/value.cpp

- Fix MDEV-12422 CONNECT Engine to support CHECK TABLE
  Adding a fake check function returning HA_ADMIN_OK.
  modified:   storage/connect/ha_connect.h

- Treat TBL (thread) as local when connected to the current server
  and return by timeout when a TBL remote table connection fail (Thread only)
  modified:   storage/connect/myconn.cpp
  modified:   storage/connect/tabmysql.h
  modified:   storage/connect/tabtbl.cpp
  modified:   storage/connect/tabtbl.h

- Update some tests and result files
  Add test output to tbl_thread.test tryng to understand failure
  modified:   storage/connect/mysql-test/connect/r/tbl_thread.result
  modified:   storage/connect/mysql-test/connect/t/tbl_thread.test
  modified:   storage/connect/mysql-test/connect/r/updelx.result

- Add the GetCsName function
  modified:   storage/connect/reldef.h
2017-08-27 17:23:27 +02:00
Marko Mäkelä
582545a384 MDEV-13637 InnoDB change buffer housekeeping can cause redo log overrun and possibly deadlocks
The function ibuf_remove_free_page() may be called while the caller
is holding several mutexes or rw-locks. Because of this, this
housekeeping loop may cause performance glitches for operations that
involve tables that are stored in the InnoDB system tablespace.
Also deadlocks might be possible.

The worst impact of all is that due to the mutexes being held, calls to
log_free_check() had to be skipped during this housekeeping.
This means that the cyclic InnoDB redo log may be overwritten.
If the system crashes during this, it would be unable to recover.

The entry point to the problematic code is ibuf_free_excess_pages().
It would make sense to call it before acquiring any mutexes or rw-locks,
in any 'pessimistic' operation that involves the system tablespace.

fseg_create_general(), fseg_alloc_free_page_general(): Do not call
ibuf_free_excess_pages() while potentially holding some latches.

ibuf_remove_free_page(): Do call log_free_check(), like every operation
that is about to generate redo log should do.

ibuf_free_excess_pages(): Remove some assertions that are replaced
by stricter assertions in the log_free_check() that is now called by
ibuf_remove_free_page().

row_ins_sec_index_entry(), row_undo_ins_remove_sec_low(),
row_undo_mod_del_mark_or_remove_sec_low(),
row_undo_mod_del_unmark_sec_and_undo_update(): Call
ibuf_free_excess_pages() if the operation may involve allocating pages
and change buffering in the system tablespace.
2017-08-25 14:01:51 +03:00
Vladislav Vaintroub
cd35dd6a05 Windows : Do not use CRT routine to dump memory leaks.
Its output is useless,and, in case of large output, it also may prevent
with search_pattern_in_file.inc from working.
2017-08-24 15:49:50 +00:00
Jan Lindström
efc0ec7631 Merge remote-tracking branch 'origin/bb-10.1-galera' into 10.1 2017-08-24 11:18:21 +03:00
Vladislav Vaintroub
dd229430a9 Windows compile : make compilation fail on "uninitialized variable used" warning C4700
This is a genuine error, and will crash debug buildd in runtime checks if not fixed.
it is better to fail during compile.
2017-08-24 08:05:11 +00:00
Vladislav Vaintroub
7aa846e9e3 CONNECT engine: install ha_connect.lib 2017-08-23 23:30:51 +00:00
Vladislav Vaintroub
7b36395ee9 MDEV-13630 : dont install connect-specific JAR files if connect is not built. 2017-08-23 23:29:59 +00:00
Vladislav Vaintroub
db51ad1e01 Remove workaround for ancient and already fixed CMake bug in MSI creation.
The workaround, an extra cmake calls, somehow makes the
connect/cmake_install.cmake to lose installation of connect-engine's
specific jar files.
2017-08-23 18:11:24 +00:00
Sachin Setiya
5077cc0b1a Fix Merge Error 2017-08-23 16:49:42 +05:30
Marko Mäkelä
b8b3ba632b MDEV-13606 XA PREPARE transactions should survive innodb_force_recovery=1 or 2
When MySQL 5.0.3 introduced InnoDB support for two-phase commit,
it also introduced the questionable logic to roll back XA PREPARE
transactions on startup when innodb_force_recovery is 1 or 2.

Remove this logic in order to avoid unwanted side effects when
innodb_force_recovery is being set for other reasons. That is,
XA PREPARE transactions will always remain in that state until
InnoDB receives an explicit XA ROLLBACK or XA COMMIT request
from the upper layer.

At the time the logic was introduced in MySQL 5.0.3, there already
was a startup parameter that is the preferred way of achieving
the behaviour: --tc-heuristic-recover=ROLLBACK.
2017-08-23 13:03:13 +03:00
Marko Mäkelä
97f9d3c080 MDEV-13167 InnoDB key rotation is not skipping unused pages
In key rotation, we must initialize unallocated but previously
initialized pages, so that if encryption is enabled on a table,
all clear-text data for the page will eventually be overwritten.
But we should not rotate keys on pages that were never allocated
after the data file was created.

According to the latching order rules, after acquiring the
tablespace latch, no page latches of previously allocated user pages
may be acquired. So, key rotation should check the page allocation
status after acquiring the page latch, not before. But, the latching
order rules also prohibit accessing pages that were not allocated first,
and then acquiring the tablespace latch. Such behaviour would indeed
result in a deadlock when running the following tests:
encryption.innodb_encryption-page-compression
encryption.innodb-checksum-algorithm

Because the key rotation is accessing potentially unallocated pages, it
cannot reliably check if these pages were allocated. It can only check
the page header. If the page number is zero, we can assume that the
page is unallocated.

fil_crypt_rotate_page(): Detect uninitialized pages by FIL_PAGE_OFFSET.
Page 0 is never encrypted, and on other pages that are initialized,
FIL_PAGE_OFFSET must contain the page number.

fil_crypt_is_page_uninitialized(): Remove. It suffices to check the
page number field in fil_crypt_rotate_page().
2017-08-23 10:12:39 +03:00
Vladislav Vaintroub
a00b74d994 fix auth_plugin_win test
prepend enable-named-pipe (windows-only) option in auth_plugin_win.opt
with loose- prefix, to avoid warning on non-Windows.
2017-08-22 13:05:56 +00:00
Vladislav Vaintroub
9af7561eb4 MDEV-13608 : set client plugin directory with mysql_options()
if plugin_dir is specified. Also, allow to specify protocol (e.g pipe)
2017-08-21 17:16:12 +00:00
Jan Lindström
0aeff8c36a Fix compiler error. 2017-08-21 14:48:30 +03:00
Jan Lindström
c23efc7d50 Merge remote-tracking branch 'origin/10.0-galera' into 10.1 2017-08-21 13:35:00 +03:00
Jan Lindström
f1af211499 Add galera_admin to disabled. 2017-08-21 07:11:04 +03:00
Olivier Bertrand
5f83a9b751 - Fix MDEV-12422 by implementing a fake virtual "check" function.
modified:   storage/connect/ha_connect.h
2017-08-20 13:13:52 +02:00
Jan Lindström
d20923debb Add more disabled test. 2017-08-20 07:49:07 +03:00
Jan Lindström
449a996c6a Add more disabled tests and one ignored warning. 2017-08-19 07:52:31 +03:00
Olivier Bertrand
b6afeacd33 - Fix failing test tbl_thread on linux
(and mask another fail until MDEV-10179 is fixed)
  modified:   storage/connect/mysql-test/connect/r/tbl_thread.result
  modified:   storage/connect/mysql-test/connect/t/tbl_thread.test
2017-08-18 12:45:31 +02:00
Jan Lindström
f7e1b99895 Galera test fixes and add remaining test failures as disabled. 2017-08-18 11:31:03 +03:00
Jan Lindström
ce6c0e584e MDEV-8960: Can't refer the same column twice in one ALTER TABLE
Problem was that if column was created in alter table when
it was refered again it was not tried to find from list
of current columns.

mysql_prepare_alter_table:
  There is two cases
    (1) If alter table adds a new column and then later alter
        changes the field definition, there was no check from
	list of new columns, instead an incorrect error was given.
    (2) If alter table adds a new column and then later alter
        changes the default, there was no check from list of
	new columns, instead an incorrect error was given.
2017-08-18 10:11:54 +03:00
Marko Mäkelä
06106c0148 MDEV-12988 backup fails if innodb_undo_tablespaces>0
srv_undo_tablespaces_init(): In Mariabackup backup mode,
do initialize the array of undo_tablespace_ids[].
2017-08-17 16:18:02 +03:00
Jan Lindström
109b858258 MDEV-13432: Assertion failure in buf0rea.cc line 577
Page read could return DB_PAGE_CORRUPTED error that should
be reported and passed to upper layer. In case of unknown
error code we should print both number and string.
2017-08-17 07:19:12 +03:00
Jan Lindström
0a479f7860 More test failure fixes. 2017-08-16 13:10:01 +03:00
Jan Lindström
7ce37d9757 Move galera_ist_progress and galera_ist_restart_joiner tests under big_test.
This is because they could cause out of storage if run on /dev/shm.
2017-08-16 10:14:19 +03:00
Jan Lindström
81fd8ff676 Fix test failures. 2017-08-16 07:49:19 +03:00
Jan Lindström
5017c261d4 Fix test failure on test MW-86 and remove MW-360 test.
Merged from mysql-wsrep-bugs following:

GCF-1058 MTR test galera.MW-86 fails on repeated runs
Wait for the sync point sync.wsrep_apply_cb to be reached before
executing the test and clearing the debug flag sync.wsrep_apply_cb.

The race scenario:

Intended behavior:
node2: set sync.wsrep_apply_cb in order to start waiting in the background INSERT
node1: INSERT start
node2 (background): INSERT start
node1: INSERT end
node2: send signal to background INSERT: "stop waiting and continue executing"
node2: clear sync.wsrep_apply_cb as no longer needed
node2 (background): consume the signal
node2 (background): INSERT end
node2: DROP TABLE
node2: check no pending signals are left - ok

What happens occasionally (unexpected):
node2: set sync.wsrep_apply_cb in order to start waiting in the background INSERT
node1: INSERT start
node2 (background): INSERT start
node1: INSERT end
// The background INSERT still has _not_ reached the place where it starts
// waiting for the signal:
// DBUG_EXECUTE_IF("sync.wsrep_apply_cb", "now wait_for...");
node2: send signal to background INSERT: "stop waiting and continue executing"
node2: clear sync.wsrep_apply_cb as no longer needed
// The background INSERT reaches DBUG_EXECUTE_IF("sync.wsrep_apply_cb", ...)
// but sync.wsrep_apply_cb has already been cleared and the "wait" code is not
// executed. The signal remains unconsumed.
node2 (background): INSERT end
node2: DROP TABLE
node2: check no pending signals are left - failure, signal.wsrep_apply_cb is
pending (not consumed)

Remove MW-360 test case as it is not intended for MariaDB (uses
MySQL GTID).
2017-08-15 13:57:15 +03:00
Sergei Golubchik
48fe832650 Merge branch '10.0' into 10.1 2017-08-15 09:50:31 +02:00
Sergey Vojtovich
3e20a42bfb MDEV-8579 - Some sysvars in I_S are missing any meaningful help (comment) text
Follow-up to original patch: fixing test cases.
2017-08-15 10:13:57 +04:00
=Ian Gilfillan
f066c89a97 MDEV-8579 Expand system variable documentation 2017-08-15 10:13:57 +04:00
Sergei Golubchik
7581fb23e2 compilation fix for SLES 11 SP4
also fix innodb
2017-08-14 18:37:53 +02:00
Teemu Ollakka
b5323054af MW-383 Bumped wsrep patch version 2017-08-14 14:41:50 +03:00
Teemu Ollakka
dd72d66c45 MW-373 Wait for wsrep_ready at startup when provider is loaded
Conflicts:
	mysql-test/include/kill_and_restart_mysqld.inc
2017-08-14 14:41:12 +03:00
sjaakola
224ae5770f Refs: MW-363 * enabling binlog file copying even for binlog files with basename "0" * mtr suite uses binlog files with names: 0.000001, 0.000002.. and so on 2017-08-14 14:41:05 +03:00