Commit graph

680 commits

Author SHA1 Message Date
Marko Mäkelä
e3d692aa09 Merge 10.2 into 10.3 2020-10-22 08:26:28 +03:00
Daniel Black
ff8ffef3e1 MDEV-23201: mysql_upgrade order mysql.user for 5.7 cross-upgrade
As MariaDB tables are expected in a fixed order, the cross upgrade
previously placed roles, default_role and max_statement_time after
now unused mysql-5.7 columns.

Test:

$ cp -a /usr/local/mysql-5.7.31/data/ /tmp/m57data
$ sql/mysqld --no-defaults --skip-networking --datadir=/tmp/m57data  --socket=/tmp/${PWD##*/}.sock   --verbose --lc-messages-dir=$PWD/sql/share
2020-10-02 11:02:05 140135193212864 [Note] sql/mysqld (mysqld 10.2.34-MariaDB) starting as process 1457667 ...
2020-10-02 11:02:05 140135193212864 [Note] InnoDB: Mutexes and rw_locks use GCC atomic built

$ client/mysql_upgrade --no-defaults -u root -pbob  -S /tmp/build-mariadb-server-10.2.sock
MySQL upgrade detected
Phase 1/7: Checking and upgrading mysql database
Processing databases
mysql
mysql.columns_priv                                 OK
mysql.db                                           OK
mysql.engine_cost                                  OK
mysql.event                                        OK
mysql.func                                         OK
mysql.gtid_executed                                OK
mysql.help_category                                OK
mysql.help_keyword                                 OK
mysql.help_relation                                OK
mysql.help_topic                                   OK
mysql.innodb_index_stats                           OK
mysql.innodb_table_stats                           OK
mysql.ndb_binlog_index                             OK
mysql.plugin                                       OK
mysql.proc                                         OK
mysql.procs_priv                                   OK
mysql.proxies_priv                                 OK
mysql.server_cost                                  OK
mysql.servers                                      OK
mysql.slave_master_info                            OK
mysql.slave_relay_log_info                         OK
mysql.slave_worker_info                            OK
mysql.tables_priv                                  OK
mysql.time_zone                                    OK
mysql.time_zone_leap_second                        OK
mysql.time_zone_name                               OK
mysql.time_zone_transition                         OK
mysql.time_zone_transition_type                    OK
mysql.user                                         OK
Upgrading from a version before MariaDB-10.1
Phase 2/7: Installing used storage engines

Result:

| user  | CREATE TABLE `user` (
  `Host` char(60) COLLATE utf8_bin NOT NULL DEFAULT '',
  `User` char(80) COLLATE utf8_bin NOT NULL DEFAULT '',
  `Password` char(41) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL DEFAULT '',
  `Select_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Insert_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Update_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Delete_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Drop_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Reload_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Shutdown_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Process_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `File_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Grant_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `References_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Index_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Alter_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Show_db_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Super_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_tmp_table_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Lock_tables_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Execute_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Repl_slave_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Repl_client_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_view_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Show_view_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_routine_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Alter_routine_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_user_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Event_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Trigger_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_tablespace_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `ssl_type` enum('','ANY','X509','SPECIFIED') CHARACTER SET utf8 NOT NULL DEFAULT '',
  `ssl_cipher` blob NOT NULL,
  `x509_issuer` blob NOT NULL,
  `x509_subject` blob NOT NULL,
  `max_questions` int(11) unsigned NOT NULL DEFAULT 0,
  `max_updates` int(11) unsigned NOT NULL DEFAULT 0,
  `max_connections` int(11) unsigned NOT NULL DEFAULT 0,
  `max_user_connections` int(11) NOT NULL DEFAULT 0,
  `plugin` char(64) CHARACTER SET latin1 NOT NULL DEFAULT '',
  `authentication_string` text COLLATE utf8_bin NOT NULL,
  `password_expired` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `is_role` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `default_role` char(80) COLLATE utf8_bin NOT NULL DEFAULT '',
  `max_statement_time` decimal(12,6) NOT NULL DEFAULT 0.000000,
  `password_last_changed` timestamp NULL DEFAULT NULL,
  `password_lifetime` smallint(5) unsigned DEFAULT NULL,
  `account_locked` enum('N','Y') COLLATE utf8_bin NOT NULL DEFAULT 'N',
  PRIMARY KEY (`Host`,`User`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='Users and global privileges' |

MariaDB [(none)]>  CREATE ROLE `aRole`;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> SET ROLE `aRole`;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> SET ROLE `aRole`;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> SELECT `User`, `is_role` FROM `mysql`.`user`;
+---------------+---------+
| User          | is_role |
+---------------+---------+
| root          | N       |
| mysql.session | N       |
| mysql.sys     | N       |
| dan           | N       |
| aRole         | Y       |
+---------------+---------+
5 rows in set (0.00 sec)

Reviewer: Anel Husakovic
2020-10-15 20:11:29 +11:00
Marko Mäkelä
50a11f396a Merge 10.4 into 10.5 2020-08-01 14:42:51 +03:00
mkaruza
6b8b7b1e8c MDEV-21905: Galera test galera_var_notify_cmd causes hang
Fixed wsrep_notify.sh script so it only reports status changes on
'joined', 'synced', 'donor'.
2020-07-24 12:05:39 +03:00
Monty
dbcd3384e0 MDEV-7947 strcmp() takes 0.37% in OLTP RO
This patch ensures that all identical character sets shares the same
cs->csname.
This allows us to replace strcmp() in my_charset_same() with comparisons
of pointers. This fixes a long standing performance issue that could cause
as strcmp() for every item sent trough the protocol class to the end user.

One consequence of this patch is that we don't allow one to add a character
definition in the Index.xml file that changes the csname of an existing
character set. This is by design as changing character set names of existing
ones is extremely dangerous, especially as some storage engines just records
character set numbers.

As we now have a hash over character set's csname, we can in the future
use that for faster access to a specific character set. This could be done
by changing the hash to non unique and use the hash to find the next
character set with same csname.
2020-07-23 10:54:33 +03:00
Marko Mäkelä
1813d92d0c Merge 10.4 into 10.5 2020-07-02 09:41:44 +03:00
Marko Mäkelä
f347b3e0e6 Merge 10.3 into 10.4 2020-07-02 07:39:33 +03:00
Marko Mäkelä
1df1a63924 Merge 10.2 into 10.3 2020-07-02 06:17:51 +03:00
Marko Mäkelä
ea2bc974dc Merge 10.1 into 10.2 2020-07-01 12:03:55 +03:00
Daniel Black
37c88445e3 mtr: use env for perl
On FreeBSD, perl isn't in /usr/bin, its in /usr/local/bin or
elsewhere in the path.

Like storage/{maria/unittest/,}ma_test_* , we use /usr/bin/env to
find perl and run it.
2020-06-23 03:24:46 +02:00
Elena Stepanova
f67522ede6 MDEV-22249 Upgrade testing between major versions in MTR 2020-06-01 00:45:24 +03:00
Marko Mäkelä
4f4fccecb2 Fix perfschema.statement_program_concurrency 2020-03-11 08:29:48 +02:00
Alexander Barkov
6f65931f88 MDEV-19906 Port show_old_temporals from MySQL 5.6
Old temporal data types (created with a pre-10.0 version of MariaDB)
are now displayed with a /* mariadb-5.3 */ comment in:

- SHOW CREATE TABLE
- DESCRIBE
- INFORMATION_SCHEMA.COLUMNS.COLUMN_TYPE

For example:

CREATE TABLE `t1` (
  `t0` datetime /* mariadb-5.3 */ DEFAULT NULL,
  `t6` datetime(6) /* mariadb-5.3 */ DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1

Note, new temporal data types are displayed without a format comment.
2020-01-17 07:16:36 +04:00
Alexander Barkov
2c7b6214e7 A cleanup for MDEV-17088 Provide tools to encode/decode mysql-encoded file system names
- Load and convert the entire input file at once,
  rather than reading string-by-string using fgets().
  This change makes it possible to convert from UCS2, UTF16, UTF32 data.
- Adding the --delimiter command, to treat the specified
  characters as delimiters rather than data to convert.
  Useful in combination with `-f filename` or `-t filename`.
  The delimiter characters are not converted,
  they are copied from the input to the output as is.
- Adding diagnostics with line number and position if:
  * an illegal input byte sequence was found
  * a character cannot be converted to the target character set
2019-12-05 09:45:28 +04:00
Marko Mäkelä
d04f2de80a Merge 10.4 into 10.5 2019-10-11 08:41:36 +03:00
Marko Mäkelä
c11e5cdd12 Merge 10.3 into 10.4 2019-10-10 11:19:25 +03:00
Marko Mäkelä
892378fb9d Merge 10.2 into 10.3 2019-10-09 13:25:11 +03:00
Sachin Setiya
27664ef29d MDEV-20574 Position of events reported by mysqlbinlog is wrong with encrypted binlogs, SHOW BINLOG EVENTS reports the correct one.
Analysis

Mysqlbinlog output for encrypted binary log
#Q> insert into tab1 values (3,'row 003')
#190912 17:36:35 server id 10221  end_log_pos 980 CRC32 0x53bcb3d3  Table_map: `test`.`tab1` mapped to number 19
# at 940
#190912 17:36:35 server id 10221  end_log_pos 1026 CRC32 0xf2ae5136     Write_rows: table id 19 flags: STMT_END_F

Here we can see Table_map_log_event ends at 980 but Next event starts at 940.
And the reason for that is we do not send START_ENCRYPTION_EVENT to the slave

Solution:-
Send Start_encryption_log_event as Ignorable_log_event to slave(mysqlbinlog),
So that mysqlbinlog can update its log_pos.
Since Slave can request multiple FORMAT_DESCRIPTION_EVENT while master does not
have so We only update slave master pos when master actually have the
FORMAT_DESCRIPTION_EVENT. Similar logic should be applied for START_ENCRYPTION_EVENT.

Also added the test case when new server reads the data from old server which
does not send START_ENCRYPTION_EVENT to slave.

Master Slave Upgrade Scenario.
When Slave is updated first, Slave will have extra logic of handling
START_ENCRYPTION_EVENT But master willnot be sending START_ENCRYPTION_EVENT.
So there will be no issue.
When Master is updated first, It will send  START_ENCRYPTION_EVENT to
slave , But slave will ignore this event in queue_event.
2019-10-08 14:35:34 +05:30
Marko Mäkelä
4081b7b27a Merge 10.4 into 10.5 2019-09-06 17:16:40 +03:00
Sergei Golubchik
244f0e6dd8 Merge branch '10.3' into 10.4 2019-09-06 11:53:10 +02:00
Alexander Barkov
7e08ac0b41 Merge 10.2 (up to commit ef00ac4c86) into 10.3 2019-09-04 10:19:58 +04:00
Alexander Barkov
ef00ac4c86 Part2: MDEV-18156 Assertion 0' failed or btr_validate_index(index, 0, false)' in row_upd_sec_index_entry or error code 126: Index is corrupted upon DELETE with PAD_CHAR_TO_FULL_LENGTH
This patch allows the server to open old tables that have
"bad" generated columns (i.e. indexed virtual generated columns,
persistent generated columns) that depend on sql_mode,
for general things like SELECT, INSERT, DROP, etc.
Warning are issued in such cases.

Only these commands are now disallowed and return an error:
- CREATE TABLE introducing a "bad" generated column
- ALTER TABLE introducing a "bad" generated column
- CREATE INDEX introdicing a "bad" generated column
  (i.e. adding an index on a virtual generated column
   that depends on sql_mode).

Note, these commands are allowed:
- ALTER TABLE removing a "bad" generate column
- ALTER TABLE removing an index from a "bad" virtual generated column
- DROP INDEX removing an index from a "bad" virtual generated column
but only if the table does not have any "bad" columns as a result.
2019-09-03 09:51:35 +04:00
Marko Mäkelä
624dd71b94 Merge 10.4 into 10.5 2019-08-13 18:57:00 +03:00
Marko Mäkelä
e9c1701e11 Merge 10.3 into 10.4 2019-07-25 18:42:06 +03:00
Pali
5cc2096f93 Switch Perl DBI scripts from DBD::mysql to DBD::MariaDB driver
Perl DBD::MariaDB driver is available CPAN and is already used in
production environment.
2019-07-19 13:39:20 +03:00
Eugene Kosov
0f83c8878d Merge 10.2 into 10.3 2019-07-16 18:39:21 +03:00
Jan Lindström
24aa723a28 Update Galera failing test list and record correct results
for passing ones.

Changes to be committed:
	new file:   mysql-test/std_data/galera-cert.pem
	new file:   mysql-test/std_data/galera-key.pem
	new file:   mysql-test/std_data/galera-upgrade-ca-cert.pem
	new file:   mysql-test/std_data/galera-upgrade-server-cert.pem
	new file:   mysql-test/std_data/galera-upgrade-server-key.pem
	modified:   mysql-test/suite/galera/disabled.def
	modified:   mysql-test/suite/galera/r/MW-416.result
	modified:   mysql-test/suite/galera/r/MW-44.result
	modified:   mysql-test/suite/galera/r/galera_sst_mysqldump_with_key,debug.rdiff
	modified:   mysql-test/suite/galera/r/galera_sst_mysqldump_with_key.result
	modified:   mysql-test/suite/galera/t/MW-416.test
	modified:   mysql-test/suite/galera/t/galera_kill_applier.test
2019-07-04 19:42:30 +03:00
Oleksandr Byelkin
f859789e7d Merge branch '10.4' into 10.5 2019-06-03 13:24:41 +02:00
Georg Richter
92df31dfbf Added new file client-certkey.pem for testing CONC-386:
client-certkey.pem contains both certificate and corresponding
private key.
2019-06-02 13:12:39 +02:00
Marko Mäkelä
826f9d4f7e Merge 10.4 into 10.5 2019-05-23 10:32:21 +03:00
Monty
ab38b7511b MDEV-17841 S3 storage engine
A read-only storage engine that stores it's data in (aws) S3

To store data in S3 one could use ALTER TABLE:
ALTER TABLE table_name ENGINE=S3

libmarias3 integration done by Sergei Golubchik
libmarias3 created by Andrew Hutchings
2019-05-23 02:28:23 +03:00
Oleksandr Byelkin
c07325f932 Merge branch '10.3' into 10.4 2019-05-19 20:55:37 +02:00
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Oleksandr Byelkin
c51f85f882 Merge branch '10.2' into 10.3 2019-05-12 17:20:23 +02:00
Vicențiu Ciorbaru
cb248f8806 Merge branch '5.5' into 10.1 2019-05-11 22:19:05 +03:00
Vicențiu Ciorbaru
5543b75550 Update FSF Address
* Update wrong zip-code
2019-05-11 21:29:06 +03:00
Marko Mäkelä
b132b8895e Merge 10.3 into 10.4 2019-05-05 10:23:14 +03:00
Oleksandr Byelkin
8cbb14ef5d Merge branch '10.1' into 10.2 2019-05-04 17:04:55 +02:00
Vladislav Vaintroub
e116f11f0a MDEV-18131 MariaDB does not verify IP addresses from subject alternative
names

Added a call to X509_check_ip_asc() in case server_hostname represents
an IP address.
2019-04-28 12:49:59 +02:00
Marko Mäkelä
4d59f45260 Merge 10.2 into 10.3 2019-04-27 20:41:31 +03:00
Alexander Barkov
5cfc7799a3 MDEV-16518 MYSQL57_GENERATED_FIELD: The code in TABLE_SHARE::init_from_binary_frm_image() is not safe 2019-04-26 14:01:21 +04:00
Marko Mäkelä
a2fc36989e Merge 10.2 into 10.3 2019-03-04 17:01:00 +02:00
Sergei Golubchik
8d47d9ed88 SSL test fixes
* fix CRL tests to work
* regenerate certificates to be at least 2048 bit
  (fixes buster and rhel8 in buildbot)
* update generate-ssl-cert.sh to generate crl files
* make all SSL tests to use certificates generated
  in generate-ssl-cert.sh, remove unused certificates

Backport from 10.4 9c60535f86
2019-03-01 12:41:05 -05:00
Alexander Barkov
80c3fd184d Backporting MDEV-15597 Add class Load_data_outvar and avoid using Item::STRING_ITEM for Item_user_var_as_out_param detection
This is a part of "MDEV-18045 Backporting the MDEV-15497 changes to 10.2 branch"
2019-02-23 17:43:59 +04:00
Alexander Barkov
8036ad541e Backporting MDEV-15497 Wrong empty value in a GEOMETRY column on LOAD DATA
This is a part of "MDEV-18045 Backporting the MDEV-15497 changes to 10.2 branch"
2019-02-23 17:43:59 +04:00
Alexander Barkov
0ef8848526 Backporting MDEV-14628 Wrong autoinc value assigned by LOAD XML in the NO_AUTO_VALUE_ON_ZERO mode
This is a part of "MDEV-18045 Backporting the MDEV-15497 changes to 10.2 branch"
2019-02-23 17:43:59 +04:00
Daniele Sciascia
9b8fc089bd MDEV-18198 Fix MTR test galera_sr.galera_sr_table_contents
* Created new binlog-header file
* Fixed warning on SELECT INTO DUMPFILE
* Re-recorded the test result
2019-02-18 08:25:57 +02:00
Sergei Golubchik
9c60535f86 SSL test fixes
* fix CRL tests to work
* regenerate certificates to be at least 2048 bit
  (fixes buster and rhel8 in buildbot)
* update generate-ssl-cert.sh to generate crl files
* make all SSL tests to use certificates generated
  in generate-ssl-cert.sh, remove unused certificates
2019-01-28 14:41:39 +01:00
Marko Mäkelä
734510a44d Merge 10.3 into 10.4 2019-01-06 17:43:02 +02:00
Sergei Golubchik
6bb11efa4a Merge branch '10.2' into 10.3 2019-01-03 13:09:41 +01:00
Sergei Golubchik
fca44b7c1f MDEV-17909 Problem by MariaDB Update 10.1.32 -> 10.2.19 (Incorrect information in file: .frm)
use frm_version, not mysql_version when parsing frm

In particular, virtual columns are stored according to
frm_version. And CHECK TABLE will overwrite mysql_version
to the current server version, so it cannot correctly
describe frm format.
2018-12-20 08:06:55 +01:00
Vladislav Vaintroub
19d3d3e861 MDEV-16266 - New command FLUSH SSL to reload server's SSL certificate(private key,CRL,etc) 2018-12-12 22:51:20 +01:00
Alexander Barkov
1c37ac84ef MDEV-8894 Inserting fractional seconds into MySQL 5.6 master breaks consistency on MariaDB 10 slave
The previous patch 269da4bf19 was
actually for MDEV-8894 (not for MDEV-5377). It was erroneously
pushed with a wrong title.

This patch is a small cleanup for MDEV-8894.
CREATE TABLE is now not a part of binary logs recorded with MySQL,
only INSERT statements are. This will allow to reuse the same binary
logs in combinations with different CREATE TABLE statements,
to tests different data types.
2018-12-04 18:11:45 +04:00
Alexander Barkov
269da4bf19 MDEV-5377 Row-based replication of MariaDB temporal data types with FSP>0 into a different column type 2018-12-04 15:44:14 +04:00
Sergei Golubchik
36e59752e7 Merge branch '10.2' into 10.3 2018-06-30 16:39:20 +02:00
Sergei Golubchik
b942aa34c1 Merge branch '10.1' into 10.2 2018-06-21 23:47:39 +02:00
Alexander Barkov
f5b128dfad Merge remote-tracking branch 'origin/10.0' into 10.1 2018-06-19 14:04:53 +04:00
Alexander Barkov
c450f7d8d5 Merge remote-tracking branch 'origin/5.5' into 10.0 2018-06-19 14:03:41 +04:00
Alexander Barkov
15b92915ed MDEV-15834 The code in TABLE_SHARE::init_from_binary_frm_image() is not safe 2018-06-19 13:02:02 +04:00
Vicențiu Ciorbaru
65eefcdc60 Merge remote-tracking branch '10.2' into 10.3 2018-04-12 12:41:19 +03:00
Vicențiu Ciorbaru
45e6d0aebf Merge branch '10.1' into 10.2 2018-04-10 17:43:18 +03:00
Daniele Sciascia
eeb684221d MDEV-13549 Fix and re-enable MTR test galera.galera_gra_log
Test galera checks that a `GRA_x_x.log` file is created whenever
wsrep applier fails to apply some replication event. The file
contains the corresponding binlog event that failed to apply.
The test creates a new file by concatenating a pre-recorded
file containing the binlog header (see `std-data/binlog-header.log`)
and the `GRA_x_x.log` file. The test then checks that the resulting
file, containing the binlog header and the event that failed to
apply, is correctly read by `mysqlbinlog` program.
The test fails in MariaDB because the GRA_x_x.log file created
by MariaDB already contains the binlog header (see MDEV-7867).
This patch fixes/simplifies test `galera.galera_gra_log` so that
it doesn't concatenate `std-data/binlog-header.log` with the
`GRA_x_x.log` file. File `std-data/binlog-header.log` is deleted
altoghether, because not used by any other test.
2018-04-03 16:30:58 +02:00
Alexander Barkov
e263530bea MDEV-15597 Add class Load_data_outvar and avoid using Item::STRING_ITEM for Item_user_var_as_out_param detection 2018-03-20 13:02:44 +04:00
Alexander Barkov
6ec3de5d2d MDEV-15497 Wrong empty value in a GEOMETRY column on LOAD DATA
- Adding a new virtual method Field::load_data_set_no_data().
- Overriding Field_timestamp::load_data_set_no_data() and moving
  the TIMESTAMP specific code there.
- Overriding Field_geom::load_data_set_no_data() and implementing
  GEOMETRY specific behavior, to prevent writing empty strings
  when the loaded file ends unexpectedly. This fixes the bug.
- Adding a new test gis-loaddaata.test.
- The test in loaddata.test for CHAR was added simply to record behavior.
  The CHAR data type did not change its behaviour (only GEOMRYRY did).
- Additionally, moving duplicate code into a new method
  Field::load_data_set_value() and reusing it in three places.
2018-03-07 19:55:12 +04:00
Alexey Botchkov
b4a2baffa8 MDEV-11084 Select statement with partition selection against MyISAM table opens all partitions.
Now we don't open partitions if it was explicitly cpecified.
        ha_partition::m_opened_partition bitmap added to track
        partitions that were actually opened.
2018-01-29 11:01:14 +04:00
Alexander Barkov
a53e087ea9 MDEV-14628 Wrong autoinc value assigned by LOAD XML in the NO_AUTO_VALUE_ON_ZERO mode
The fixes for these bugs:

Bug#27586 Wrong autoinc value assigned by LOAD DATA in the NO_AUTO_VALUE_ON_ZERO mode
Bug#22372 Disable spatial key, load data, enable spatial key, crashes table

fixed only LOAD DATA INFILE, but did not fix LOAD XML INFILE.

This patch does for LOAD XML FILE what patches for Bug#27586 and Bug#22372
earlier did for LOAD DATA INFILE.

1. Fixing the auto_increment problem:
   a. table->auto_increment_field_not_null is not set to TRUE
      anymore when a column does not have a corresponding XML tag.
   b. Adding "table->auto_increment_field_not_null= false"
      in the end of read_xml_field().
   These two changes resemble the patch for Bug#27586.

2. Fixing the GEOMETRY problem:
   The result for "reset()" was not tested for errors in read_xml_field(),
   which made it possible for empty string to sneak into a "GEOMETRY NOT NULL"
   column when this column does not have a corresponding XML tag with data.
   After this patch the result of reset() is tested and and an error is
   returned in such cases.
   This change effectively resembles the patch for Bug#22372

3. Spliting the code into a new virtual method Field::load_data_set_null().

   Rationale:
   a. To avoid duplicate code in read_sep_field() and read_xml_field():
      Changes #1 and #2 made the code handling NULL values for Field
      exactly the same in read_sep_field() and read_xml_field().

  b. To avoid tests for field_type(), which is not friendly to
     upcoming data type plugins.
     This change makes it possible for data type plugins
     to implement their own special way for handling NULL values in LOAD DATA
     by overriding Field_xxx::load_data_set_null(),
     like Field_geom and Field_timestamp do.
2017-12-13 13:22:45 +04:00
Sergei Golubchik
cb1e76e4de Merge branch '10.1' into 10.2 2017-08-17 11:38:34 +02:00
Sergei Golubchik
8e8d42ddf0 Merge branch '10.0' into 10.1 2017-08-08 10:18:43 +02:00
Vicențiu Ciorbaru
786ad0a158 Merge remote-tracking branch 'origin/5.5' into 10.0 2017-07-25 00:41:54 +03:00
Sergei Golubchik
f6bcdb9e3c test case for loadxml and spaces 2017-07-18 14:59:33 +02:00
Sergei Golubchik
47687eef41 MDEV-12936 upgrade to 10.2.6 failed upon tables with virtual columns
when opening 10.1- table that has virtual columns:

1. don't error out if it has vcols over autoinc columns.
   just issue a warning.
2. set vcol type properly
3. in innodb: use table->s->stored_fields instead of table->s->fields,
   because that's what was stored in innodb data dictionary
2017-07-05 17:15:58 +02:00
Sergei Golubchik
fbdf18f86e MDEV-12696 Crash with LOAD XML and non-updatable VIEW column
extend the fix from 5.5 (in read_sep_field()) to apply to read_xml_field()
2017-05-09 00:28:43 +02:00
Marko Mäkelä
14c6f00a9f Merge 10.1 into 10.2
Also, include fixes by Vladislav Vaintroub to the
aws_key_management plugin. The AWS C++ SDK specifically depends on
OPENSSL_LIBRARIES, not generic SSL_LIBRARIES (such as YaSSL).
2017-05-06 14:36:46 +03:00
Marko Mäkelä
f9cc391863 Merge 10.1 into 10.2
This only merges MDEV-12253, adapting it to MDEV-12602 which is already
present in 10.2 but not yet in the 10.1 revision that is being merged.

TODO: Error handling in crash recovery needs to be improved.
If a page cannot be decrypted (or read), we should cleanly abort
the startup. If innodb_force_recovery is specified, we should
ignore the problematic page and apply redo log to other pages.
Currently, the test encryption.innodb-redo-badkey randomly fails
like this (the last messages are from cmake -DWITH_ASAN):

2017-05-05 10:19:40 140037071685504 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1635994
2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Missing MLOG_FILE_NAME or MLOG_FILE_DELETE before MLOG_CHECKPOINT for tablespace 1
2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[2201] with error Data structure corruption
2017-05-05 10:19:41 140037071685504 [Note] InnoDB: Starting shutdown...
i=================================================================
==5226==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x612000018588 in thread T0
    #0 0x736750 in operator delete(void*) (/mariadb/server/build/sql/mysqld+0x736750)
    #1 0x1e4833f in LatchCounter::~LatchCounter() /mariadb/server/storage/innobase/include/sync0types.h:599:4
    #2 0x1e480b8 in LatchMeta<LatchCounter>::~LatchMeta() /mariadb/server/storage/innobase/include/sync0types.h:786:17
    #3 0x1e35509 in sync_latch_meta_destroy() /mariadb/server/storage/innobase/sync/sync0debug.cc:1622:3
    #4 0x1e35314 in sync_check_close() /mariadb/server/storage/innobase/sync/sync0debug.cc:1839:2
    #5 0x1dfdc18 in innodb_shutdown() /mariadb/server/storage/innobase/srv/srv0start.cc:2888:2
    #6 0x197e5e6 in innobase_init(void*) /mariadb/server/storage/innobase/handler/ha_innodb.cc:4475:3
2017-05-05 10:38:53 +03:00
Alexander Barkov
db0917f68f MDEV-12696 Crash with LOAD XML and non-updatable VIEW column 2017-05-05 11:05:55 +04:00
Sergei Golubchik
1b27c25473 MDEV-10594 SSL hostname verification fails for SubjectAltNames
use X509_check_host for OpenSSL 1.0.2+
This adds:
* support for subjectAltNames
* wildcards
* sub-domain matching
2017-04-27 19:12:44 +02:00
Sergei Golubchik
0636637e37 regenerate SSL certificates again
and make sure that private ca key is not deleted at the end of
the procedure, so that we could generate additional certificates
any time without regenerating everything
2017-04-27 19:12:44 +02:00
Jan Lindström
765a43605a MDEV-12253: Buffer pool blocks are accessed after they have been freed
Problem was that bpage was referenced after it was already freed
from LRU. Fixed by adding a new variable encrypted that is
passed down to buf_page_check_corrupt() and used in
buf_page_get_gen() to stop processing page read.

This patch should also address following test failures and
bugs:

MDEV-12419: IMPORT should not look up tablespace in
PageConverter::validate(). This is now removed.

MDEV-10099: encryption.innodb_onlinealter_encryption fails
sporadically in buildbot

MDEV-11420: encryption.innodb_encryption-page-compression
failed in buildbot

MDEV-11222: encryption.encrypt_and_grep failed in buildbot on P8

Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing
and replaced these with dict_table_t::file_unreadable. Table
ibd file is missing if fil_get_space(space_id) returns NULL
and encrypted if not. Removed dict_table_t::is_corrupted field.

Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(),
buf_page_decrypt_after_read(), buf_page_encrypt_before_write(),
buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats().

Added test cases when enrypted page could be read while doing
redo log crash recovery. Also added test case for row compressed
blobs.

btr_cur_open_at_index_side_func(),
btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is
NULL.

buf_page_get_zip(): Issue error if page read fails.

buf_page_get_gen(): Use dberr_t for error detection and
do not reference bpage after we hare freed it.

buf_mark_space_corrupt(): remove bpage from LRU also when
it is encrypted.

buf_page_check_corrupt(): @return DB_SUCCESS if page has
been read and is not corrupted,
DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match. In read
case only DB_SUCCESS is possible.

buf_page_io_complete(): use dberr_t for error handling.

buf_flush_write_block_low(),
buf_read_ahead_random(),
buf_read_page_async(),
buf_read_ahead_linear(),
buf_read_ibuf_merge_pages(),
buf_read_recv_pages(),
fil_aio_wait():
        Issue error if page read fails.

btr_pcur_move_to_next_page(): Do not reference page if it is
NULL.

Introduced dict_table_t::is_readable() and dict_index_t::is_readable()
that will return true if tablespace exists and pages read from
tablespace are not corrupted or page decryption failed.
Removed buf_page_t::key_version. After page decryption the
key version is not removed from page frame. For unencrypted
pages, old key_version is removed at buf_page_encrypt_before_write()

dict_stats_update_transient_for_index(),
dict_stats_update_transient()
        Do not continue if table decryption failed or table
        is corrupted.

dict0stats.cc: Introduced a dict_stats_report_error function
to avoid code duplication.

fil_parse_write_crypt_data():
        Check that key read from redo log entry is found from
        encryption plugin and if it is not, refuse to start.

PageConverter::validate(): Removed access to fil_space_t as
tablespace is not available during import.

Fixed error code on innodb.innodb test.

Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown
to innodb-bad-key-change2.  Removed innodb-bad-key-change5 test.
Decreased unnecessary complexity on some long lasting tests.

Removed fil_inc_pending_ops(), fil_decr_pending_ops(),
fil_get_first_space(), fil_get_next_space(),
fil_get_first_space_safe(), fil_get_next_space_safe()
functions.

fil_space_verify_crypt_checksum(): Fixed bug found using ASAN
where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly
accessed from row compressed tables. Fixed out of page frame
bug for row compressed tables in
fil_space_verify_crypt_checksum() found using ASAN. Incorrect
function was called for compressed table.

Added new tests for discard, rename table and drop (we should allow them
even when page decryption fails). Alter table rename is not allowed.
Added test for restart with innodb-force-recovery=1 when page read on
redo-recovery cant be decrypted. Added test for corrupted table where
both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted.

Adjusted the test case innodb_bug14147491 so that it does not anymore
expect crash. Instead table is just mostly not usable.

fil0fil.h: fil_space_acquire_low is not visible function
and fil_space_acquire and fil_space_acquire_silent are
inline functions. FilSpace class uses fil_space_acquire_low
directly.

recv_apply_hashed_log_recs() does not return anything.
2017-04-26 15:19:16 +03:00
Sergei Golubchik
2195bb4e41 Merge branch '10.1' into 10.2 2017-02-10 17:01:45 +01:00
Vicențiu Ciorbaru
8e15768731 Merge branch '10.0' into 10.1 2017-01-16 03:18:14 +02:00
Vicențiu Ciorbaru
d00d46f4ce Merge remote-tracking branch 'origin/5.5' into 10.0 2017-01-10 12:34:51 +02:00
Alexander Barkov
3e63fde52e Adding LOAD DATA tests for MDEV-11079 and MDEV-11631
c3cf7f47f0 reverted the patch
for BUG#24487120. After merging the reverting patch from MySQL
to MariaDB the problems described in MDEV-11079 and MDEV-11631 disappeared.
Adding test cases only.
2017-01-09 14:19:02 +04:00
vicentiu
e9aed131ea Merge remote-tracking branch 'origin/5.5' into 10.0 2017-01-06 17:09:59 +02:00
Sergei Golubchik
4a5d25c338 Merge branch '10.1' into 10.2 2016-12-29 13:23:18 +01:00
Sergei Golubchik
9fefe97336 Merge branch 'mysql/5.5' into 5.5 2016-12-22 12:49:06 +01:00
Sergei Golubchik
2f20d297f8 Merge branch '10.0' into 10.1 2016-12-11 09:53:42 +01:00
Alexander Barkov
dd0ff30278 MDEV-11343 LOAD DATA INFILE fails to load data with an escape character followed by a multi-byte character
Partially backporting MDEV-9874 from 10.2 to 10.0

READ_INFO::read_field() raised the ER_INVALID_CHARACTER_STRING error
when reading an escape character followed by a multi-byte character.

Raising wellformedness errors in READ_INFO::read_field() was wrong,
because the main goal of READ_INFO::read_field() is to *unescape* the
data which was presumably escaped using mysql_real_escape_string(),
using the same character set with the one specified in
"LOAD DATA INFILE ... CHARACTER SET ..." (or assumed by default).

During LOAD DATA, multi-byte characters are not always scanned as a single
entity! In case of escaped data, parts of a multi-byte character can be
scanned on different loop iterations. So the old code erroneously tested
welformedness in the middle of a multi-byte character.

Moreover, the data after unescaping can go into a BLOB field, not a text field.
Wellformedness tests are meaningless in this case.

Ater this patch, wellformedness is only checked later, during
Field::store(str,length,cs) time. The loop that scans bytes only
makes sure to revert the changes made by mysql_real_escape_string().

Note, in some cases users can supply data which did not really go through
mysql_real_escape_string() and was escaped by some other means,
or was not escaped at all. The file reported in this MDEV contains
the string "\ä", which is an example of such improperly escaped data, as
- either there should be two backslashes:   "\\ä"
- or there should be no backslashes at all: "ä"
mysql_real_escape_string() could not generate "\ä".
2016-11-29 06:51:12 +04:00
Alexander Barkov
0259b3cbbe MDEV-11255 LDML: allow defining 2-level UCA collations 2016-11-08 20:57:19 +04:00
Jan Lindström
885577fb10 MDEV-11004: Unable to start (Segfault or os error 2) when encryption key missing
Two problems:

(1) When pushing warning to sql-layer we need to check that thd != NULL
to avoid NULL-pointer reference.

(2) At tablespace key rotation if used key_id is not found from
encryption plugin tablespace should not be rotated.
2016-10-29 10:09:06 +03:00
Thayumanavar S
c3cf7f47f0 BUG#24487120 - SLAVE'S SLAVE_SQL_RUNNING IS STOPPED DURING
LOAD DATA AT MASTER.

Revert "BUG#23080148 - BACKPORT BUG 14653594 AND BUG 20683959 TO"

This reverts commit 1d31f5b3090d129382b50b95512f2f79305715a1.
The commit causes replication incompatibility between minor revisions
and based on discussion with Srinivasarao, the patch is reverted.
2016-10-28 14:45:03 +02:00
Alexander Barkov
0f8a1a314d MDEV-10877 xxx_unicode_nopad_ci collations 2016-09-23 14:19:07 +04:00
Sergei Golubchik
06b7fce9f2 Merge branch '10.1' into 10.2 2016-09-09 08:33:08 +02:00
Alexander Barkov
ee19806b8e MDEV-9711 NO PAD collations
Based on the patch from Daniil Medvedev (a Google Summer of Code task)
2016-09-06 12:50:02 +04:00
Alexander Barkov
e4f6fd5e12 MDEV-10743 LDML: a new syntax to reuse sort order from another 8bit simple collation 2016-09-06 12:37:11 +04:00
Alexander Barkov
1ca595fbf7 LDML refactoring for "MDEV-9711 NO PAD collations"
- Moving detection of the MY_CS_CSSORT, MY_CS_PUREASCII, MY_CS_NONASCII
  flags of loadable collations from add_collation() in mysys.c
  to my_cset_init_8bit() and my_coll_init_simple() in ctype-simple.c.

- Adding tests that these flags are set properly for loadable collations

- Moving LDML test related *.xml files from mysql-test/std_data/
  to mysql-test/std_data/ldml/, as there will be more *.xml test files
2016-09-03 09:05:56 +04:00
Sergei Golubchik
6b1863b830 Merge branch '10.0' into 10.1 2016-08-25 12:40:09 +02:00
Sergei Golubchik
309c08c17c Merge branch '5.5' into 10.0 2016-08-10 19:19:05 +02:00
Sergei Golubchik
a350e53b61 Merge branch 'mysql/5.5' into 5.5
without a fix for Bug#12818255 (MDEV-6581)
2016-08-03 20:38:25 +02:00
Sergei Golubchik
932646b1ff Merge branch '10.1' into 10.2 2016-06-30 16:38:05 +02:00