This patch adds a way to override default collations
(or "character set collations") for desired character sets.
The SQL standard says:
> Each collation known in an SQL-environment is applicable to one
> or more character sets, and for each character set, one or more
> collations are applicable to it, one of which is associated with
> it as its character set collation.
In MariaDB, character set collations has been hard-coded so far,
e.g. utf8mb4_general_ci has been a hard-coded character set collation
for utf8mb4.
This patch allows to override (globally per server, or per session)
character set collations, so for example, uca1400_ai_ci can be set as a
character set collation for Unicode character sets
(instead of compiled xxx_general_ci).
The array of overridden character set collations is stored in a new
(session and global) system variable @@character_set_collations and
can be set as a comma separated list of charset=collation pairs, e.g.:
SET @@character_set_collations='utf8mb3=uca1400_ai_ci,utf8mb4=uca1400_ai_ci';
The variable is empty by default, which mean use the hard-coded
character set collations (e.g. utf8mb4_general_ci for utf8mb4).
The variable can also be set globally by passing to the server startup command
line, and/or in my.cnf.
The `--vertical` option does not appear to behave as documented and
research shows it never appears to have behaved as intended.
As this appears to be a completely unused feature, let's remove it.
Performs an upgrade of mariadb from an earlier version to the rpms built
in CI. Then checks whether log contains evidence of upgrade in the form
of "Needs upgrade" or "Table rebuild required". Designed to check minor
version upgrades which should not trigger rebuilds.
The test is written in bash script so it can be executed from other CI
systems.
$ test_upgrade.sh source_version target_version
$ test_upgrade.sh source_version # defaults to rpm/
$ test_upgrade.sh source_version --rpm-dir <directory>
Binaries must be created with performance schema enabled, or
mysql_upgrade complains about missing tables.
Upgrade testing is parallelized with a "matrix" of source versions.
Others can be introduced later.
This was partially designed to catch issues like that seen in
https://jira.mariadb.org/browse/MDEV-28727 where a minor version upgrade
(e.g. 10.4.8 -> 10.4.26) triggered a system table rebuild.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
Allow ALTER TABLE ... IMPORT TABLESPACE without creating the table
followed by discarding the tablespace.
That is, assuming we want to import table t1 to t2, instead of
CREATE TABLE t2 LIKE t1;
ALTER TABLE t2 DISCARD TABLESPACE;
FLUSH TABLES t1 FOR EXPORT;
--copy_file $MYSQLD_DATADIR/test/t1.cfg $MYSQLD_DATADIR/test/t2.cfg
--copy_file $MYSQLD_DATADIR/test/t1.ibd $MYSQLD_DATADIR/test/t2.ibd
UNLOCK TABLES;
ALTER TABLE t2 IMPORT TABLESPACE;
We can simply do
FLUSH TABLES t1 FOR EXPORT;
--copy_file $MYSQLD_DATADIR/test/t1.cfg $MYSQLD_DATADIR/test/t2.cfg
--copy_file $MYSQLD_DATADIR/test/t1.frm $MYSQLD_DATADIR/test/t2.frm
--copy_file $MYSQLD_DATADIR/test/t1.ibd $MYSQLD_DATADIR/test/t2.ibd
UNLOCK TABLES;
ALTER TABLE t2 IMPORT TABLESPACE;
We achieve this by creating a "stub" table in the second scenario
while opening the table, where t2 does not exist but needs to import
from t1. The "stub" table is similar to a table that is created but
then instructed to discard its tablespace.
We include tests with various row formats, encryption, with indexes
and auto-increment.
Kenyan Swahili is enabled for error messages as well as datetime.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
The error code is non-deterministic, presumably due to some race
condition from the SLEEP statement above. The correct error should be
12701 ER_SPIDER_REMOTE_SERVER_GONE_AWAY_NUM as it is the last failure.
Nevertheless, this contrived test is needed to cover the error
reporting when setting lock wait timeout, until we find a better test
case and/or fixing the non-deterministic error reporting (MDEV-31586)
log_t::write_checkpoint(), log_t::resize_start(): Invoke buf_flush_ahead()
with buf_pool.get_oldest_modification(0)+1 so that another checkpoint will
be invoked, to complete the log resizing.
Tested by: Oleg Smirnov (on Microsoft Windows, where this hung most often)
ha_innobase::delete_table(): Also on DROP SEQUENCE, do try to drop any
persistent statistics. They should really not be created for
SEQUENCE objects (which internally are 1-row no-rollback tables),
but that is how happened to always work.
i_s_innodb_buffer_page_get_info(): Correct a condition.
After crash recovery, there may be some buffer pool pages in FREED state,
containing garbage (invalid data page contents). Let us ignore such pages
in the INFORMATION_SCHEMA output.
The test innodb.innodb_defragment_fill_factor will be removed, because
the queries that it is invoking on information_schema.innodb_buffer_page
would start to fail. The defragmentation feature was removed in
commit 7ca89af6f8 in MariaDB Server 11.1.
Tested by: Matthias Leich
Allocate a temporary buffer instead of using the same buffer in some
cases, and add assertions to verify the buffers do not overlap. See [1]
for reasonsing.
[1] https://github.com/MariaDB/server/pull/2438#discussion_r1137403645
Signed-off-by: Trevor Gross <tmgross@umich.edu>
This patch ensures that all direct and indirect calls to
encryption_crypt provide a `dlen` value correctly initialized to the
destination buffer length, allowing encryption plugins to verify
available space. It also adds assertions to verify related invariants.
Signed-off-by: Trevor Gross <tmgross@umich.edu>
* removed universal suppression of everything "Event Schedule" everywhere
* added suppressions in tests as needed
* moved events test to the events suite
* renamed -master.opt -> .opt
* added standard test header
* verified in the test that the error, indeed, was written into the log
* removed useless suppressions
* removed ER_EVENTS_NO_ACL, replaced with ER_OPTION_PREVENTS_STATEMENT
* fixed error message to say exactly what option disabled event scheduler
instead of "this or that or that, you figure it out"
* also fixed old message for SET event_scheduler=
(it was also non-translatable)
* changed to use sql_print_error() when an error is not sent to the user
* removed duplicate hard-coded error message
btr_search_hash_table_validate(), btr_search_validate(): Add the
parameter THD for checking if the statement has been killed.
Any non-QUICK CHECK TABLE will validate the entire adaptive hash index
for all InnoDB tables, which may be extremely slow when running
multiple concurrent CHECK TABLE.
During the upgrade procedure on Windows mysqld.exe is started with
the named pipe connection protocol. mysqladmin.exe then pings the
server to check if is up and running. Command line looks like:
mysqladmin.exe --protocol=pipe --socket=mysql_upgrade_service_xxx ping
But the "socket" parameter resets the "protocol" which was previously
initialized with the "pipe" value, setting it to "socket".
As a result, connection cannot be established and the upgrade
procedure fails.
"socket" in Windows is used to pass the name of the pipe so resetting
the protocol is not valid in this case.
This commit fixes resetting of the "protocol" parameter with "socket"
parameter in the case when protocol has been previously initialized
to "pipe" value
Fake_select_lex->join was prepared at the unit execution stage so
the validation of fake_select_lex before the unit pushdown
was incomplete. That caused pushing down of statements having
an incorrect ORDER BY clause.
This commit moves preparation of the fake_select_lex->join to the unit
prepare() method, before initializing of the pushdown handler,
so incorrect clauses error out before being pushed down
During the upgrade procedure on Windows mysqld.exe is started with
the named pipe connection protocol. mysqladmin.exe then pings the
server to check if is up and running. Command line looks like:
mysqladmin.exe --protocol=pipe --socket=mysql_upgrade_service_xxx ping
But the "socket" parameter resets the "protocol" which was previously
initialized with the "pipe" value, setting it to "socket".
As a result, connection cannot be established and the upgrade
procedure fails.
"socket" in Windows is used to pass the name of the pipe so resetting
the protocol is not valid in this case.
This commit fixes resetting of the "protocol" parameter with "socket"
parameter in the case when protocol has been previously initialized
to "pipe" value
The fix is in replacing the waiting for the whole purge finishing
with the the waiting for only delete-marked records purging finishing.
Reviewed by: Marko Mäkelä
- InnoDB bulk insert operation aborts the server for redundant
table. InnoDB miscalculates the record size in temporary file
for the redundant table. CHAR in redundant row format table
always fixed length, but in temporary file, it is variable-length
for variable-length character sets.
The issue is caused by MDEV-30400 fix.
There are two cursors in btr_estimate_n_rows_in_range() - p1 and p2, but
both share the same mtr. Each cursor contains mtr savepoint for the
previously fetched block to release it then the current block is
fetched.
Before MDEV-30400 the block was released with
mtr_t::release_block_at_savepoint(), it just unfixed a block and
released its page patch. In MDEV-30400 it was replaced with
mtr_t::rollback_to_savepoint(), which does the same as the former
mtr_t::release_block_at_savepoint(ulint begin, ulint end) but also
erases the corresponding slots from mtr memo, what invalidates any
stored mtr's memo savepoints, greater or equal to "begin".
The idea of the fix is to get rid of savepoints at all in
btr_estimate_n_rows_in_range() and
btr_estimate_n_rows_in_range_on_level(). As
mtr_t::rollback_to_savepoint() erases elements from mtr_t::m_memo, we
know what element of mtr_t::m_memo can be deleted on the certain case,
so there is no need to store savepoints.
See also the following slides for details:
https://docs.google.com/presentation/d/1RFYBo7EUhM22ab3GOYctv3j_3yC0vHtBY9auObZec8U
Reviewed by: Marko Mäkelä
When setting the server lockwait timeout, spider should do some basic
checks first, like whether the remote server is still reachable. So
instead of directly calling spider_db_mbase::exec_query(), it should
call spider_db_query().
The reset of the lock wait timeout does not need to do such checks,
because they happen after the successfully setting the lock wait
timeout, implying the checks have been passed already.
recv_sys_t::parse(): For undo tablespace truncation mini-transactions,
remember the start_lsn instead of the end LSN. This is what we expect
after commit 461402a564 (MDEV-30479).
Allocate a temporary buffer instead of using the same buffer in some
cases, and add assertions to verify the buffers do not overlap. See [1]
for reasonsing.
[1] https://github.com/MariaDB/server/pull/2438#discussion_r1137403645
Signed-off-by: Trevor Gross <tmgross@umich.edu>
This patch ensures that all direct and indirect calls to
encryption_crypt provide a `dlen` value correctly initialized to the
destination buffer length, allowing encryption plugins to verify
available space. It also adds assertions to verify related invariants.
Signed-off-by: Trevor Gross <tmgross@umich.edu>
mysql.proc. The table is probably corrupted"
Analysis: When mysql_upgrade runs statements for upgrade, characterset is
converted to utf8mb4 because server starts with old_mode that interprets
utf8 to utf8mb4, but mysql.proc table has "utf8mb3" as hardcoded, so
it crashes with corrupted table.
Fix: Changed Table_check_intact::check() definition to allow both
utf8mb3 and utf8mb4 by checking prefix and changing the upgrade scripts
to explicitly use utf8mb3
MDEV-29253 Detect incompatible MySQL partition scheme and either convert
them or report to user and in error log.
This task is about converting in place MySQL 5.6 and 5.7 partition tables
to MariaDB as part of mariadb-upgrade.
- Update TABLE_SHARE::init_from_binary_frm_image() to be able to read
MySQL frm files with partitions.
- Create .par file, if it do not exists, on open of partitioned table.
Executing mariadb-upgrade will create all the missing .par files.
The MySQL .frm file will be changed to MariaDB format after next
ALTER TABLE.
Other changes:
- If we are using stored mysql_version to distingush between MySQL and
MariaDB .frm file information, do not upgrade mysql_version in the
.frm file as part of CHECK TABLE .. FOR UPGRADE as this would cause
problems next time we parse the .frm file.
- Moved view checks after privilege tables are fixed. This is to avoid
warnings about wrongly defined mysql.proc when checking views.
- Don't use stat tables before they have been fixed.
- Don't run mysql_fix_view() if 'FOR MYSQL' is used if the view is
already a MariaDB view.
- Added 'FOR UPGRADE' as an option for 'REPAIR VIEW' to be able to
detect if the REPAIR command comes from mariadb_upgrade. In this
case we get a warning, instead of an error, if a definer of a view
does not exists.
