Commit graph

27624 commits

Author SHA1 Message Date
Alexander Barkov
f582ea4d5b A cleanup for MDEV-12668 SRID is not preserved in UNION, VIEW, MIN, MAX
Fixing storage/mroonga according to MDEV-12668
2024-04-29 16:52:35 +04:00
Alexander Barkov
fd247cc21f MDEV-31340 Remove MY_COLLATION_HANDLER::strcasecmp()
This patch also fixes:
  MDEV-33050 Build-in schemas like oracle_schema are accent insensitive
  MDEV-33084 LASTVAL(t1) and LASTVAL(T1) do not work well with lower-case-table-names=0
  MDEV-33085 Tables T1 and t1 do not work well with ENGINE=CSV and lower-case-table-names=0
  MDEV-33086 SHOW OPEN TABLES IN DB1 -- is case insensitive with lower-case-table-names=0
  MDEV-33088 Cannot create triggers in the database `MYSQL`
  MDEV-33103 LOCK TABLE t1 AS t2 -- alias is not case sensitive with lower-case-table-names=0
  MDEV-33109 DROP DATABASE MYSQL -- does not drop SP with lower-case-table-names=0
  MDEV-33110 HANDLER commands are case insensitive with lower-case-table-names=0
  MDEV-33119 User is case insensitive in INFORMATION_SCHEMA.VIEWS
  MDEV-33120 System log table names are case insensitive with lower-cast-table-names=0

- Removing the virtual function strnncoll() from MY_COLLATION_HANDLER

- Adding a wrapper function CHARSET_INFO::streq(), to compare
  two strings for equality. For now it calls strnncoll() internally.
  In the future it will turn into a virtual function.

- Adding new accent sensitive case insensitive collations:
    - utf8mb4_general1400_as_ci
    - utf8mb3_general1400_as_ci
  They implement accent sensitive case insensitive comparison.
  The weight of a character is equal to the code point of its
  upper case variant. These collations use Unicode-14.0.0 casefolding data.

  The result of
     my_charset_utf8mb3_general1400_as_ci.strcoll()
  is very close to the former
     my_charset_utf8mb3_general_ci.strcasecmp()

  There is only a difference in a couple dozen rare characters, because:
    - the switch from "tolower" to "toupper" comparison, to make
      utf8mb3_general1400_as_ci closer to utf8mb3_general_ci
    - the switch from Unicode-3.0.0 to Unicode-14.0.0
  This difference should be tolarable. See the list of affected
  characters in the MDEV description.

  Note, utf8mb4_general1400_as_ci correctly handles non-BMP characters!
  Unlike utf8mb4_general_ci, it does not treat all BMP characters
  as equal.

- Adding classes representing names of the file based database objects:

    Lex_ident_db
    Lex_ident_table
    Lex_ident_trigger

  Their comparison collation depends on the underlying
  file system case sensitivity and on --lower-case-table-names
  and can be either my_charset_bin or my_charset_utf8mb3_general1400_as_ci.

- Adding classes representing names of other database objects,
  whose names have case insensitive comparison style,
  using my_charset_utf8mb3_general1400_as_ci:

  Lex_ident_column
  Lex_ident_sys_var
  Lex_ident_user_var
  Lex_ident_sp_var
  Lex_ident_ps
  Lex_ident_i_s_table
  Lex_ident_window
  Lex_ident_func
  Lex_ident_partition
  Lex_ident_with_element
  Lex_ident_rpl_filter
  Lex_ident_master_info
  Lex_ident_host
  Lex_ident_locale
  Lex_ident_plugin
  Lex_ident_engine
  Lex_ident_server
  Lex_ident_savepoint
  Lex_ident_charset
  engine_option_value::Name

- All the mentioned Lex_ident_xxx classes implement a method streq():

  if (ident1.streq(ident2))
     do_equal();

  This method works as a wrapper for CHARSET_INFO::streq().

- Changing a lot of "LEX_CSTRING name" to "Lex_ident_xxx name"
  in class members and in function/method parameters.

- Replacing all calls like
    system_charset_info->coll->strcasecmp(ident1, ident2)
  to
    ident1.streq(ident2)

- Taking advantage of the c++11 user defined literal operator
  for LEX_CSTRING (see m_strings.h) and Lex_ident_xxx (see lex_ident.h)
  data types. Use example:

  const Lex_ident_column primary_key_name= "PRIMARY"_Lex_ident_column;

  is now a shorter version of:

  const Lex_ident_column primary_key_name=
    Lex_ident_column({STRING_WITH_LEN("PRIMARY")});
2024-04-18 15:22:10 +04:00
Dave Gosselin
ee3d4ec414 MDEV-12404 Index condition pushdown on partitioned tables
Support index condition pushdown within partitioned tables.
- ha_partition will pass the pushed index condition into all of the used
  partitions.
  - We require that all of the partitions to handle the pushed index
    condition in the same way.
- When using ICP, one may read rows (e.g. call h->index_read_map(buf, ...)
  only to buf= table->record[0], for two reasons:
  * Pushed index condition's Item_field objects point into record[0]
  * InnoDB requires this: it calls offset() which assumes record[0].
  So, when using ICP, ha_partition will read partition records to
  table->record[0] and then will copy record away if it needs it to be
  elsewhere.
2024-04-17 09:07:19 -04:00
Yuchen Pei
593392ba8b
MDEV-31789 Deprecate spider_casual_read
The corresponding table param was deprecated as part of MDEV-28861
2024-03-25 12:51:13 +11:00
Alexander Barkov
929c2e06aa MDEV-31531 Remove my_casedn_str() and my_caseup_str()
Under terms of MDEV 27490 we'll add support for non-BMP identifiers
and upgrade casefolding information to Unicode version 14.0.0.
In Unicode-14.0.0 conversion to lower and upper cases can increase octet length
of the string, so conversion won't be possible in-place any more.

This patch removes virtual functions performing in-place casefolding:
  - my_charset_handler_st::casedn_str()
  - my_charset_handler_st::caseup_str()
and fixes the code to use the non-inplace functions instead:
  - my_charset_handler_st::casedn()
  - my_charset_handler_st::caseup()
2024-02-28 22:20:29 +04:00
Vladislav Vaintroub
59df9f26bb MDEV-32841 - review fixes 2024-02-28 13:31:19 +01:00
Vladislav Vaintroub
f8bb99bf4a MDEV-32841 Use chrono::duration<double> instead of chrono::duration<float>
Requested by Marko during review
2024-02-28 13:31:19 +01:00
Vladislav Vaintroub
01466adc13 MDEV-32841 Provide Innodb async IO statistics
Provide some statistics about asynchronous IO reads and writes:
 - number of pending operations
 - number of completion callbacks that are currently being executed
 - number of completion callbacks that are currently queued
   (due to restriction on number of IO threads)
 - total number of IOs finished
 - total time to wait for free IO slot
 - total number of completions that were queued.

Also revert tpool InnoDB perfschema instrumentation (MDEV-31048)
That instrumentation of cache mutex did not bring any revelation (
the mutex is taken for a couple of instructions), and made it impossible
to use tpool outside of the server (e.g in mariadbimport/dump)
2024-02-28 13:31:19 +01:00
Sergei Golubchik
eeba940311 remove deprecated since 10.4 2024-02-17 17:10:25 +01:00
Oleksandr Byelkin
fa69b085b1 Merge branch '11.3' into 11.4 2024-02-15 13:53:21 +01:00
Yuchen Pei
068a6819eb MDEV-33441 Do not deinit plugin variables when retry requested
After MDEV-31400, plugins are allowed to ask for retries when failing
initialisation. However, such failures also cause plugin system
variables to be deleted (plugin_variables_deinit()) before retrying
and are not re-added during retry.

We fix this by checking that if the plugin has requested a retry the
variables are not deleted. Because plugin_deinitialize() also calls
plugin_variables_deinit(), if the retry fails, the variables will
still be deleted.

Alternatives considered:

- remove the plugin_variables_deinit() from plugin_initialize() error
handling altogether. We decide to take a more conservative approach
here.

- re-add the system variables during retry. It is more complicated
than simply iterating over plugin->system_vars and call
my_hash_insert(). For example we will need to assign values to
the test_load field and extract more code from test_plugin_options(),
if that is possible.
2024-02-14 18:45:03 +01:00
Sergei Golubchik
abcd23add2 MDEV-31857 enable --ssl-verify-server-cert by default in the internal client
enable ssl + ssl_verify_server_cert in the internal client too

* fix replication tests to disable master_ssl_verify_server_cert
  because accounts are passwordless - except rpl.rpl_ssl1
* fix federated/federatedx/connect to disable SSL_VERIFY_SERVER_CERT
  because they cannot configure an ssl connection
* fix spider to disable ssl_verify_server_cert, if configuration
  says so, as spider _can_ configure an ssl connection
* memory leak in embedded test-connect
2024-02-04 22:19:19 +01:00
Oleksandr Byelkin
d21cb43db1 Merge branch '11.2' into 11.3 2024-02-04 16:42:31 +01:00
Sergei Golubchik
03094bbc8a cleanup: octet2hex takes an uchar* argument
char is a character, uchar is an octet.
casts removed (or added) as needed
2024-02-03 11:22:21 +01:00
Sergei Golubchik
d7699c51eb test.cnf files should !include default_my.cnf
not default_mysqld.cnf. The latter has only server settings,
it misses mtr-specific client configuration

Except for spider, that doesn't use mysqld.1 server
and default_my.cnf starts it automatically.
Spider tests have to include both default_mysqld.cnf and
default_client.cnf
2024-02-03 11:22:20 +01:00
Sergei Golubchik
79580f4f96 Merge branch '11.1' into 11.2 2024-02-02 17:43:57 +01:00
Sergei Golubchik
b6680e0101 Merge branch '11.0' into 11.1 2024-02-02 11:30:47 +01:00
Sergei Golubchik
6ef0b2ee5f Merge branch '10.11' into 11.0 2024-02-01 18:57:08 +01:00
Sergei Golubchik
87e13722a9 Merge branch '10.6' into 10.11 2024-02-01 18:36:14 +01:00
Sergei Golubchik
2278f3503e fix columnstore compilation on fc39 2024-01-31 22:02:59 +01:00
Sergei Golubchik
3f6038bc51 Merge branch '10.5' into 10.6 2024-01-31 18:04:03 +01:00
Sergei Golubchik
01f6abd1d4 Merge branch '10.4' into 10.5 2024-01-31 17:32:53 +01:00
Sergei Golubchik
d1744ee7a2 MDEV-33343 spider.mdev_28739_simple fails in buildbot
test disabled, until fixed
2024-01-31 15:32:37 +01:00
Oleksandr Byelkin
fe490f85bb Merge branch '10.11' into 11.0 2024-01-30 08:54:10 +01:00
Oleksandr Byelkin
14d930db5d Merge branch '10.6' into 10.11 2024-01-30 08:17:58 +01:00
Oleksandr Byelkin
25c0806867 Merge branch '10.5' into 10.6 2024-01-30 07:43:15 +01:00
Oleksandr Byelkin
50107c4b22 Merge branch '10.4' into 10.5 2024-01-30 07:26:17 +01:00
Yuchen Pei
1070575a89
MDEV-33191 spider: fix dbton_id when iterating over links
There are two array fields in spider_share with similar names:

share->use_sql_dbton_ids that maps from i to the i-th dbton used by
share. Thus it should be used only when i iterates over all distinct
dbtons used by share.

share->sql_dbton_ids that maps from i to the dbton used by the i-th
link of the share. Thus it should be used only when i iterates over
all links of a share.

We correct instances where share->sql_dbton_ids should be used instead
of share->use_sql_dbton_ids.
2024-01-25 12:33:52 +11:00
Yuchen Pei
20741b9237
MDEV-28861 Deprecate spider table options by comment/connection
Also deprecating table params not implemented in MDEV-28856.
2024-01-24 12:18:18 +11:00
Monty
26c86c39fc Fixed some mtr tests that failed on windows
Most things where wrong in the test suite.
The one thing that was a bug was that table_map_id was in some places
defined as ulong and in other places as ulonglong. On Linux 64 bit this
is not a problem as ulong == ulonglong, but on windows this caused failures.
Fixed by ensuring that all instances of table_map_id are ulonglong.
2024-01-23 13:03:12 +02:00
Monty
740d3e7a74 Trivial fixes:
- Removed not used variable 'file' from MYSQL_BIN_LOG::open()
- Assigned not initialized variable in connect/tabext.cpp
2024-01-23 13:03:12 +02:00
Daniel Black
5ce6a352b6 MDEV-33290: Disable ColumnStore based on boost version
MCOL-5611 supporting with Boost-1.80, the version "next_prime"
disappears from https://github.com/boostorg/unordered/blob/boost-1.79.0/include/boost/unordered/detail/implementation.hpp
makes it the currenly highest supported versions.

Lets check this version.

While CMake-3.19+ supports version ranges in package determinations this
isn't supported for Boost in Cmake-3.28. So we check for the 1.80 and
don't compile ColumnStore.
2024-01-23 15:20:25 +11:00
Oleksandr Byelkin
a9172b8a43 Update minizip files for connect enginbe from last zlib 1.3. 2024-01-22 19:44:04 +01:00
Marko Mäkelä
35cc4b6c05 Merge 10.11 into 11.0 2024-01-22 10:10:50 +02:00
Marko Mäkelä
7f11fad85a MDEV-32968: After-merge fix
This fixes up merge commit 9d20853c74
2024-01-22 10:04:11 +02:00
Marko Mäkelä
b3ca7fa089 Merge 10.6 into 10.11 2024-01-22 08:49:04 +02:00
Marko Mäkelä
495e7f1b3d MDEV-33053 fixup: Correct a condition before a message 2024-01-22 08:24:08 +02:00
Marko Mäkelä
5c243d4caf Merge 10.5 into 10.6 2024-01-22 08:20:08 +02:00
Marko Mäkelä
e237925963 MDEV-33031 test fixup for HAVE_PERFSCHEMA=NO 2024-01-22 08:19:00 +02:00
Daniel Black
0c23f84d8d MDEV-32983 cosmetic improvement on path separator near ib_buffer_pool
A mix of path separators looks odd.

  InnoDB: Loading buffer pool(s) from C:\xampp\mysql\data/ib_buffer_pool

This was changed in cf552f5886

Both forward slashes and backward slashes work on Windows. We do not
use \\?\ names.

So we improve the consistent look of it so it doesn't look like a bug.

Normalize, in this case, the path separator to \ for making the filename.

Reported thanks to Github user @celestinoxp.

Closes: https://github.com/ApacheFriends/xampp-build/issues/33
Reviewed by: Marko Mäkelä and Vladislav Vaintroub
2024-01-22 16:56:00 +11:00
Sisi Huang
7e8e51eb3a MDEV-32990 federatedx time_zone round trips
Modified `federatedx_io_mysql::actual_query` to set the time zone to '+00:00' only upon establishing a new connection instead of with each query execution.
2024-01-22 13:44:53 +11:00
Sergei Golubchik
3a33ae8601 MDEV-33091 pcre2 headers aren't found on Solaris
use pkg-config to find pcre2, if possible

rename PCRE_INCLUDES to use PKG_CHECK_MODULES naming, PCRE_INCLUDE_DIRS
2024-01-19 20:23:45 +01:00
Thirunarayanan Balathandayuthapani
7573fe8b07 MDEV-32968 InnoDB fails to restore tablespace first page from doublewrite buffer when page is empty
recv_dblwr_t::find_first_page(): Free the allocated memory
to read the first 3 pages from tablespace.

innodb.doublewrite: Added sleep to ensure page cleaner thread
wake up from my_cond_wait
2024-01-19 17:01:36 +05:30
Marko Mäkelä
21560bee9d Revert "MDEV-32899 InnoDB is holding shared dict_sys.latch while waiting for FOREIGN KEY child table lock on DDL"
This reverts commit 569da6a7ba,
commit 768a736174, and
commit ba6bf7ad9e
because of a regression that was filed as MDEV-33104.
2024-01-19 12:46:11 +02:00
Marko Mäkelä
7e65e3027e MDEV-33275 buf_flush_LRU(): mysql_mutex_assert_owner(&buf_pool.mutex) failed
In commit a55b951e60 (MDEV-26827)
an error was introduced in a rarely executed code path of the
buf_flush_page_cleaner() thread. As a result, the function
buf_flush_LRU() could be invoked while not holding buf_pool.mutex.

Reviewed by: Debarun Banerjee
2024-01-19 12:40:32 +02:00
Marko Mäkelä
d34479dc66 MDEV-33053 InnoDB LRU flushing does not run before running out of buffer pool
buf_flush_LRU(): Display a warning if no pages could be evicted and
no writes initiated.

buf_pool_t::need_LRU_eviction(): Renamed from buf_pool_t::ran_out().
Check if the amount of free pages is smaller than innodb_lru_scan_depth
instead of checking if it is 0.

buf_flush_page_cleaner(): For the final LRU flush after a checkpoint
flush, use a "budget" of innodb_io_capacity_max, like we do in the
case when we are not in "furious" checkpoint flushing.

Co-developed by: Debarun Banerjee
Reviewed by: Debarun Banerjee
Tested by: Matthias Leich
2024-01-19 12:40:16 +02:00
Marko Mäkelä
9374772ecd Merge 10.11 into 11.0 2024-01-19 09:07:48 +02:00
Daniel Black
82e8633420 innodb: IO Error message missing space
Noted by Susmeet Khaire - thanks.
2024-01-19 17:33:44 +11:00
Marko Mäkelä
a6290a5bc5 MDEV-33095 innodb_flush_method=O_DIRECT creates excessive errors on Solaris
The directio(3C) function on Solaris is supported on NFS and UFS
while the majority of users should be on ZFS, which is a copy-on-write
file system that implements transparent compression and therefore
cannot support unbuffered I/O.

Let us remove the call to directio() and simply treat
innodb_flush_method=O_DIRECT in the same way as the previous
default value innodb_flush_method=fsync on Solaris. Also, let us
remove some dead code around calls to os_file_set_nocache() on
platforms where fcntl(2) is not usable with O_DIRECT.

On IBM AIX, O_DIRECT is not documented for fcntl(2), only for open(2).
2024-01-19 15:34:33 +11:00
Marko Mäkelä
9d20853c74 Merge 10.6 into 10.11 2024-01-18 19:22:23 +02:00