Commit graph

194507 commits

Author SHA1 Message Date
Vladislav Vaintroub
a296c52627 MDEV-26713 UTF8 support on Windows, mysql_upgrade_service preparation
- Tolerate situation, when datadir for service seems invalid/non-existing
prior to upgrade. It could be that my.ini contains legacy ANSI characters
for the data directory. Those can't be read correctly by
mysql_upgrade_service, which uses a different ANSI codepage(UTF8)
.
- schedule upgrade_config_file at later stage, because once we
convert it to UTF-8 (followup patch), this will render config file uselss
with the older version of mariadbd.exe

- Refactor upgrade_conf_file.cc, prepare for UTF-8 conversion.
2021-12-15 19:13:57 +01:00
Vladislav Vaintroub
a4fc41b6b4 MDEV-26713 Treat codepage 65001 as utf8mb4, not utf8mb3
Also, fix the "UTF8" option in MSI, which is responsible for character-set-server
setting
2021-12-15 19:13:57 +01:00
Vladislav Vaintroub
ba9d231b5a MDEV-26713 Set activeCodePage=UTF8 for windows programs
- Use corresponding entry in the manifest, as described in
https://docs.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page

- If if ANSI codepage is UTF8 (i.e for Windows 1903 and later)
  Use UTF8 as default client charset
  Set console codepage(s) to UTF8, in case process is using console

- Allow some previously disabled MTR tests, that used Unicode for in "exec",
  for the recent Windows versions
2021-12-15 19:13:57 +01:00
Vladislav Vaintroub
4d3ac32848 MDEV-27093 Do not pass root password in HEX(clear text) from mariadb-install-db.exe to bootstrap
Previously, password was passed as hex(clear_text_password).
The hex encoding was used to avoid masking apostrophe and backslash etc.

However, bootstrap still manages to misinterpert UTF8 password, so that
root would not connect later.

So the fix is to compute the native password hash inside mysql_install_db
already instead, and create user with that hash, rather than letting bootstrap
calculate it by using PASSWORD() function.
2021-12-15 19:13:57 +01:00
Vladislav Vaintroub
ea0a5cb0a4 MDEV-27092 Windows - services that have non-ASCII characters do not work with activeCodePage=UTF8
CreateServiceA, OpenServiceA, and couple of other functions do not work
correctly with non-ASCII character, in the special case where application
has defined activeCodePage=UTF8.

Workaround by redefining affected ANSI functions to own wrapper, which
works by converting narrow(ANSI) to wide, then calling wide function.

Deprecate original ANSI service functions, via declspec, so that we can catch
their use.
2021-12-15 19:13:57 +01:00
Vladislav Vaintroub
99e5ae3b1a MDEV-27090 Windows client - ReadConsoleA does not work correctly with UTF8 codepage
Corresponding Windows bug  https://github.com/microsoft/terminal/issues/4551

Use ReadConsoleW instead and convert to console's input codepage, to
workaround.

Also, disable VT sequences in the console output, as we do not knows what
type of data comes with SELECT, we do not want VT escapes there.

Remove my_cgets()
2021-12-15 19:13:57 +01:00
Vladislav Vaintroub
9e9b211f22 MDEV-27089 Windows : incorrect handling of non-ASCIIs in get_tty_password
Prior to patch, get_password would echo multple mask characters '*', for
a single multibyte input character.

Fixed the behavior by using "wide" version of getch, _getwch.
Also take care of possible characters outside of BMP (i.e we do not print
'*' for high surrogates).

The function will now internally construct the "wide" password string,
and conver to the console codepage. Some characters could still be lost
in that conversion, unless the codepage is utf8, but this is not any new
bug.
2021-12-15 19:13:57 +01:00
Sergei Golubchik
6208228b78 disable galera_3nodes.galera_ipv6_mariabackup
and galera_3nodes.galera_ipv6_mariabackup_section
until their corresponding MDEV's are fixed.

They fail almost everywhere.
2021-12-15 15:43:03 +01:00
Marko Mäkelä
4434fb4a75 Merge 10.7 into 10.8 2021-12-14 14:29:04 +02:00
Marko Mäkelä
92a4e76a2c Merge 10.6 into 10.7 2021-12-14 14:27:35 +02:00
mkaruza
b1d647ae85 MDEV-27001 Galera crashes when converting table to partition
SQL statments could have table entries added in lexer. This entries
should not invalidate `next_global` member.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-12-14 09:00:08 +02:00
Thirunarayanan Balathandayuthapani
660cfe4782 MDEV-27014 InnoDB fails to restore page 0 from the doublewrite buffer
- Addressing the format issue in deferred_dblwr() and changed the
function comment.
2021-12-12 15:09:59 +05:30
Thirunarayanan Balathandayuthapani
18c335a39e MDEV-27111 atomic.rename_table test case fails
InnoDB fails to identify the deferred tablespace after recovery.
Problem is that InnoDB fails to rename the tablespace present
in recovered tablespace. Fix is that InnoDB should try to rename the
recovered tablespace when tablespace is being deferred
2021-12-12 09:58:54 +05:30
Thirunarayanan Balathandayuthapani
be5990d0c8 MDEV-27014 InnoDB fails to restore page 0 from the doublewrite buffer
This patch reverts the commit cab8f4b552.
InnoDB fails to restore page0 from doublewrite buffer when the
tablespace is being deferred. In that case, InnoDB doesn't find
INIT_PAGE redo log record for page0 and it leads to failure.
InnoDB should recovery page0 from doublewrite buffer for the
deferred tablespace before applying the redo log records.

Added deferred_dblwr() to restore page0 of deferred tablespace
from doublewrite buffer
2021-12-12 09:58:54 +05:30
Sergei Krivonos
ea94895369 MDEV-27206: [ERROR] Duplicated key: cause, Assertion `is_uniq_key' failed with optimizer trace 2021-12-10 22:12:01 +02:00
Marko Mäkelä
ccdf5711a8 Merge 10.7 into 10.8 2021-12-10 13:05:06 +02:00
Marko Mäkelä
979b23d5bf Add forgotten changes to the parent commit 2021-12-10 13:04:46 +02:00
Marko Mäkelä
978116d991 Merge 10.7 into 10.8 2021-12-10 12:29:14 +02:00
Marko Mäkelä
28b27b96d0 Cleanup: Remove some ib::logger in recovery messages 2021-12-10 12:26:26 +02:00
Marko Mäkelä
26fdbd7632 Merge 10.6 into 10.7 2021-12-10 11:32:49 +02:00
Marko Mäkelä
50ed0bd891 MDEV-27219 Some error messages might report table names incorrectly on LLP64
In commit 49e2c8f0a6 (MDEV-25743)
some more use of the printf-style format "%.*s" was added.
The length parameter is of type int, not size_t.
On 64-bit platforms that follow the LLP64 convention (such as
64-bit Microsoft Windows), sizeof(int)==4 and sizeof(size_t)==8.

Let us explicitly cast the lengths to the correct type in order
to avoid any trouble.
2021-12-10 11:29:06 +02:00
Sergei Petrunia
c88e37ff85 MDEV-27204: [ERROR] Json_writer: a member name was expected, Assertion `got_name
[Adjusting Sergei Krivonos's patch]

"duplicates_removal" may contain multiple elements inside it and
so should have a JSON array as a value (and not object).
2021-12-09 16:49:40 +03:00
Sergei Golubchik
f502ae8226 SUMMARY/DESCRIPTION for compression provider RPMs 2021-12-08 14:57:37 +01:00
Sergei Golubchik
ef77c05126 Merge branch '10.6' into 10.7 2021-12-08 10:33:36 +01:00
Sergei Golubchik
60d565e66a cmake: detect lz4 version, require >= 1.6 2021-12-08 10:31:00 +01:00
Sergei Golubchik
186c1fa250 Merge branch '10.5' into 10.6 2021-12-07 22:11:30 +01:00
Sergei Golubchik
62ea1b4407 BUG#31761802 STATISTICS ANY QUERIES USING VIEWS ARE SUMMARIZED TOGETHER WITH THE VIEW DEFINITION SELECT
test case only
2021-12-07 21:31:27 +01:00
Sergei Golubchik
e27b1c3137 require system pcre2 in rpms
now when SLES12.3 is gone, we can enforce it
2021-12-07 21:31:13 +01:00
Sergei Golubchik
88ac91c7cc ColumnStore and S3 SUMMARY/DESCRIPTION for RPM 2021-12-07 21:31:00 +01:00
Sergei Golubchik
de70f921ce Merge branch '10.4' into 10.5 2021-12-07 21:30:27 +01:00
Marko Mäkelä
cfcfdc65df MDEV-27190 InnoDB upgrade from 10.2, 10.3, 10.4 is not crash-safe
During startup, InnoDB must write a FILE_CHECKPOINT record.
However, before MDEV-12353 (in MariaDB Server 10.2, 10.3, 10.4)
the corresponding record MLOG_CHECKPOINT was encoded in a different way.

When we are upgrading from a logically empty 10.2, 10.3, or 10.4 redo log,
we must not write anything to the old log file, because if the server were
killed during the upgrade, we would end up with a corrupted log file, and
both the old and the new server would refuse to start up.

On upgrade, we must simply create a new logically empty log file
and replace the old ib_logfile0 with that.
2021-12-07 17:00:46 +02:00
Eugene Kosov
890c55177d MDEV-27183 optimize std::map lookup in during crash recovery
This is a low hanging fruit. Before this patch std::map::emplace() was
a ~50% of the whole recv_sys_t::parse() operation in by test.
After the fix it's only ~20%.

recv_sys_t::parse() recv_sys_t::pages is a collection of all pages
to recovery. Often, there are multiple changes for a single page.
Often, they go in a row and for such cases let's avoid
lookup in a std::map. cached_pages_it serves as a cache
of size 1.

recv_sys_t::add(): replace page_id argument with a std::map::iterator
2021-12-07 15:50:00 +06:00
Sergei Golubchik
1e8bcbd0a0 Revert "MDEV-27036: re-enable my_json_writer-t unit test"
This reverts commit 2d21917e7d.

No explainations, lots of code moved, wrong cmake changes
2021-12-07 09:57:51 +01:00
Sergei Golubchik
e8a91c18ea Merge branch '10.3' into 10.4 2021-12-07 09:47:42 +01:00
Sergei Golubchik
153b75b576 Merge branch '10.2' into 10.3 2021-12-06 22:23:07 +01:00
Sergei Golubchik
71027eceac fix srpm builds after fe065f8d90 2021-12-06 18:50:19 +01:00
Sergei Golubchik
214cad8c3b fix ./mtr --manual warning after f5441ef4da 2021-12-06 18:50:19 +01:00
Sergei Golubchik
23bfacf1cc Revert "Improve LibFMT detection"
This reverts commit 9feaa6be07.

No, we do NOT support fmt 6.x, it formats values differently.
And we MUST use FMT_HEADER_ONLY
2021-12-06 18:45:21 +01:00
Eugene Kosov
0064316f19 cleanup: reduce code bloat 2021-12-06 14:06:17 +06:00
Sergei Krivonos
2d21917e7d MDEV-27036: re-enable my_json_writer-t unit test 2021-12-04 22:25:46 -05:00
Sergei Krivonos
9feaa6be07 Improve LibFMT detection
Signed-off-by: Sergei Krivonos <sergei.krivonos@mariadb.com>
2021-12-04 16:05:24 -07:00
Marko Mäkelä
467c7b2b24 Merge 10.7 into 10.8 2021-12-04 13:43:52 +02:00
Marko Mäkelä
182bf9b333 Merge 10.6 into 10.7 2021-12-04 13:23:14 +02:00
Marko Mäkelä
6deaff58a9 MDEV-27058 fixup: GCC 11 -march=i686 -Warray-bounds
page_zip_des_t::clear(): Avoid a bogus GCC warning with
some pointer arithmetics. Yes, storing the unrelated member "fix"
in this object is ugly, but it avoids memory alignment overhead
on 64-bit architectures.
2021-12-04 12:55:26 +02:00
Marko Mäkelä
3f7040fa97 Merge 10.5 into 10.6 2021-12-04 12:42:29 +02:00
Marko Mäkelä
fa1325512b Correct some comments
In commit aae3f921ad
recv_apply_hashed_log_recs() was replaced by recv_sys_t::apply().
2021-12-03 17:01:48 +02:00
Eugene Kosov
5d7da02793 MDEV-27139 32-bit systems fail to use big innodb-log-file-size
log_write_buf(): do not cast to size_t which prevents to write to files
which a bigger that 4G and remove useless assertion
2021-12-03 14:57:23 +06:00
Marko Mäkelä
d2a7710635 Merge 10.4 into 10.5 2021-12-03 10:27:35 +02:00
Marko Mäkelä
f458acc81e MDEV-27160 Out of memory in main.long_unique
A part of the test main.long_unique attempts to insert records
with two 60,000,001-byte columns. Let us move that test into
a separate file main.long_unique_big, declared as big test,
so that it can be skipped in environments with limited memory.
2021-12-03 09:56:30 +02:00
Christopher Odenbach
658a1e1fed Use mysqladmin ping instead of pid files
How 'bout know?
2021-12-03 15:34:08 +11:00