Commit graph

10320 commits

Author SHA1 Message Date
Oleksandr Byelkin
ca41fdba22 fix MDEV-27217 (4d5ae2b325)
Add error from later versions to avoid chaging HA_ERR_*
accross versions and in already released versions.
2022-01-30 17:52:15 +01:00
Sergei Golubchik
bc10f58a58 MDEV-24909 JSON functions don't respect KILL QUERY / max_statement_time limit
pass the pointer to thd->killed down to the json library,
check it while scanning,
use thd->check_killed() to generate the proper error message
2022-01-30 12:07:31 +01:00
Oleksandr Byelkin
a576a1cea5 Merge branch '10.3' into 10.4 2022-01-30 09:46:52 +01:00
Oleksandr Byelkin
41a163ac5c Merge branch '10.2' into 10.3 2022-01-29 15:41:05 +01:00
Oleksandr Byelkin
880d543554 Merge branch 'merge-perfschema-5.7' into 10.5 2022-01-28 11:57:52 +01:00
Vladislav Vaintroub
be1d965384 MDEV-27373 wolfSSL 5.1.1
- compile wolfcrypt with kdf.c, to avoid undefined symbols in tls13.c
- define WOLFSSL_HAVE_ERROR_QUEUE to avoid endless loop SSL_get_error
- Do not use SSL_CTX_set_tmp_dh/get_dh2048, this would require additional
  compilation options in WolfSSL. Disable it for WolfSSL build, it works
  without it anyway.
- fix "macro already defined" Windows warning.
2022-01-25 11:19:00 +01:00
Sergei Golubchik
4504e6d14e test cases for MySQL bugs
also fix a comment, and update a macro just in case
2022-01-21 16:02:34 +01:00
Marko Mäkelä
c1d7b4575e MDEV-26870 --skip-symbolic-links does not disallow .isl file creation
The InnoDB DATA DIRECTORY attribute is not implemented via
symbolic links but something similar, *.isl files that contain
the names of data files.

InnoDB failed to ignore the DATA DIRECTORY attribute even though
the server was started with --skip-symbolic-links.

Native ALTER TABLE in InnoDB will retain the DATA DIRECTORY attribute
of the table, no matter if the table will be rebuilt or not.

Generic ALTER TABLE (with ALGORITHM=COPY) as well as TRUNCATE TABLE
will discard the DATA DIRECTORY attribute.

All tests have been run with and without the ./mtr option
--mysqld=--skip-symbolic-links
and some tests that use the InnoDB DATA DIRECTORY attribute
have been adjusted for this.
2022-01-21 14:43:59 +02:00
Alexander Barkov
b915f79e4e MDEV-25904 New collation functions to compare InnoDB style trimmed NO PAD strings 2022-01-21 12:16:07 +04:00
Vladislav Vaintroub
d42c2efbaa MDEV-25785 Add support for OpenSSL 3.0
Summary of changes

- MD_CTX_SIZE is increased

- EVP_CIPHER_CTX_buf_noconst(ctx) does not work anymore, points
  to nobody knows where. The assumption made previously was that
  (since the function does not seem to be documented)
  was that it points to the last partial source block.
  Add own partial block buffer for NOPAD encryption instead

- SECLEVEL in CipherString in openssl.cnf
  had been downgraded to 0, from 1, to make TLSv1.0 and TLSv1.1 possible
   (according to https://github.com/openssl/openssl/blob/openssl-3.0.0/NEWS.md
   even though the manual for SSL_CTX_get_security_level claims that it
   should not be necessary)

- Workaround Ssl_cipher_list issue, it now returns TLSv1.3 ciphers,
  in addition to what was set in --ssl-cipher

- ctx_buf buffer now must be aligned to 16 bytes with openssl(
  previously with WolfSSL only), ot crashes will happen

- updated aes-t , to be better debuggable
  using function, rather than a huge multiline macro
  added test that does "nopad" encryption piece-wise, to test
  replacement of EVP_CIPHER_CTX_buf_noconst
2022-01-20 16:01:10 +01:00
Sergei Petrunia
ce4956f322 Code cleanup 2022-01-19 18:14:07 +03:00
Sergei Petrunia
3936dc3353 MDEV-26724 Endless loop in json_escape_to_string upon ... empty string
Part#3:
- make json_escape() return different errors on conversion error
  and on out-of-space condition.
- Make histogram code handle conversion errors.
2022-01-19 18:10:11 +03:00
Sergei Petrunia
2a1cdbabec Fix JSON parsing: future-proof data representation in JSON, code cleanup 2022-01-19 18:10:09 +03:00
Michael Okoko
fb2edab3eb Extract json parser functions from class
Signed-off-by: Michael Okoko <okokomichaels@outlook.com>
2022-01-19 18:10:08 +03:00
Michael Okoko
6bc2df5fa4 Add parser to read JSON array (of histograms) into string vector
Signed-off-by: Michael Okoko <okokomichaels@outlook.com>
2022-01-19 18:10:08 +03:00
Vladislav Vaintroub
e222e44d1b Merge branch 'preview-10.8-MDEV-26713-Windows-i18-support' into 10.8 2022-01-18 21:37:52 +01:00
Daniel Black
d434250ee1 MDEV-25342: autosize innodb_buffer_pool_chunk_size
The previous default innodb_buffer_pool_chunk_size of 128M
made sense when the innodb buffer pool size was a few GB.

When the pool size is 128GB this means the chunk size is 0.1%
of this. Fine tuning the buffer pool size on such a fine
increment doesn't make practical sense. Also on extremely
large buffer pool systems, initializing on the default 128M can
also take a considerable amount of time.

When large pages are enabled, the chunk size has to be a multiple
of an available large page size or memory allocation without
use can occur.

Previously the default 0 was documented as disabling resizing.
With srv_buf_pool_chunk_unit > 0 assertions in the code and the
minimium value set, I doubt this was ever the case.

As such the autosizing (based on default 0) takes place as follows:
* a 64th of the innodb_buffer_pool_size
* if large pages, this is rounded down the the nearest multiple
  of the large page size.
* If less than 1MB, set to 1MB.

This does mean the new default innodb_buffer_pool_chunk size is
2MB, derived form the above formular with 128MB as the buffer pool
size.

The innodb_buffer_pool_chunk_size is changed to a size_t for
better compatiblity with the memory allocations which use size_t.
The previous upper limit is changed to the maxium of a size_t. The
maximium value used is the buffer pool size anyway.

Getting this default value of the chunk size to a more practical
size facilitates further development of more automated resizing
without significant overhead or memory fragmentation.

innodb_buffer_pool_resize test adjusted based on 1M default
chunk size thanks Wlad.
2022-01-18 14:20:57 +02:00
Sergei Golubchik
7b555ff2c5 MDEV-27341 Use SET PASSWORD to change PAM service
SET PASSWORD = PASSWORD('foo') would fail for pam plugin with

ERROR HY000: SET PASSWORD is ignored for users authenticating via pam plugin

but SET PASSWORD = 'foo' would not.

Now it will.
2022-01-17 18:19:29 +01:00
Aleksey Midenkov
4d5ae2b325 MDEV-27217 DELETE partition selection doesn't work for history partitions
LIMIT history switching requires the number of history partitions to
be marked for read: from first to last non-empty plus one empty. The
least we can do is to fail with error message if the needed partition
was not marked for read. As this is handler interface we require new
handler error code to display user-friendly error message.

Switching by INTERVAL works out-of-the-box with
ER_ROW_DOES_NOT_MATCH_GIVEN_PARTITION_SET error.
2022-01-13 23:35:16 +03:00
Vladislav Vaintroub
555a53f10d Windows : fix broken build with OpenSSL 2022-01-09 12:04:22 +01:00
Daniel Black
4f4d9586de mysys: my_rdtsc note about ARM counter
As reported in https://github.com/MariaDB/mariadb-docker/issues/338
and later https://github.com/hardkernel/linux/issues/423.

While modern kernels support this, it seems older hardware may be
stuck at kernel versions without this initialization.
2022-01-06 10:36:55 +11:00
alexfanqi
d18f6f2631 MDEV-27429: Support RISC-V cycle timer
Adapted from https://github.com/google/benchmark/pull/833
authored by Sam Elliot at lowRISC.

This requires the RISCV kernel to set the CY bit of the mcountern register
which is done on Linux, but documenting here in case another OS hits
a SIGILL here.

When CY bit of the mcounteren register is unset, reading the cycle register
will cause illegal instruction exception in the next privilege level ( user
mode or supervisor mode ). See the privileged isa manual section 3.1.11 in
https://github.com/riscv/riscv-isa-manual/releases/latest
2022-01-06 10:36:55 +11:00
Marko Mäkelä
daf4fa5238 Merge 10.7 into 10.8 2022-01-04 10:30:45 +02:00
Marko Mäkelä
7dfaded962 Merge 10.6 into 10.7 2022-01-04 09:55:58 +02:00
Marko Mäkelä
3f5726768f Merge 10.5 into 10.6 2022-01-04 09:26:38 +02:00
Julius Goryavsky
55bb933a88 Merge branch 10.4 into 10.5 2021-12-26 12:51:04 +01:00
sjaakola
49791cbc6f 10.4-MDEV-27275 CREATE TABLE with FK not safe for PA
This commit contains a fix, where the replication write set for a CREATE TABLE
will contain, as certification keys, table names for all FK references.
With this, all DML for the FK parent tables will conflict with the CREATE TABLE
statement.

There is also new test galera.MDEV-27276 to verify the fix.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-12-20 13:34:54 +02:00
sjaakola
c1846c4fcf MDEV-26803 PA unsafety with FK cascade delete operation
This commit has a mtr test where two two transactions delete a row from
two separate tables, which will cascade a FK delete for the same row in
a third table. Second replica node is configured with 2 applier threads,
and the test will fail if these two transactions are applied in parallel.

The actual fix, in this commit, is to mark a transaction as unsafe for
parallel applying when it traverses into cascade delete operation.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-12-17 09:38:23 +02:00
Vladislav Vaintroub
9ea83f7fbd MDEV-26713 set console codepage to what user set in --default-character-set
If someone on whatever reasons uses --default-character-set=cp850,
this will avoid incorrect display, and inserting incorrect data.

Adjusting console codepage sometimes also needs to happen with
--default-charset=auto, on older Windows. This is because autodetection
is not always exact. For example, console codepage on US editions of
Windows is 437. Client autodetects it as cp850, a rather loose
approximation, given 46 code point differences. We change the console
codepage to cp850, so that there is no discrepancy.

That fix is currently Windows-only, and serves people who used combination
of chcp to achieve WYSIWYG effect (although, this would mostly likely used
with utf8 in the past)

Now, --default-character-set would be a replacement for that.

Fix fs_character_set() detection of current codepage.
2021-12-15 19:13:57 +01:00
Vladislav Vaintroub
ea0a5cb0a4 MDEV-27092 Windows - services that have non-ASCII characters do not work with activeCodePage=UTF8
CreateServiceA, OpenServiceA, and couple of other functions do not work
correctly with non-ASCII character, in the special case where application
has defined activeCodePage=UTF8.

Workaround by redefining affected ANSI functions to own wrapper, which
works by converting narrow(ANSI) to wide, then calling wide function.

Deprecate original ANSI service functions, via declspec, so that we can catch
their use.
2021-12-15 19:13:57 +01:00
Vladislav Vaintroub
99e5ae3b1a MDEV-27090 Windows client - ReadConsoleA does not work correctly with UTF8 codepage
Corresponding Windows bug  https://github.com/microsoft/terminal/issues/4551

Use ReadConsoleW instead and convert to console's input codepage, to
workaround.

Also, disable VT sequences in the console output, as we do not knows what
type of data comes with SELECT, we do not want VT escapes there.

Remove my_cgets()
2021-12-15 19:13:57 +01:00
Sergei Golubchik
1e8bcbd0a0 Revert "MDEV-27036: re-enable my_json_writer-t unit test"
This reverts commit 2d21917e7d.

No explainations, lots of code moved, wrong cmake changes
2021-12-07 09:57:51 +01:00
Sergei Krivonos
2d21917e7d MDEV-27036: re-enable my_json_writer-t unit test 2021-12-04 22:25:46 -05:00
Eric Herman
6a282df0be Remove DYNAMIC_ARRAY get_index_dynamic()
The DYNAMIC_ARRAY copies values in and copies values out. Without a
comparitor function, get_index_dynamic() does not make sense.

This function is not used. If we have a need for a function like it
in the future, I propose we write a new one with unit tests showing
how it is used and demostrating that it behaves as expected.
2021-11-24 16:39:38 +02:00
Alexander Barkov
0a3d1d106a Refactoring for MDEV-27042 and MDEV-27009
This patch prepares the code for upcoming changes:

MDEV-27009 Add UCA-14.0.0 collations
MDEV-27042 UCA: Resetting contractions to ignorable does not work well

1. Adding "const" qualifiers to return type and parameters in functions:
- my_uca_contraction2_weight()
- my_wmemcmp()
- my_uca_contraction_weight()
- my_uca_scanner_contraction_find()
- my_uca_previous_context_find()
- my_uca_context_weight_find()

2. Adding a helper function my_uca_true_contraction_eq()

3. Changing the way how scanner->wbeg is set during context weight handling.
   It was previously set inside functions:
   - my_uca_scanner_contraction_find()
   - my_uca_previous_context_find()
   Now it's set inside scanner_next(), which makes the code more symmetric
   for context-free and context-dependent sequences.
   This makes then upcoming fix for MDEV-27042 simpler.
2021-11-24 13:35:57 +04:00
Vladislav Vaintroub
0102732686 Revert "MDEV-26713 Windows - improve utf8 support for command line tools"
This reverts commit several commits pushed by mistake.
2021-11-19 09:46:57 +01:00
Vladislav Vaintroub
2dcb823631 MDEV-26713 Windows- UTF8 encoding in the installer
Workaround Windows' bug in services "ANSI" when process ANSI codepage is
UTF8.
They turn out to work unlike any other API .

Expected behavior : strings  be converted from GetACP() to Unicode,
and "wide" function would be then called.

Actual current behavior :
it seems to handle strings as-if they would be encoded in system-default
ACP, rather than process-specific GetACP()

Fix: redefine the OpenService,CreateService and ChangeServiceConfig
and do ANSI-Wide conversion outselves.

Tell compiler to deprecate some ANSI service functions.

xxx
2021-11-18 17:25:40 +01:00
Sergei Golubchik
72fb37ea89 cleanup: uuid 2021-10-29 18:29:01 +02:00
Marko Mäkelä
d7af7bfc2b Merge 10.6 into 10.7 2021-10-28 09:14:51 +03:00
Marko Mäkelä
d8c6c53a06 Merge 10.5 into 10.6 2021-10-28 09:08:58 +03:00
Marko Mäkelä
a8ded39557 Merge 10.4 into 10.5 2021-10-28 08:48:36 +03:00
Marko Mäkelä
3a79e5fd31 Merge 10.3 into 10.4 2021-10-28 08:28:39 +03:00
Marko Mäkelä
657bcf928e Merge 10.2 into 10.3 2021-10-28 07:50:05 +03:00
Sergei Golubchik
867f05ded3 Avoid ASAN odr error
like

==31311==ERROR: AddressSanitizer: odr-violation (0x7f3cda2e1480):
  [1] size=8 'provider_service_lz4' libservices/provider_service_lz4.c:14:17
  [2] size=8 'provider_service_lz4' sql/sql_plugin_services.ic:301:33
2021-10-27 15:55:14 +02:00
Kartik Soneji
bf8b699f64 MDEV-12933 sort out the compression library chaos
bzip2/lz4/lzma/lzo/snappy compression is now provided via *services*

they're almost like normal services, but in include/providers/
and they're supposed to provide exactly the same interface
as original compression libraries (but not everything,
only enough of if for the code to compile).

the services are implemented via dummy functions that return
corresponding error values (LZMA_PROG_ERROR, LZO_E_INTERNAL_ERROR, etc).

the actual compression libraries are linked into corresponding
provider plugins. Providers are daemon plugins that when loaded
replace service pointers to point to actual compression functions.

That is, run-time dependency on compression libraries is now on plugins,
and the server doesn't need any compression libraries to run, but
will automatically support the compression when a plugin is loaded.

InnoDB and Mroonga use compression plugins now. RocksDB doesn't,
because it comes with standalone utility binaries that cannot
load plugins.
2021-10-27 15:55:14 +02:00
Marko Mäkelä
da46c37bc7 Merge 10.6 into 10.7 2021-10-27 10:24:08 +03:00
Marko Mäkelä
d4a89b9262 Merge 10.5 into 10.6 2021-10-27 10:06:02 +03:00
Marko Mäkelä
44f9736e0b Merge 10.4 into 10.5 2021-10-27 09:48:22 +03:00
Eugene Kosov
d74d95961a MDEV-18543 IMPORT TABLESPACE fails after instant DROP COLUMN
ALTER TABLE IMPORT doesn't properly handle instant alter metadata.
This patch makes IMPORT read, parse and apply instant alter metadata at the
very beginning of operation. So, cases when source table has some metadata
and destination table doesn't have it now works fine.

DISCARD already removes instant metadata so importing normal table into
instant table worked fine before this patch.

decrypt_decompress(): decrypts and decompresses page if needed

handle_instant_metadata(): this should be the first thing to read source
table. Basically, it applies instant metadata to a destination
dict_table_t object. This is the first thing to read FSP flags so
all possible checks of it were moved to this function.

PageConverter::update_index_page(): it doesn't now read instant metadata.
This logic were moved into handle_instant_metadata()

row_import::match_flags(): this is a first part row_import::match_schema().
As a separate function it's used by handle_instant_metadata().

fil_space_t::is_full_crc32_compressed(): added convenient function

ha_innobase::discard_or_import_tablespace(): do not reload table definition
to read instant metadata because handle_instant_metadata() does it better.
The reverted code was originally added in
4e7ee166a9

ANONYMOUS_VAR: this is a handy thing to use along with make_scope_exit()

full_crc32_import.test shows different results, because no
dict_table_close() and dict_table_open_on_id() happens.
Thus, SHOW CREATE TABLE shows a little bit older table definition.
2021-10-26 22:50:58 +06:00
Aleksey Midenkov
d324c03d0c Vanilla cleanups and refactorings
Dead code cleanup:

part_info->num_parts usage was wrong and working incorrectly in
mysql_drop_partitions() because num_parts is already updated in
prep_alter_part_table(). We don't have to update part_info->partitions
because part_info is destroyed at alter_partition_lock_handling().

Cleanups:

- DBUG_EVALUATE_IF() macro replaced by shorter form DBUG_IF();
- Typo in ER_KEY_COLUMN_DOES_NOT_EXITS.

Refactorings:

- Splitted write_log_replace_delete_frm() into write_log_delete_frm()
  and write_log_replace_frm();
- partition_info via DDL_LOG_STATE;
- set_part_info_exec_log_entry() removed.

DBUG_EVALUATE removed

DBUG_EVALUTATE was only added for consistency together with
DBUG_EVALUATE_IF. It is not used anywhere in the code.

DBUG_SUICIDE() fix on release build

On release DBUG_SUICIDE() was statement. It was wrong as
DBUG_SUICIDE() is used in expression context.
2021-10-26 17:07:46 +02:00