Commit graph

193969 commits

Author SHA1 Message Date
Sergei Golubchik
513c8b4c25 cannot allocate a new String[] in the ::val_str() method
String inherits from Sql_alloc, so it's allocated on the thd's memroot,
this cannot be done per row.

Moved String[] allocation into the Item_func_sformat constructor
(not fix_fields(), because we want it on the same memroot where the item
is).
2021-10-12 13:28:10 +02:00
Sergei Golubchik
8150f52e47 support charsets with mbminlen > 1 2021-10-12 13:28:10 +02:00
Sergei Golubchik
fe65ca0176 don't build bundled libfmt, we use it in header-only mode anyway 2021-10-12 13:28:10 +02:00
Sergei Golubchik
6777c67a39 support {:L} with a ',' thousand separator 2021-10-12 13:28:10 +02:00
Sergei Golubchik
519bb2e41c disallow {:p}
by printing a String as an std::string_view, not as a char*
2021-10-12 13:28:10 +02:00
Sergei Golubchik
a363ccd681 misc cleanups
* comment - use the standard style
* no need to reimplement for with while
* TODO comments for not implemented types
* rename the error not to refer to the underlying library, it's
  the implementation detail
* use res->length(0) to set length to 0, but preserve the already
  allocated buffer
* rename main.sformat -> main.func_sformat
* removed a duplicated part of the test
2021-10-12 13:28:10 +02:00
Alan Cueva
e214e60201 MDEV-25015 Custom formatting of strings in MariaDB queries
SFORMAT() SQL function that uses fmtlib (https://fmt.dev/)
for python-like (also Rust, C++20, etc) string formatting

Only fmtlib 7.0.0+ is supported, older fmtlib
produces different results in the test.

No native support for temporal and decimal values,
* TIME_RESULT is handled as STRING_RESULT
* DECIMAL_RESULT as REAL_RESULT
2021-10-12 13:27:36 +02:00
Oleksandr Byelkin
cd390af982 MDEV-26637: (hash) ASAN: main.metadata and user_variables.basic MTR failures after MDEV-26572
Explicitly devide two function of 0 length in the hash keys comparing.
2021-10-12 10:16:29 +02:00
Oleksandr Byelkin
3c0f48a4c1 MDEV-26637: (roles) ASAN: main.metadata and user_variables.basic MTR failures after MDEV-26572
No empty name roles exists (no sens to look for).
The problem already present in our test cases.
2021-10-12 10:16:07 +02:00
Oleksandr Byelkin
8f7edb784f MDEV-26637: (variables) ASAN: main.metadata and user_variables.basic MTR failures after MDEV-26572
Prohibit user variables without name
2021-10-12 10:16:07 +02:00
Oleksandr Byelkin
af8b2c6cec MDEV-26637: (plugin) ASAN: main.metadata and user_variables.basic MTR failures after MDEV-26572
Get rid of locking "empty" plugin.
This problem present in our tests in case of malformed frm-file, for example.
2021-10-12 10:15:06 +02:00
Oleksandr Byelkin
763bdee81b MDEV-26637: (explicit length) ASAN: main.metadata and user_variables.basic MTR failures after MDEV-26572
Use explicit length for hash record length
2021-10-12 10:01:07 +02:00
Otto Kekäläinen
6cf7d6a093 Deb: Sync build and runtime dependencies from downstream to upstream
- Add libconfig-inifiles-perl to mariadb-client depends as mytop uses it
  7a748da87a
  http://bugs.debian.org/875708

- Make libzstd dependency versioned as RocksDB need at least 1.3.3
  da4eb54fc0

  However, https://github.com/facebook/rocksdb/blob/master/INSTALL.md
  implies that zstandard 1.1.3 would suffice. Version 1.3.x is available
  from Debian Stretch-backports and Ubuntu Xenial onwards.
2021-10-12 10:52:10 +11:00
Rucha Deodhar
5cc9cf9a05 MDEV-26681: ROW_NUMBER is not available within compound statement blocks
Fixed after patch MDEV-26606 because root cause was same.

Analysis: m_current_row_for_warning is reset to 1 during cleanup phase of
stored procedure. When we perform a copy because some statement of procedure
created warning, this reset value is passed to push _warning().
Fix: Add a parameter in relevant functions to pass correct value of
error index and don't use m_current_row_for_warning directly.
2021-10-07 00:31:19 +05:30
Rucha Deodhar
af95c991dd MDEV-26684: Unexpected ROW_NUMBER in a condition raised by a diagnostics
statement

Not repeatable after MDEV-26606. Test case added.
2021-10-06 23:56:06 +05:30
Rucha Deodhar
479e303ef3 MDEV-26606: ROW_NUMBER property value isn't passed from inside a stored
procedure

Analysis: m_current_row_for_warning is reset to 1 during cleanup phase of
stored procedure. When we perform a copy because some statement of procedure
created warning, this reset value is passed to push_warning().
Hence the output is always 1.
Fix: Add a parameter in relevant functions to pass correct value of
row_number and don't use m_current_row_for_warning directly.
2021-10-06 23:56:06 +05:30
Marko Mäkelä
25921c997e Merge 10.6 into 10.7 2021-10-06 10:28:59 +03:00
Marko Mäkelä
41c66ef6f7 Merge 10.5 into 10.6 2021-10-06 10:28:07 +03:00
Marko Mäkelä
4eb7217ec3 Merge 10.4 into 10.5 2021-10-06 09:45:12 +03:00
mkaruza
a75813d467 MDEV-22708 Assertion `!mysql_bin_log.is_open() || thd.is_current_stmt_binlog_format_row()' failed in Delayed_insert::handle_inserts and in Diagnostics_area::set_eof_status
Function `upgrade_lock_type` should check global binlog_format variable
instead of thread one.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-10-06 07:20:02 +03:00
Marko Mäkelä
48f1b6946e Update libmariadb 2021-10-06 07:09:24 +03:00
Marko Mäkelä
b7ff385d43 Merge 10.6 into 10.7 2021-10-05 19:28:18 +03:00
Marko Mäkelä
9791d4b4af After-merge fix: Correct s3.mysqldump result 2021-10-05 19:28:09 +03:00
Marko Mäkelä
c073c9a930 Merge 10.5 into 10.6 2021-10-05 17:04:51 +03:00
Marko Mäkelä
1146b5cb58 MDEV-26761: main.mysql_client_test fails with MemorySanitizer
Updated libmariadb and enable the test.
2021-10-05 16:49:30 +03:00
Marko Mäkelä
822066acfc Merge 10.6 into 10.7 2021-10-05 13:38:53 +03:00
Rucha Deodhar
d5e606c605 MDEV-26611: ERROR_INDEX isn't intuitively clear
Fixup for MDEV-10075

Analysis: ERROR_INDEX implemented in MDEV-10075 was not intuitively clear.

Fix: changed parser to use ROW_NUMBER instead of ERROR_INDEX. Removed
ERROR_INDEX and ERROR_INDEX_SYM from related files. Changed m_error_index
to m_row_number.
2021-10-05 12:44:55 +05:30
Marko Mäkelä
4ca56e8348 Merge 10.5 into 10.6 2021-10-05 09:21:34 +03:00
Marko Mäkelä
df94aa344b MDEV-26445 followup: Try to work around GCC 4.8.5 ICE on ARMv8
GCC 4.8.5 would crash when compiling trx_purge_truncate_history().
Let us try to avoid that by disabling optimizations for the function.
2021-10-05 07:13:14 +03:00
Marko Mäkelä
ead38354e6 Merge 10.4 into 10.5 2021-10-04 19:32:13 +03:00
Marko Mäkelä
f3bd278063 MDEV-22083/MDEV-26758: Fix uninitialized memory in mysql_client_test
The test was passing some uninitialized data to libmariadb.
Mostly, the MemorySanitizer wrapper of send() detected that
some bytes were uninitialized.

The test_mdev19838() is for now disabled under MemorySanitizer,
to be fixed in MDEV-26761.
2021-10-04 19:28:43 +03:00
Daniel Black
32839c4df2 MDEV-19867: postfix - syntax spelling ALTER TABLE
Fixes: 5c5ea59bf8
2021-10-04 22:31:18 +11:00
Daniel Black
e9d8002721 MDEV-25152: mysqldump - test fix for s3.mysqldump 2021-10-04 21:41:41 +11:00
Marko Mäkelä
097b7b8c9e Merge 10.3 into 10.4 2021-10-04 12:36:47 +03:00
Marko Mäkelä
1d57892949 Merge 10.2 into 10.3 2021-10-04 11:34:26 +03:00
Marko Mäkelä
d836f8a50d Work around MDEV-26754 main.sp test fails for embedded server 2021-10-04 11:28:57 +03:00
Vladislav Vaintroub
d28b118d7b Fix MSVC warning with bison 3.8.2 2021-10-03 16:49:54 +02:00
Marko Mäkelä
668a5f3d12 MDEV-26720: Optimize single-bit atomic operations on IA-32 and AMD64
This is mostly working around a bad compiler optimization.

The Intel 80386 processor introduced some bit operations that would be
the perfect translation for atomic single-bit read-modify-and-write
operations. Alas, even the latest compilers as of today
(GCC 11, clang 13, Microsoft Visual C 19.29) would generate a loop around
LOCK CMPXCHG instead of emitting the instructions
LOCK BTS (fetch_or()), LOCK BTR (fetch_and()), LOCK BTC (fetch_xor()).

fil_space_t::clear_closing(): Clear the CLOSING flag.

fil_space_t::set_stopping_check(): Special variant of
fil_space_t::set_stopping() that will return the old value
of the STOPPING flag after atomically setting it.

fil_space_t::clear_stopping(): Use fetch_sub() to toggle
the STOPPING flag. The flag is guaranteed to be set upon
calling this function, hence we will toggle it to clear it.
On IA-32 and AMD64, this will translate into
the 80486 LOCK XADD instruction.

fil_space_t::check_pending_operations(): Replace a Boolean
variable with a goto label, to allow more compact code
generation for fil_space_t::set_stopping_check().

trx_rseg_t: Define private accessors ref_set() and ref_reset()
for setting and clearing the flags.

trx_lock_t::clear_deadlock_victim(), trx_lock_t::set_wsrep_victim():
Accessors for clearing and setting the flags.
2021-10-03 13:49:40 +03:00
Marko Mäkelä
0144d1d2a6 MDEV-26720: rw_lock: Prefer fetch_sub() to fetch_and()
rw_lock::write_unlock(): Revert part of
commit d46b42489a (MDEV-24142)
to make the IA-32 and AMD64 implementation faster.
2021-10-02 11:29:44 +03:00
Marko Mäkelä
d301cc8edb Merge 10.5 into 10.6 2021-10-02 11:19:55 +03:00
Marko Mäkelä
98a7fa0ce7 MDEV-26720: optimize rw_lock for IA-32, AMD64 2021-10-02 11:14:14 +03:00
Marko Mäkelä
ec619a1def MDEV-26467 fixup: Prefer fetch_add() to fetch_or() on IA-32 and AMD64 2021-10-02 09:25:40 +03:00
mkaruza
86a2e2ba90 MDEV-22708 Assertion `!mysql_bin_log.is_open() || thd.is_current_stmt_binlog_format_row()' failed in Delayed_insert::handle_inserts and in Diagnostics_area::set_eof_status
Variable wsrep_forced_binlog_format has higher priority than
binlog_format. In situation where STATEMENT is used and DELAYED INSERT
is executing we should fall back to non-delay INSERT.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-10-01 11:38:14 +03:00
Daniel Black
4930209b12 MDEV-25152: Insert linebreaks in mysqldump --extended-insert
Per review by Serg, include start row with new line.

We are hoping we haven't annoyed people that prefered the old
way.

Adding an option for new lines seems like over-engineering in advance.
So if there are complaints, let them be known (JIRA), and we'll add
this under an option.

Test cases updated.
2021-10-01 11:03:47 +10:00
rmfalves
586d6a2520 MDEV-25152 Insert linebreaks in mysqldump --extended-insert
Currently, mysqldump --extended-insert outputs all rows on a single line,
which makes it difficult to use with various Unix command-line utilities
such as grep and diff.

We change mysqldump to emit a linebreak between each row, that would make
it easier to work with, without significantly affecting dump or restore
performance, or affecting compatibility.

Closes: #1865
2021-10-01 09:18:50 +10:00
mkaruza
2f5ae0da71 MDEV-25883 Galera Cluster hangs while "DELETE FROM mysql.wsrep_cluster"
Using `innodb_thread_concurrency` will call `wsrep_thd_is_aborting` to
check WSREP thread state. This call should be protected by taking
`LOCK_thd_data` before entering function.
Applier and TOI threads should no be affected with usage of
`innodb_thread_concurrency` variable so returning before any checks.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-09-30 12:25:26 +03:00
sjaakola
d5a15f04f4 MDEV-24978 crash with transaction on table with no PK and long fulltext column
If a table has no unique indexes, write set key information will be collected on all columns in the table.
The write set key information has space only for max 3500 bytes for individual column, and if a varchar colummn of such non-primary key table is longer than
 this limit, currently a crash follows.
The fix in this commit, is to truncate key values extracted from such long varhar columns to max 3500 bytes.
This may potentially lead to false positive certification failures for transactions, which operate on separate cluster nodes, and update/insert/delete table rows, which differ only in the part of such long columns after 3500 bytes border.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-09-30 11:19:34 +03:00
Marko Mäkelä
b36d6f92a8 Merge 10.6 into 10.7 2021-09-30 11:01:07 +03:00
Marko Mäkelä
a49e394399 Merge 10.5 into 10.6 2021-09-30 10:38:44 +03:00
Marko Mäkelä
be803f037f MDEV-25215 Excessive logging "InnoDB: Cannot close file"
In commit 45ed9dd957 (MDEV-23855)
when removing fil_system.LRU we failed to rate-limit the output
for reporting violations of innodb_open_files or open_files_limit.

If the server is run with a small limit of open files that is
well below the number of .ibd files that are being accessed by
the workload, and if at the same time innodb_log_file_size is
very small so that log checkpoints will occur frequently,
the process of enforcing the open files limit may be run very often.

fil_space_t::try_to_close(): Display at most one message per call,
and only if at least 5 seconds have elapsed since the last time a
message was output.

fil_node_open_file(): Only output a summary message if
fil_space_t::try_to_close() displayed a message during this run.
(Note: multiple threads may execute fil_node_open_file() on
different files at the same time.)

fil_space_t::get(): Do not dereference a null pointer if n & STOPPING.
This was caught by the test case below.

Unfortunately, it is not possible to create a fully deterministic
test case (expecting exactly 1 message to be emitted). The following with
--innodb-open-files=10 --innodb-log-file-size=4m
would occasionally fail to find the message in the log:

--source include/have_innodb.inc
--source include/have_partition.inc
--source include/have_sequence.inc

call mtr.add_suppression("InnoDB: innodb_open_files=10 is exceeded");

CREATE TABLE t1 (pk INT AUTO_INCREMENT PRIMARY KEY) ENGINE=InnoDB
PARTITION BY key (pk) PARTITIONS 100;

INSERT INTO t1 SELECT * FROM seq_1_to_100;
--disable_query_log
let $n=400;
while ($n)
{
BEGIN; DELETE FROM t1; ROLLBACK;
dec $n;
}
--enable_query_log

let SEARCH_FILE= $MYSQLTEST_VARDIR/log/mysqld.1.err;
let SEARCH_PATTERN= \[Note\] InnoDB: Cannot close file;
-- source include/search_pattern_in_file.inc

DROP TABLE t1;
2021-09-30 10:01:10 +03:00