If a query has many OR-ed constructs which can use multiple indexes
(key1=1 AND key2=10) OR
(key1=2 AND key2=20) OR
(key1=3 AND key2=30) OR
...
The range optimizer would construct and then discard a lot of potential
index_merge plans. This process
1. is CPU-intensive
2. can hit the @@optimizer_max_sel_args limitation after which all
potential range or index_merge plans are discarded.
The fix is to apply a heuristic: if there is an OR clause with more than
MAX_OR_ELEMENTS_FOR_INDEX_MERGE=100 branches (hard-coded constant),
disallow construction of index_merge plans for the OR branches.
In the `check_join_cache_usage()` function there is a branching issue
where an accidental fall-through to BKA/BKAH buffers may occur, even
when the join_cache_level setting does not permit their use.
This patch corrects the condition to ensure that BKA/BKAH join caching
is only enabled when explicitly allowed by join_cache_level
Reviewer: Sergei Petrunia <sergey@mariadb.com>
* `get_master_version_and_clock()` de-duplicate label using fall-through
* `io_slave_killed()` & `check_io_slave_killed()`:
* reüse the result from the level lower
* add distinguishing docs
* `try_to_reconnect()`: extract `'` from `if`-`else`
* `handle_slave_io()`: Both `while`s have the same condition;
looks like the outer `while` can simply be an `if`.
* `connect_to_master()`:
* assume `mysql_errno()` is not 0 on connection error
* utilize 0’s falsiness in the loop
* extend docs
* `sql/sql_show.cc`: refactor SHOW ALL REPLICAS filter’s condition
* `sql/mysqld.cc`: init `master-retry-count` with `master_retry_count`
Reviewed-by: Kristian Nielsen <knielsen@knielsen-hq.org>
When the IO thread (re)connect to a primary,
no updates are available besides unique errors that cause the failure.
These new `Master_info` numbers supplement SHOW SLAVE STATUS’s (most-
recent) ‘Connecting’ state with statistics on (re)connect attempts:
* `Connects_Tried`: how many retries have been attempted so far
This was previously a local variable that only counted re-attempts;
it’s now meaningful even after the “Connecting” state concludes.
* `Master_Retry_Count` (from MDEV-25674): out of how many configured
Side-note: Some of the tests updated by this commit dump the entire
SHOW SLAVE STATUS, which might include non-deterministic entries.
Reviewed-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
This new CHANGE MASTER TO field specifies the `--master-retry-count`
(global option: the number of Primary connection attempts)
for each multi-source replica; i.e, per-channel `performance_schema.`
`replication_connection_configuration.CONNECTION_RETRY_COUNT`.
`--master-retry-count` remains the default for new `CHANGE MASTER TO`s.
This new keyword and `master-info` entry
matches those of pre-‘REPLICATION SOURCE’ MySQL.
`try_to_reconnect()` wraps `safe_reconnect()` with logging, but the
latter already loops reconnection attempts up to `master_retry_count`
times with `mi->connect_retry`-msec sleeps inbetween.
This means `try_to_reconnect()` has been counting disconnects of the
replication session (since it doesn’t loop) while `safe_reconnect()`
was counting actual tries (which may be multiple per disconnect).
In practice, this outer counter’s only benefit was to override the edge
case `--master-retry-count=0` that the inner loop doesn’t cover with 1.
Added caching of database directories that did not have a db.opt file.
This was common for older MariaDB installaiton or if a user created
a database with 'mkdir'.
Other things:
- Give a note "no db.opt file" if one uses SHOW CREATE DATABASE one
a database without a db.opt file.
This fixes server startup segv introduced in MDEV-34716
d2eba35653 when upgrading from server
versions lower than 11.7.
Also construct the JSON Options column when upgrading from a version
without the column.
This fixes compilation when using gcc 7.5.0
Apparantly this version of gcc does not support
enum privilege_t: unsigned long long forr printf
argument checking.
The fix for MDEV-35318 has introduced LEX::needs_reprepare and logic in
Prepared_statement::execute_loop() and sp_lex_keeper::
validate_lex_and_exec_core() to re-prepare the statement if it has hit
an error when doing once-per-statement-life optimizations.
But there is also third code path: PS with Array Binding is handled in
Prepared_statement::execute_bulk_loop(). Add handling there as well.
Migrate `mysys/errors.c`, `sql-common/errmsg.c` and a couple of
insignificant loose ends to use suffix-based, `-Wformat`-compatible
`my_snprintf` format extensions introduced in MDEV-21978
This commit is the final batch of MDEV-21978’s migration process.
While GCC `-Wformat` (with `ATTRIBUTE_FORMAT`) can catch obsolete or
malformed format string literals, formats originating from other sources
(such as those strings headers) (still) require manual review.
Thus, after all the automatic `-Wformat` complaints fixed in previous
commits, I’ve done a manual `grep` and caught these final matches.
* Migrate `sql/share/errmsg-utf8.txt` to use suffix-based, `-Wformat`
-compatible `my_snprintf` format extensions introduced in MDEV-21978
* Update relevant tests caught by BuildBot as well
While GCC `-Wformat` (with `ATTRIBUTE_FORMAT`) can catch obsolete or
malformed format string literals, formats originating from other sources
(such as this translations file) (still) require manual review.
This commit also escapes the only (1) instance of existing strings
conflicted by the introduction of suffixes:
(Not all `printf`s goes to `my_snprintf`, thus I `grep`ped and
confirmed that this does indeed land on `my_snprintf` eventually.)
chi "不能%sSLAVE'%.*s'"
This commit also fixes the following: (You’re welcome.)
* Delete extraneous spaces after the `%` (they’re all Swahili)
* Update `extra/comp_err.c`
* Add the missing standard C/C++ specifiers `c`, `i`, `o`, `p` and `X`
(Especially `%i`: it otherwise was complaining about the new `%iE`)
* Removed the old and obsolete extension formats `%b`, `%M` and `%T`
This commit is the final batch of #3360’s `ATTRIBUTE_FORMAT` process,
covering various insignificant (as in, requires few-to-no
changes in addition to gaining this attribute) functions.
One of the main focus of this PR is to enable GCC `-Wformat` (by tagging
`ATTRIBUTE_FORMAT`) on ALL `my_snprintf` utilities. To be throughout,
functions that delegate to `my_vsnprintf` must also inherit this
attribute because `-Wformat` doesn’t trace argument across call stacks.
The function pointer typedef `my_error_reporter` is already tagged.
This commit inherits this attribute to all `my_getopt_error_reporter`s
and `my_charset_error_reporter`s for consistency.
(It future-proofs for deliberate direct uses of those functions.)
(Re)ënable the `ATTRIBUTE_FORMAT` on `my_dbug.h`’s `_db_doprnt_`
(better known by its frontend `DBUG_PRINT`) and `ma_recovery_util.h`’s
`tprint` & `eprint` to leverage GCC `-Wformat` checking
c4bf4b7aef introduced `WAITING_FOR_BUGFIX_TO_VSPRINTF` to conditionally
(read: temporarily) disable their `ATTRIBUTE_FORMAT`s.
Whatever that bug was aside, MDEV-21978 Zulip suggested that the
preference for `%b` was probably intended, although c52e62a76f
reverted the one in `storage/maria/ma_recovery.c` back to `%s`.
All extension migrations (e.g., `%b` ➡ `%sB`)
in this commit were on `DBUG_PRINT`.
* Let GCC `-Wformat` check formats sent to these `my_vsnprintf_ex` users
* Migrate them from the old extension specifiers
to the new `-Wformat`-compatible suffixes
Problem was missing case from wsrep_handle_mdl_conflict. Test case
was trying to confirm that LOCK TABLE thread is not BF-aborted.
However as case was missing it was BF-aborted. Test case passed
because BF-aborting takes time and used wait condition might
see expected thread status before it was BF-aborted. Test naturally
failed if BF-aborting was done early enough.
Fix is to add missing case for SQLCOM_LOCK_TABLES to
wsrep_handle_mdl_conflict.
Note that using LOCK TABLE is still not recomended on cluster
because it could cause cluster hang.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
[Breaking]
The `my_print_error` service passes formats and args directly
to `my_vsnprintf`. Just like the `my_snprintf` service,
I increased this service’s major version because:
* Custom suffixes are now a thing
(and custom specifiers will soon no longer be).
* GCC `-Wformat` now checks formats sent to them.
[Breaking]
Good news:
GCC now checks your `my_snprintf` (direct) calls (`-Wformat` was on);
Bad news:
The build process no longer lets your incorrect
formats/arguments sneak past (`-Werror` was also on).
As such, this commit also migrates all direct `my_snprintf` calls from
the old specifiers to MDEV-21978’s new `-Wformat`-compatible suffixes.
The next commits will cover non-direct calls to `my_snprintf`.
(I call them “`my_snprintf` descendants”.)
This commit does not update the ABI records because
there’re more ABI “changes” to come – half a dozen
`include/mysql/plugin_*.h.pp`s are now missing the new `__attribute__`s.
Followup to fix for MDEV-35318: In Prepared_statement::execute_loop(),
if we enter this function with lex->needs_reprepare=true, we need to
re-prepare the statement.
We also need to call set_parameters() to get PS parameter values. We
failed to do this and that's the bug.
Note that we need to also handle the other reprepare scenario: when
we get into Prepared_statement::execute_loop(), call set_parameters()
and then hit a "DDL was changed" error with reprepare_observer,
we should NOT call set_parameters() again.
The parameters have already been transferred to the reprepared statement
in Prepared_statement::reprepare()|swap_parameter_array().
(Variant 3) (commit in 11.4)
When a derived table has a GROUP BY clause:
SELECT ...
FROM (SELECT ... GROUP BY col1, col2) AS tbl
The optimizer would use inner join's output cardinality as an estimate
of derived table size, ignoring the fact that GROUP BY operation would
produce much fewer groups.
Add code to produce tighter bounds:
- The GROUP BY list is split into per-table lists. If GROUP BY list has
expressions that refer to multiple tables, we fall back to join output
cardinality.
- For each table, the first cardinality estimate is join_tab->read_records.
- Then, we try to get a tighter bound by using index statistics.
- If indexes do not cover all GROUP BY columns, we try to use per-column
EITS statistics.
Backport of commit 74f70c3944 to 10.11.
The new logic is disabled by default, to enable, use
optimizer_adjust_secondary_key_costs=fix_derived_table_read_cost.
== Original commit comment ==
Fixed costs in JOIN_TAB::estimate_scan_time() and HEAP
Estimate_scan_time() calculates the cost of scanning a derivied table.
The old code did not take into account that the temporary table heap table
may be converted to Aria.
Things fixed:
- Added checking if the temporary tables data will fit in the heap.
If not, then calculate the cost based on the designated internal
temporary table engine (Aria).
- Removed MY_MAX(records, 1000) and instead trust the optimizer's
estimate of records. This reduces the cost of temporary tables a bit
for small tables, which caused a few changes in mtr results.
- Fixed cost calculation for HEAP.
- HEAP costs->row_next_find_cost was not set. This does not affect old
costs calculation as this cost slot was not used anywhere.
Now HEAP cost->row_next_find_cost is set, which allowed me to remove
some duplicated computation in ha_heap::scan_time()
MDEV-35958 Cost estimates for materialized derived tables are poor
(Backport 11.8->11.4, the same patch)
Estimate_scan_time() calculates the cost of scanning a derivied table.
The old code did not take into account that the temporary table heap table
may be converted to Aria.
Things fixed:
- Added checking if the temporary tables data will fit in the heap.
If not, then calculate the cost based on the designated internal
temporary table engine (Aria).
- Removed MY_MAX(records, 1000) and instead trust the optimizer's
estimate of records. This reduces the cost of temporary tables a bit
for small tables, which caused a few changes in mtr results.
- Fixed cost calculation for HEAP.
- HEAP costs->row_next_find_cost was not set. This does not affect old
costs calculation as this cost slot was not used anywhere.
Now HEAP cost->row_next_find_cost is set, which allowed me to remove
some duplicated computation in ha_heap::scan_time()
Reviewed by: Sergei Petrunia <sergey@mariadb.com>
Propagate discard/import tablespace request to hlindexes.
Let FLUSH TABLES ... FOR EXPORT open/lock hlindexes, so that InnoDB
prepares hlindexes for export.
Moved reset_hlindexes() to external_lock(F_UNLCK), so that hlindexes
are available for export until UNLOCK TABLES.
Closes#3631
It was not possible to use a package body variable as a
fetch target:
CREATE PACKAGE BODY pkg AS
vc INT := 0;
FUNCTION f1 RETURN INT AS
CURSOR cur IS SELECT 1 AS c FROM DUAL;
BEGIN
OPEN cur;
FETCH cur INTO vc; -- this returned "Undeclared variable: vc" error.
CLOSE cur;
RETURN vc;
END;
END;
FETCH assumed that all fetch targets reside of the same sp_rcontext
instance with the cursor. This patch fixes the problem.
Now a cursor and its fetch target can reside in different sp_rcontext
instances.
Details:
- Adding a helper class sp_rcontext_addr
(a combination of Sp_rcontext_handler pointer and an offset in the rcontext)
- Adding a new class sp_fetch_target deriving from sp_rcontext_addr.
Fetch targets in "FETCH cur INTO target1, target2 ..." are now collected
into this structure instead of sp_variable.
sp_variable cannot be used any more to store fetch targets,
because it does not have a pointer to Sp_rcontext_handler
(it only has the current rcontext offset).
- Removing members sp_instr_set members m_rcontext_handler and m_offset.
Deriving sp_instr_set from sp_rcontext_addr instead.
- Renaming sp_instr_cfetch member "List<sp_variable> m_varlist"
to "List<sp_fetch_target> m_fetch_target_list".
- Fixing LEX::sp_add_cfetch() to return the pointer to the
created sp_fetch_target instance (instead of returning bool).
This helps to make the grammar in sql_yacc.c simpler
- Renaming LEX::sp_add_cfetch() to LEX::sp_add_instr_cfetch(),
as `if(sp_add_cfetch())` changed its meaning to the opposite,
to avoid automatic wrong merge from earlier versions.
- Chaning the "List<sp_variable> *vars" parameter to sp_cursor::fetch
to have the data type "List<sp_fetch_target> *".
- Changing the data type of "List<sp_variable> &vars" in
sp_cursor::Select_fetch_into_spvars::send_data_to_variable_list()
to "List<sp_fetch_target> &".
- Adding THD helper methods get_rcontext() and get_variable().
- Moving the code from sql_yacc.yy into a new LEX method
LEX::make_fetch_target().
- Simplifying the grammar in sql_yacc.yy using the new LEX method.
Changing the data type of the bison rule sp_fetch_list from "void"
to "List<sp_fetch_target> *".
MDEV-35958 Cost estimates for materialized derived tables are poor
Estimate_scan_time() calculates the cost of scanning a derivied table.
The old code did not take into account that the temporary table heap table
may be converted to Aria.
Things fixed:
- Added checking if the temporary tables data will fit in the heap.
If not, then calculate the cost based on the designated internal
temporary table engine (Aria).
- Removed MY_MAX(records, 1000) and instead trust the optimizer's
estimate of records. This reduces the cost of temporary tables a bit
for small tables, which caused a few changes in mtr results.
- Fixed cost calculation for HEAP.
- HEAP costs->row_next_find_cost was not set. This does not affect old
costs calculation as this cost slot was not used anywhere.
Now HEAP cost->row_next_find_cost is set, which allowed me to remove
some duplicated computation in ha_heap::scan_time()
Reviewed by: Sergei Petrunia <sergey@mariadb.com>
a start node in the shared context must be atomically assigned
to a valid and fully initialized node otherwise a concurrent
thread might use it before it's safe
* rpl.rpl_system_versioning_partitions updated for MDEV-32188
* innodb.row_size_error_log_warnings_3 changed error for MDEV-33658
(checks are done in a different order)
Reintroduces delete_while_scanning optimization for multi_delete.
Reverse some test changes from the initial feature devlopment now
that we delete-on-the-fly once again.
