MariaDB Compression pluging pacakages mariadb-plugin-provider-*
have only one shared object and those are not linked against
libc and it's intentional so supressing error
library-not-linked-against-libc
There is needed dependency for Systemd DH plugin which
makes error missing-build-dependency-for-dh-addon obsolette.
Reworked debian/control that makes most of
version-substvar-for-external-package error not correct
so remove those that are not available anymore and update
those that are still relevant.
NOTE TO MERGER: This is only up-to MariaDB 10.10.
Lintian erros with
missing-build-dependency-for-dh-addon systemd (*)
One of these premises should be installed:
debhelper:any (>= 9.20160709~)
debhelper-compat:any
dh-sequence-systemd:any
dh-systemd:any
As there is package debhelper version 10 or higher required
for build. This is false positive Lintian error which should
be supressed..
NOTE TO MERGER: This is only up-to MariaDB 10.10.
Salsa-CI file has got in malfunction state and
if fails with YAML error:
mariadb-10.3 with Buster backports upgrade job:
undefined need: build buster-backports
Also remove Salsa-CI MariaDB 10.3 double
upgrade target.
There is also several upgrades which makes
test green again.
NOTE TO MERGER: This is only up-to MariaDB 10.10.
Remove
In debian/control file there is several mariadb-*-10.9 Conflicts
or Replaces that which are not neede on MariaDB 10.9 anymore
as there is not suffix anymore.
Package libmariadbclient-dev is part of Ubuntu and it's
not build with official package so there can't be
variable: '${source:Version}' which is used with packages
that are part of package.
NOTE TO MERGER: This is only up-to MariaDB 10.10.
recv_log_recover_10_5(): Make reads aligned by 4096 bytes, to avoid
any trouble in case the file was opened in O_DIRECT mode and
the physical block size is larger than 512 bytes.
Because innodb_log_file_size used to be defined in whole megabytes,
reading multiples of 4096 bytes instead of 512 should not be an issue.
The new statistics is enabled by adding the "engine", "innodb" or "full"
option to --log-slow-verbosity
Example output:
# Pages_accessed: 184 Pages_read: 95 Pages_updated: 0 Old_rows_read: 1
# Pages_read_time: 17.0204 Engine_time: 248.1297
Page_read_time is time doing physical reads inside a storage engine.
(Writes cannot be tracked as these are usually done in the background).
Engine_time is the time spent inside the storage engine for the full
duration of the read/write/update calls. It uses the same code as
'analyze statement' for calculating the time spent.
The engine statistics is done with a generic interface that should be
easy for any engine to use. It can also easily be extended to provide
even more statistics.
Currently only InnoDB has counters for Pages_% and Undo_% status.
Engine_time works for all engines.
Implementation details:
class ha_handler_stats holds all engine stats. This class is included
in handler and THD classes.
While a query is running, all statistics is updated in the handler. In
close_thread_tables() the statistics is added to the THD.
handler::handler_stats is a pointer to where statistics should be
collected. This is set to point to handler::active_handler_stats if
stats are requested. If not, it is set to 0.
handler_stats has also an element, 'active' that is 1 if stats are
requested. This is to allow engines to avoid doing any 'if's while
updating the statistics.
Cloned or partition tables have the pointer set to the base table if
status are requested.
There is a small performance impact when using --log-slow-verbosity=engine:
- All engine calls in 'select' will be timed.
- IO calls for InnoDB reads will be timed.
- Incrementation of counters are done on local variables and accesses
are inline, so these should have very little impact.
- Statistics has to be reset for each statement for the THD and each
used handler. This is only 40 bytes, which should be neglectable.
- For partition tables we have to loop over all partitions to update
the handler_status as part of table_init(). Can be optimized in the
future to only do this is log-slow-verbosity changes. For this to work
we have to update handler_status for all opened partitions and
also for all partitions opened in the future.
Other things:
- Added options 'engine' and 'full' to log-slow-verbosity.
- Some of the new files in the test suite comes from Percona server, which
has similar status information.
- buf_page_optimistic_get(): Do not increment any counter, since we are
only validating a pointer, not performing any buf_pool.page_hash lookup.
- Added THD argument to save_explain_data_intern().
- Switched arguments for save_explain_.*_data() to have
always THD first (generates better code as other functions also have THD
first).
on aarch64 `char` by default is unsigned for performance reasons.
let's adjust checks to work for both signed and unsigned `char`
followup for ef84f8137b
optimizer implicitly assumed that if `a` in `a=b` is not NULL,
then it's safe to convert `a` to the type of `b` and search the
result in the index(b).
which is not always the case, as converting a non-null value
to a different type might produce NULL. And searching for NULL
in the index might find NULL there, so NULL will be equal to NULL,
making `a=b` behave as if it was `a<=>b`
* UUIDs version >= 6 are now stored without byte-swapping
* UUIDs with version >=8 and variant=0 are now considered invalid
* old tables are supported
* old (always byte swapped) and new (swapped for version < 6) UUIDs
can be compared and converted transparently
fil_node_open_file_low(): Always acquire an advisory lock on
the system tablespace. Originally, we already did this in
SysTablespace::open_file(), but SysTablespace::open_or_create()
would release those locks when it is closing the file handles.
This is a 10.5+ specific follow up to
commit 0ee1082bd2 (MDEV-28495).
Thanks to Daniel Black for verifying this bug.
When InnoDB attempts to buffer a change operation of a secondary index
leaf page (to insert, delete-mark or remove a record) and the
change buffer is too large, InnoDB used to trigger a change buffer merge
that could affect any tables. This could lead to huge variance in
system throughput and potentially unpredictable crashes, in case the
change buffer was corrupted and a crash occurred while attempting to
merge changes to a table that is not being accessed by the current
SQL statement.
ibuf_insert_low(): Simply return DB_STRONG_FAIL when the maximum size
of the change buffer is exceeded.
ibuf_contract_after_insert(): Remove.
ibuf_get_merge_page_nos_func(): Remove a constant parameter.
The function ibuf_contract() will be our only caller, during
shutdown with innodb_fast_shutdown=0.
debug-only issue. the test was doing
set debug_sync='now SIGNAL go3';
...
set debug_sync='reset';
which translated into
add "go3" to the hash of active signals
pthread_broadcast to wake up waiting threads
...
clear the hash of active signals
as a result a waiting thread was awoken, but the hash was emptied
before the thread checked if its signal was in the hash. so the
thread didn't find its signal and went back to sleep.
let's wait until the awoken thread has completely finished
disconnecting and was added to the thread cache.
debug_sync refactoring introduced a statically instantiated object
debug_sync_global of the structure st_debug_sync_globals.
st_debug_sync_globals includes Hash_set<> which allocates memory
in the constructor. sf_malloc() calls _my_thread_var()->dbug_id
which is pthread_getspecific(THR_KEY_mysys), and THR_KEY_mysys is 0
before pthread_key_create(). pthread_getspecific(0) returns a valid
pointer, not EINVAL. And safemalloc dereferences it.
let's statically initialize THR_KEY_mysys to -1, this makes
pthread_getspecific(THR_KEY_mysys) to fail before pthread_key_create()
is called.
followup for 8885225de6
file /usr/lib64/libmariadb.so.3 from install of MariaDB-shared-10.11.3-1.fc38.x86_64 conflicts with file from package mariadb-connector-c-3.3.5-1.fc38.x86_64
log_t::write_checkpoint(), log_t::resize_start(): Invoke buf_flush_ahead()
with buf_pool.get_oldest_modification(0)+1 so that another checkpoint will
be invoked, to complete the log resizing.
Tested by: Oleg Smirnov (on Microsoft Windows, where this hung most often)
ha_innobase::delete_table(): Also on DROP SEQUENCE, do try to drop any
persistent statistics. They should really not be created for
SEQUENCE objects (which internally are 1-row no-rollback tables),
but that is how happened to always work.
i_s_innodb_buffer_page_get_info(): Correct a condition.
After crash recovery, there may be some buffer pool pages in FREED state,
containing garbage (invalid data page contents). Let us ignore such pages
in the INFORMATION_SCHEMA output.
The test innodb.innodb_defragment_fill_factor will be removed, because
the queries that it is invoking on information_schema.innodb_buffer_page
would start to fail. The defragmentation feature was removed in
commit 7ca89af6f8 in MariaDB Server 11.1.
Tested by: Matthias Leich
btr_search_hash_table_validate(), btr_search_validate(): Add the
parameter THD for checking if the statement has been killed.
Any non-QUICK CHECK TABLE will validate the entire adaptive hash index
for all InnoDB tables, which may be extremely slow when running
multiple concurrent CHECK TABLE.
During the upgrade procedure on Windows mysqld.exe is started with
the named pipe connection protocol. mysqladmin.exe then pings the
server to check if is up and running. Command line looks like:
mysqladmin.exe --protocol=pipe --socket=mysql_upgrade_service_xxx ping
But the "socket" parameter resets the "protocol" which was previously
initialized with the "pipe" value, setting it to "socket".
As a result, connection cannot be established and the upgrade
procedure fails.
"socket" in Windows is used to pass the name of the pipe so resetting
the protocol is not valid in this case.
This commit fixes resetting of the "protocol" parameter with "socket"
parameter in the case when protocol has been previously initialized
to "pipe" value
When CURSOR parameters get parsed, their sp_assignment_lex instances
(one instance per parameter) get collected to List<sp_assignment_lex>.
These instances get linked to sphead only in the end of the list.
If a syntax error happened in the middle of the parameter list,
these instances were not deleted, which caused memory leaks.
Fix:
using a Bison %destructor to free rules of the <sp_assignment_lex_list>
type (on syntax errors).
Afte the fix these sp_assignment_lex instances from CURSOR parameters
deleted as follows:
- If the CURSOR statement was fully parsed, then these instances
get properly linked to sp_head structures, so they are deleted
during ~sp_head (this did not change)
- If the CURSOR statement failed on a syntax error, then by Bison's
%destructor (this is being added in the current patch).
During the upgrade procedure on Windows mysqld.exe is started with
the named pipe connection protocol. mysqladmin.exe then pings the
server to check if is up and running. Command line looks like:
mysqladmin.exe --protocol=pipe --socket=mysql_upgrade_service_xxx ping
But the "socket" parameter resets the "protocol" which was previously
initialized with the "pipe" value, setting it to "socket".
As a result, connection cannot be established and the upgrade
procedure fails.
"socket" in Windows is used to pass the name of the pipe so resetting
the protocol is not valid in this case.
This commit fixes resetting of the "protocol" parameter with "socket"
parameter in the case when protocol has been previously initialized
to "pipe" value
The parser works as follows:
The rule expr_lex returns a pointer to a newly created sp_expr_lex
instance which is not linked to any MariaDB structures yet - it is
pointed only from a Bison stack variable. The sp_expr_lex instance
gets linked to other structures (such as sp_instr_jump_if_not) later,
after scanning some following grammar.
Problem before the fix:
If a parse error happened immediately after expr_lex (before it got linked),
the created sp_expr_lex value got lost causing a memory leak.
Fix:
- Using Bison's "destructor" directive to free the results of expr_lex
on parse/oom errors.
- Moving the call for LEX::cleanup_lex_after_parse_error() from
MYSQL_YYABORT and yyerror inside parse_sql().
This is needed because Bison calls destructors after yyerror(),
while it's important to delete the sp_expr_lex instance before
LEX::cleanup_lex_after_parse_error().
The latter frees the memory root containing the sp_expr_lex instance.
After this change the code block are executed in the following order:
- yyerror() -- now only raises the error to DA (no cleanup done any more)
- %destructor { delete $$; } <expr_lex> -- destructs the sp_expr_lex instance
- LEX::cleanup_lex_after_parse_error() -- frees the memory root containing
the sp_expr_lex instance
- Removing the "delete sublex" related code from restore_lex():
- restore_lex() is called in most cases on success, when delete is not needed.
- There is one place when restore_lex() is called on error:
In sp_create_assignment_instr(). But in this case LEX::sp_lex_in_use
is true anyway.
The patch adds a new DBUG_ASSERT(lex->sp_lex_in_use) to guard this.
The fix is in replacing the waiting for the whole purge finishing
with the the waiting for only delete-marked records purging finishing.
Reviewed by: Marko Mäkelä
- InnoDB bulk insert operation aborts the server for redundant
table. InnoDB miscalculates the record size in temporary file
for the redundant table. CHAR in redundant row format table
always fixed length, but in temporary file, it is variable-length
for variable-length character sets.
The issue is caused by MDEV-30400 fix.
There are two cursors in btr_estimate_n_rows_in_range() - p1 and p2, but
both share the same mtr. Each cursor contains mtr savepoint for the
previously fetched block to release it then the current block is
fetched.
Before MDEV-30400 the block was released with
mtr_t::release_block_at_savepoint(), it just unfixed a block and
released its page patch. In MDEV-30400 it was replaced with
mtr_t::rollback_to_savepoint(), which does the same as the former
mtr_t::release_block_at_savepoint(ulint begin, ulint end) but also
erases the corresponding slots from mtr memo, what invalidates any
stored mtr's memo savepoints, greater or equal to "begin".
The idea of the fix is to get rid of savepoints at all in
btr_estimate_n_rows_in_range() and
btr_estimate_n_rows_in_range_on_level(). As
mtr_t::rollback_to_savepoint() erases elements from mtr_t::m_memo, we
know what element of mtr_t::m_memo can be deleted on the certain case,
so there is no need to store savepoints.
See also the following slides for details:
https://docs.google.com/presentation/d/1RFYBo7EUhM22ab3GOYctv3j_3yC0vHtBY9auObZec8U
Reviewed by: Marko Mäkelä
