Commit graph

189050 commits

Author SHA1 Message Date
Daniel Black
9bf3a3a47a HAVE_LARGE_PAGES no longer global 2020-04-05 17:20:50 +04:00
Daniel Black
71337a4452 my_large_malloc/free fall back to my_{malloc|free}_lock
For platforms != Windows and without MMAP.
2020-04-05 17:20:50 +04:00
Daniel Black
2a18e783ca my_large_page: more verbose errors on allocation fallback/failure 2020-04-05 17:20:50 +04:00
Daniel Black
d5568e7591 my_large_malloc_int consolidated into my_large_malloc
Consolidated single page size implementations by populating
my_large_page_sizes with a single page size.
2020-04-05 17:20:50 +04:00
Daniel Black
472d2d0492 my_large_free_int merge into my_large_free 2020-04-05 17:20:50 +04:00
Otto Kekäläinen
4ac7693678 Add author "Otto Kekäläinen" to SHOW AUTHORS 2020-04-05 00:24:24 +03:00
Otto Kekäläinen
b541defea0 Deb: Build depend on libcurl4-openssl-dev as primary option
The package libcurl4-dev is a virtual package, and no package with that
name actually exists. Depend on libcurl4-openssl-dev as the primary
option in case no libcurl4-dev providing package has been installed, and
if some of them (libcurl4-gnutls-dev, libcurl4-nss-dev,
libcurl4-openssl-dev ) are installed, they also satisfy the dependency
via the virtual package.

The Travis-CI still runs on libcurl3-dev as Travis-CI also runs Ubuntu
Xenial which does not yet have libcurl4 available.

This complements commit 2e0a40bdf0.
2020-04-05 00:24:24 +03:00
Otto Kekäläinen
6959c0922a MDEV-20298 Make mariadb-client-10.5 perl dep backwards compatible
The package libdbd-mariadb-perl has been available in Debian September
2018, so it is already included since Debian Buster and Ubuntu Disco.

Use it as the primary Recommends for MariaDB client binaries, but keep
the MySQL variant as a fall-back in case this package gets backported to
older releases that did not yet have it.

This change was done in downstream Debian packaging in
bb4ad78bc1
2020-04-05 00:24:24 +03:00
Otto Kekäläinen
333e1d82fc Deb: Update control file Depends to include only variables that are set
Don't include variables that are empty and unused.
2020-04-05 00:24:24 +03:00
Otto Kekäläinen
2cedf3eca4 MDEV-21303: Fix man page packaging for new mariadb-* named binaries
This fixes Lintian errors and warnings:

E: mariadb-client-10.5: manpage-not-compressed usr/share/man/man1/mariadb-analyze.1
E: mariadb-client-10.5: manpage-not-compressed usr/share/man/man1/mariadb-optimize.1
E: mariadb-client-10.5: manpage-not-compressed usr/share/man/man1/mariadb-repair.1
E: mariadb-client-10.5: manpage-not-compressed usr/share/man/man1/mysqlanalyze.1
E: mariadb-client-10.5: manpage-not-compressed usr/share/man/man1/mysqloptimize.1
E: mariadb-client-10.5: manpage-not-compressed usr/share/man/man1/mysqlrepair.1
W: mariadb-client-10.5: binary-without-manpage usr/bin/mariadb-conv
W: libmariadb-dev: binary-without-manpage usr/bin/mariadb_config

If the binary name is mariadb_config, the manpage needs to be mariadb_config.1.

Also run 'wrap-and-sort -a -v' to get lists in correct order.

In addition, sort .gitignore and remove duplicate lines.
2020-04-05 00:24:24 +03:00
Otto Kekäläinen
80abfa0eda Don't force to use all processors in autobake-deb.sh
The dpkg-buildpackage has by default sensible values for --jobs and
--try-jobs, and it also inherits whatever 'parallel' is set in
DEB_BUILD_OPTIONS. Overriding it here should not be needed and removing
it is better, since it allows build systems to control the level or
parallel builds via DEB_BUILD_OPTIONS if they want, which they often do
(e.g. to avoid CPU congestion or overheating laptops).
2020-04-04 20:42:00 +03:00
Otto Kekäläinen
1589cf1cb3 Deb: Remove TokuDB from Debian packages while keeping it easy to revert
Related:
- https://jira.mariadb.org/browse/MDEV-19780
- https://jira.mariadb.org/browse/MDEV-21944
2020-04-04 20:42:00 +03:00
Otto Kekäläinen
91994b6be4 Deb: Extend control dependencies for MySQL 8.0 and clean up
Make package have sensible control file relations to mysql-8.0.

Also some cleanup:
- Remove obsolete versioned control relationships. When the package
  name already has a version older than current package, the extra
  version is obsolete.
- Remove obsolete postinst script that was empty
- Remove priority "extra" which is deprecated, use "optional" that
  applies for the whole source package instead.
2020-04-04 20:42:00 +03:00
Otto Kekäläinen
400d7709ce Deb: Clean away deprecated autobake modifications
The current debian/control file and other packaging of 10.5 targets the
latest Debian unstable. In autobake-deb.sh there are automatic build
adaptations for older Debian/Ubuntu releases.

Some adaptations were for Debian versions prior to Jessie and Ubuntu
versions prior to Trusty. Those distro releases are no longer supported
anyway, so these should be cleaned away now in the 10.5 release cycle.

Summary:
- Debian Jessie has libcrack2 release 2.9.2-1, no need to have adaptations
  for any older environments
- Debian Jessie has libpcre3-dev release 2:8.35-3.3, no need to consider
  older versions
- Systemd has been available since Debian Jessie, no need to consider
  that a Debian/Ubuntu environment would not have it available
  adaptations for environments for anything older
2020-04-04 20:42:00 +03:00
Elena Stepanova
f469172380 Do not print LSAN suppression statistics in the output 2020-04-04 17:38:44 +03:00
Sergey Vojtovich
40c2cf335d MDEV-22146 - Server crashes in mysql_ha_flush_tables upon ALTER SERVER
Regression after bfdd30d. Initialize addition TABLE_LIST members which
are used by subsequent close_cached_tables() call.
2020-04-04 15:52:31 +04:00
Sergey Vojtovich
2e2138baa5 Simplified away my_get_large_page_size() 2020-04-04 14:55:51 +04:00
Sergey Vojtovich
30379b487f HAVE_LARGE_PAGE_OPTION to HAVE_LARGE_PAGES
It isn't an "option" anymore.
2020-04-04 14:55:51 +04:00
Sergey Vojtovich
4338bb8a75 Coding style fixes 2020-04-04 14:55:51 +04:00
Sergey Vojtovich
e4a960780f my_next_large_page_size() cleanup
Make it static, coding style cleanup, declare consistently with all
callers (same ifdefs).
2020-04-04 14:55:51 +04:00
Sergey Vojtovich
fc6e8b4b1b Cleanup HAVE_MMAP_ALIGNED: one cmake check less 2020-04-04 14:55:51 +04:00
Aleksey Midenkov
40f4d8c671 MDEV-21941 assertion cleanups 2020-04-04 01:07:24 +03:00
Aleksey Midenkov
105b879d0f MDEV-21941 RENAME doesn't work for system time or period fields
- Ignore system-invisible fields (as well as for setting default value);
- Handle rename of system time and period fields.
2020-04-04 00:53:37 +03:00
Aleksey Midenkov
431a740815 MDEV-21889 IF EXISTS clause does not work for RENAME COLUMN and RENAME INDEX 2020-04-04 00:53:36 +03:00
Aleksey Midenkov
6fb3e83d74 MDEV-21889 Typo fix: ER_KEY_DOES_NOT_EXISTS
libmariadb revision updated.
2020-04-04 00:52:54 +03:00
Sergey Vojtovich
d2c366c6fb Fix CentOS 6 and quantal build failures 2020-04-03 22:02:07 +04:00
Marko Mäkelä
1078a630f7 MDEV-22139 fseg_free_page_low() fails to write FREE_PAGE record, breaking recovery
This bug was introduced by MDEV-15528
commit a35b4ae898.

In the case that I analyzed, we failed to apply
an EXTENDED,INSERT_REUSE_REDUNDANT redo log record whose
preceding record points to unallocated area after PAGE_HEAP_TOP.

Had we properly written the FREE_PAGE record for the page,
recovery would have processed it, because during the checkpoint,
the log had been completely written past the LSN of the missed write.

fseg_free_page_low(): Always invoke mtr_t::free().
The other call is in fsp_free_page().
2020-04-03 17:47:56 +03:00
Daniel Black
abb0a31ec8 output_core_info - freebsd rlimits in different proc entry 2020-04-03 15:11:43 +02:00
Sergey Vojtovich
4197014ba0 Yet less TDC hash lookups
Let auto repair table and truncate table routines flush TABLE_SHARE
directly.

Part of MDEV-17882 - Cleanup refresh version
2020-04-03 13:01:21 +04:00
Sergey Vojtovich
7a947614fb Split tdc_remove_table()
TDC_RT_REMOVE_ALL -> tdc_remove_table(). Some occurrences replaced with
TDC_element::flush() (whenver TABLE_SHARE is available).

TDC_RT_REMOVE_NOT_OWN[_KEEP_SHARE] -> TDC_element::flush(). These modes
assume that current thread owns TABLE_SHARE reference, which means we can
avoid hash lookup and flush unused TABLE instances directly.

TDC_RT_REMOVE_UNUSED -> TDC_element::flush_unused(). Only [ab]used by
mysql_admin_table() currently. Should be removed eventually.

Part of MDEV-17882 - Cleanup refresh version
2020-04-03 13:01:21 +04:00
Sergey Vojtovich
06fae75859 tc_remove_all_unused_tables() cleanup
As tc_purge() never marks share flushed, let tdc_remove_table() do it
directly.

Part of MDEV-17882 - Cleanup refresh version
2020-04-03 13:01:21 +04:00
Sergey Vojtovich
14e1385691 Proper locking for mysql.gtid_slave_pos truncation
Aim of this patch is to remove tdc_remove_table(TDC_RT_REMOVE_UNUSED),
which was mistakenly introduced by 055a3334a.

InnoDB allows only one open TABLE instance while performing table
truncation. To fulfill this requirement:
1. MDL_EXCLUSIVE has to be acquired to block concurrent threads from
   accessing given table
2. cached TABLE instances have to be flushed
3. another InnoDB requirement is such that TABLE_SHARE and remaining
   TABLE instance have to be invalidated and re-opened after truncation

This goes more or less inline with what regular TRUNCATE TABLE does.

Alternative solution would be handler::ha_delete_all_rows(), but InnoDB
doesn't implement it unfortunately.

Part of MDEV-17882 - Cleanup refresh version
2020-04-03 13:01:21 +04:00
Sergey Vojtovich
bfdd30d3e9 Fixed close_cached_connection_tables() flushing
Let DROP SERVER and ALTER SERVER perform fair affected tables flushing.
That is acquire MDL_EXCLUSIVE and do tdc_remove_table(TDC_RT_REMOVE_ALL).

Aim of this patch is elimination of another inconsistent use of
TDC_RT_REMOVE_UNUSED. It fixes (to some extent) a problem described in the
beginning of sql_server.cc, when close_cached_connection_tables()
interferes with concurrent transaction.

A better fix should probably introduce proper MDL locks for server
objects?

Part of MDEV-17882 - Cleanup refresh version
2020-04-03 13:01:21 +04:00
Sergey Vojtovich
54c03cb4f0 Cleanup mysql_inplace_alter_table()
Removed redundant tdc_remove_table(TDC_RT_REMOVE_ALL). Share was marked
flushed by preceding wait_while_table_is_used() and eventually flushed by
close_all_tables_for_name().

Part of MDEV-17882 - Cleanup refresh version
2020-04-03 13:01:21 +04:00
Sergey Vojtovich
02619ed73b Cleanup close_all_tables_for_name()
close_all_tables_for_name() is always preceded by
wait_while_table_is_used(), which makes tdc_remove_table() redundant.
The only (now fixed) exception was close_cached_tables().

Part of MDEV-17882 - Cleanup refresh version
2020-04-03 13:01:21 +04:00
Sergey Vojtovich
e0743bd1a5 Let "FTWRL <table_list>" use extra(HA_EXTRA_FLUSH)
Rather than flushing caches with tdc_remove_table(TDC_RT_REMOVE_UNUSED)
flush them with extra(HA_EXTRA_FLUSH) instead. This goes inline with
regular FTWRL.

Part of MDEV-17882 - Cleanup refresh version
2020-04-03 13:01:21 +04:00
Marko Mäkelä
0870b75af7 MDEV-22126 Rename confusing constant mtr_t::OPT
The template parameter mtr_t::OPT refers to optional, not optimized.
Also the default parameter mtr_t::NORMAL refers to optimized writes.
The name MAYBE_NOP would be more descriptive, conveying the idea
that a write to a durable page might not actually have any effect.
2020-04-03 08:50:46 +03:00
Marko Mäkelä
406ca20b49 MDEV-22108 Recovery fails with InnoDB: Malformed log record
mtr_t::log_write(): Fix an off-by-one error.
2020-04-03 08:50:46 +03:00
Daniel Black
e8351934b6
Merge pull request from grooverdan/10.4-MDEV-18851-multiple-sized-large-page-support
MDEV-18851: multiple sized large page support (linux)
2020-04-02 23:54:08 +04:00
Sergei Golubchik
3bb5c6b0c2 MDEV-22113 SIGSEGV, ASAN use-after-poison, Assertion `next_insert_id == 0' in handler::ha_external_lock
if the lookup_handler is allocated on the THD's memroot, it may
not live long enough to be deleted in handler::ha_external_lock()
2020-04-02 14:03:54 +02:00
Daniel Black
a5686e14d9 MDEV-21303 Use mariadbd as the library plugins link to on non-Linux
Prevent errors like on FreeBSD

[ 56%] Linking CXX shared module ha_test_sql_discovery.so
cd /usr/home/dan/build-mariadb-server-10.5/storage/test_sql_discovery && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/test_sql_discovery.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC  -Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4 -O2 -g -DNDEBUG -D_FORTIFY_SOURCE=2 -DDBUG_OFF  -shared  -o ha_test_sql_discovery.so CMakeFiles/test_sql_discovery.dir/test_sql_discovery.cc.o -lpthread ../../libservices/libmysqlservices.a -lmysqld -lpthread
/usr/bin/ld: error: unable to find library -lmysqld

Also tested on OpenIndiana successfully.

Closes 
2020-04-02 14:03:54 +02:00
Rasmus Johansson
5f3501a6e5 MDEV-22120 Add author to SHOW AUTHORS 2020-04-02 10:54:31 +00:00
Otto Kekäläinen
a135f0ab88 Travis-CI: Update default distro from Ubuntu Xenial to Bionic
Ubuntu Bionic 18.04 LTS will be around until 2023 while Xenial is only to
2021. Thus Bionic is much more suitable as the Travis-CI default build
environment for the 10.5 branch.

For now, still keep one Xenial build to ensure backwards compatibility.

Disable main.udf test until MDEV-21976 is addressed.
2020-04-02 11:22:41 +03:00
Otto Kekäläinen
dda61ade11 Travis-CI: Slim down number of parallel jobs to speed up total run time
Number of jobs dropped from 23 -> 17 and build time from 8-10 hours to
3-4 hours.

Drop GCC 6 and Clang 6 from build dependencies and version 6 is dropped
from the jobs matrix.

Also ignore arm64 and ppc64le on Bionic builds since they sporadically
fail.
2020-04-02 11:22:41 +03:00
Marko Mäkelä
33ffcecc89 MDEV-22114 Assertion failure on SET GLOBAL innodb_buffer_pool_evict='uncompressed'
innodb_buffer_pool_evict_uncompressed(): Remove the mutex acquisition
that was inadvertently added in 1a6f708ec5.
2020-04-02 09:39:43 +03:00
Marko Mäkelä
b212f1dac2 MDEV-22107 Restore accidentally orphaned MTR_MEMO_MODIFY
In MDEV-12353, the calls to mtr_t::memo_modify_page()
were accidentally removed along with
mlog_open_and_write_index() and its callers.

Let us resurrect the function to enable better debug checks.

mtr_t::flag_modified(): Renamed from mtr_t::set_modified()
and made private.

mtr_t::set_modified(): Take const buf_block_t& as a parameter.

In several mtr_t member functions, replace const buf_page_t&
parameters with const buf_block_t&, so that we can pass the
parameter to set_modified().

mtr_t::modify(): Add a MTR_MEMO_MODIFY entry for a block that
is guaranteed to be modified in the mini-transaction.
2020-04-01 18:00:02 +03:00
Marko Mäkelä
587f3e0d9f MDEV-12353: Fix a warning in RelWithDebInfo build
page_zip_clear_rec(): Explicitly initialize len
in non-debug builds to avoid a warning about possibly
invalid memset() length.
2020-04-01 11:38:26 +03:00
Marko Mäkelä
51a9dd6793 Fix GCC 9.3.0 -Wstrict-aliasing
copy_keys_from_share(): Use reinterpret_cast instead of
manipulating a reference to a type-punned pointer.

This cleans up after the cleanup
commit 0515577d12.
2020-04-01 11:33:58 +03:00
Marko Mäkelä
abaeeffbf9 MDEV-22103 INNODB_ENCRYPTION_NUM_KEY_REQUESTS is missing from INFORMATION_SCHEMA.GLOBAL_STATUS
In commit a5584b13d1
some scrubbing-related status variables were removed along with
the background scrubbing code.
The status variable INNODB_ENCRYPTION_NUM_KEY_REQUESTS
was inadvertently removed as part of that.

innodb_status_variables[]: Restore "encryption_num_key_requests".

We introduce the test innodb.innodb_status_variables
in order to catch similar regressions in the future.
2020-04-01 11:19:12 +03:00
Marko Mäkelä
a1077ab287 MDEV-22102 Assertion w==OPT failed in trx_undo_header_create()
Because InnoDB is not freeing undo pages to the normal
free-page management, old undo log pages can be reused. Due to that,
it is possible (but unlikely) that the fields TRX_UNDO_NEEDS_PURGE
and TRX_UNDO_LOG_START relative to the free offset that is stored at
TRX_UNDO_PAGE_HDR + TRX_UNDO_PAGE_START already have the correct value.
Hence, we must pass the mtr_t::OPT template parameter to silence the
debug assertion.

Other writes in trx_undo_header_create() that are using the default
template parameter seem to be correct (the data fields should be
guaranteed to change even in the event of reusing pages).
2020-04-01 10:40:53 +03:00