Commit graph

189109 commits

Author SHA1 Message Date
Otto Kekäläinen
8c41a1df86 MDEV-21869: Add temporary Lintian overrides
With these in place Lintian will pass without errors (only warnings) and
the CI systems will at least be able to stop anybody from introducing
new Lintian errors.
2020-04-19 17:15:40 +03:00
Otto Kekäläinen
29c5d96da3 MDEV-21869: Make Debian packaging Lintian clean
Fix Lintian complaints on:
 P: maintainer-script-without-set-e
 W: mariadb-server-10.5: script-not-executable usr/share/mysql/mysql.init
 W: mariadb-client-10.5: incorrect-path-for-interpreter usr/bin/innotop
    (#!/usr/bin/env perl != /usr/bin/perl)
2020-04-19 17:15:40 +03:00
Otto Kekäläinen
a4c5480525 MDEV-6284: Sync correct Debian handling of auth_pam_tool
Manages the security risk in way that also fixes Lintian warning:
  W: mariadb-server-10.5: setuid-binary
  usr/lib/mysql/plugin/auth_pam_tool_dir/auth_pam_tool 4755 root/root

Applied downstream in
9605a48a99
2020-04-19 17:15:40 +03:00
Otto Kekäläinen
4e946b0f0c MDEV-6284: Sync mariadb-server maintainer scripts as much as possible
There are still some differences in due to systemd and service triggers,
but these differences make sense and are likely to be synced the other
way later so that downstream Debian official packaging adopts them.

The synced changes include among others:
- 9f49e4b494
- 6440c0d6e7
- 6e5ee72d64
- e62e67ae4b
- df2415a53d
- 643558da74
2020-04-19 17:15:40 +03:00
Otto Kekäläinen
9ed7e967b3 MDEV-6284: Sync breaks/replaces relationships and file locations
These changes are in line with what has been changes in downstream Debian
and they fix the following errors:

 $ check_for_missing_breaks_replaces.py -o ${WORKING_DIR}/missing_breaks.xml --changes-file ${WORKING_DIR}/*.changes
 [ERROR] mariadb-client-10.5 conflicts with mytop files: {'/usr/bin/mytop'}
 [ERROR] mariadb-client-10.5 conflicts with mariadb-client-core-10.3 files: {'/usr/bin/mysqlcheck'}
 [ERROR] mariadb-server-10.5 conflicts with mariadb-server-core-10.3 files: {'/usr/share/man/man1/resolveip.1.gz', '/usr/bin/resolveip', '/usr/share/mysql/mysql_test_data_timezone.sql'}
 [ERROR] mariadb-client-10.5 conflicts with mysql-client-core-5.7 files: {'/usr/bin/mysqlcheck'} [ERROR] mariadb-server-10.5 conflicts with mariadb-server-core-10.3 files: {'/usr/share/man/man1/resolveip.1.gz', '/usr/bin/resolveip', '/usr/share/mysql/mysql_test_data_timezone.sql'}
 [ERROR] mariadb-server-10.5 conflicts with mysql-client-5.7 files: {'/usr/bin/myisam_ftdump', '/usr/share/man/man1/myisam_ftdump.1.gz'}
 [ERROR] mariadb-server-core-10.5 conflicts with mysql-server-5.7 files: {'/usr/share/mysql/mysql_test_data_timezone.sql', '/usr/bin/resolveip', '/usr/share/man/man1/resolveip.1.gz'}
 [ERROR] mariadb-test conflicts with mysql-server-5.7 files: {'/usr/lib/mysql/plugin/adt_null.so', '/usr/lib/mysql/plugin/mypluglib.so'}
2020-04-19 17:15:40 +03:00
Otto Kekäläinen
7a0eeaaf66 MDEV-6284: Sync deb build rules etc with downstream
- Ensure cmake builds also apply CPPFLAGS flags for hardening to fully work
  fc4f33cf40
- Install Ubuntu Apport files in the same way with same filename.
- MDEV-21705: Build flags to keep WolfSSL from crashing
- Clean away sql-bench from packaging immediately after build step.
- Delete private files from libraries so they don't get shipped in the -dev
  packages.
2020-04-19 17:15:40 +03:00
Otto Kekäläinen
bc11f392f8 MDEV-6284: Correctly place files in libmariadb-dev, libmariadbd-dev etc packages
- Make full contents, also header files, explicit in -dev packages
  fb77df0c75
- Don't ship private header files at all.
- Ship both libmariadb.pc and maridb.pc.
- Ship mysql_config only as a symlink.
- Keep 'mariadbcheck' as symlink as somebody could have used it already
  d0466f74d5
- The mariadb.pc is the server pkg-config, ship in libmariadbd-dev.
- The client pkg-config is called libmariadb.pc. Ship only it in the client
  library packages.

Use versioned breaks/replaces for mysql* packages we also provide and fix:

  [ERROR] libmariadb-dev conflicts with libmysqlclient-dev files:
    {'/usr/share/man/man1/mysql_config.1.gz', '/usr/share/aclocal/mysql.m4'}
  [ERROR] libmariadb-dev conflicts with libmysqld-dev files:
    {'/usr/lib/x86_64-linux-gnu/libmysqlservices.a'}
2020-04-19 17:15:40 +03:00
Otto Kekäläinen
5cdf245d7e MDEV-6284: Sync deb build dependencies and control file with downstream
- Remove versioned depends that are far outdated and not relevant anymore
  in 2020.
- Link with libedit instead of unmaintained readline5
  d8796a0e3b
- Add [linux-any] for Linux specific depends and other smaller control
  file tweaks.
- Update .travis-ci.yml to match.
- Also fixes the Debian autopkgtest detected error:
    ERROR 1231 (42000) at line 1: Variable 'innodb_compression_algorithm'
    can't be set to the value of 'lz4'
2020-04-19 17:15:40 +03:00
Faustin Lammler
6af0bd6907 MDEV-15526 systemd unit files naming and installation
Drop Aliases from the service file directive. Aliases in the service
file only take effect when the service is enabled. This is the case
because Aliases in service files do not have to be unique across various
services.

Shipping symlinks guarantees that one can always enable mariadb service
using `systemctl enable mysql` or `systemctl enable mysqld` and makes
the commands indempotent.
2020-04-18 16:06:16 +03:00
Vicențiu Ciorbaru
afde33fc6f Update plugins.pam_* tests to no longer provide --plugin-dir to mysqltest
Post change of 8447edb747, --plugin-dir
is set in the default my.cnf file under [client]. The value is automatically
detected during mtr startup.
2020-04-18 15:18:34 +03:00
Kentoku SHIBA
181f17c3cd MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
When executing a query like "select id, 0 as const, val from ...", there are 3 columns(items) in Query->select at handlerton->create_group_by(). After that, MariaDB makes a temporary table with 2 columns. The skipped items are const item, so fixing Spider to skip const items for items at Query->select.
2020-04-17 11:52:42 +09:00
Kentoku SHIBA
b4dd996dc1 MDEV-21884 MariaDB with Spider crashes on a query 2020-04-17 10:32:57 +09:00
Vladislav Vaintroub
e2a932c9ea Post-fixes for MDEV-18851 for Windows
- On Windows, do not handle lack of SeLockMemory privilege as fatal error.
Just like on any other platform, there is a fallback to ordinary pages.
It is better than server that does not start, silently.

- On Windows, remove incorrect  irritating "fallback to conventional pages failed"
from the warning, when allocating large pages fails.
2020-04-17 13:11:07 +02:00
Marko Mäkelä
2f7d91bb6c MDEV-22242 B-trees can become extremely skewed
The test innodb.innodb_wl6326 that had been disabled in 10.4 due to
MDEV-21535 is failing on 10.5 due to a different reason: the removal
of the MLOG_COMP_END_COPY_CREATED operations in MDEV-12353
commit 276f996af9 caused PAGE_LAST_INSERT
to be set to something nonzero by the function page_copy_rec_list_end().

This in turn would cause btr_page_get_split_rec_to_right() to behave
differently: we would not attempt to split the page at all, but simply
insert the new record into the new, empty, right leaf page.

Even though the change reduced the sizes of some tables, it is better
to aim for balanced trees.

page_copy_rec_list_end(), PageBulk::finishPage():
Preserve PAGE_LAST_INSERT, PAGE_N_DIRECTION, PAGE_DIRECTION.

PageBulk::finish(): Move some common code from PageBulk::finishPage().
2020-04-14 18:43:03 +03:00
Otto Kekäläinen
87a7968c23 MDEV-22150: Symlink and move test client plugins to client plugin path
Without this, the mtr will fail to load client plugins and the following
tests would fail:

- main.plugin_auth_qa_2
- main.plugin_auth_qa_3
- main.plugin_auth
- plugins.multiauth
- plugins.auth_ed25519
2020-04-14 12:08:30 +03:00
Vicențiu Ciorbaru
8447edb747 Specify a new client_plugindir path, to be used during testing
When testing installed packages, the client (mariadb or mysql) can have
plugins stored in a separate location compared to server. This change
ensures that the correct paths are used for the plugin-dir directive in
the my.cnf generated by mysql-test-run.

The directives in my.cnf will be replaced like so:

[client]
plugin-dir=$client_plugindir

[mysqld]
plugin-dir=$plugindir

where $<variable-name> corresponds to the variable name in mtr.
2020-04-14 12:08:30 +03:00
Vicențiu Ciorbaru
609a9312dc Update dialog to not provide plugin-dir path to mysqltest
plugin-dir is provided via --defaults-file=<config-file> already part of
the exec call.
2020-04-14 11:17:28 +03:00
Otto Kekäläinen
4bc31a904f MDEV-22053: Pass INSTALL_LAYOUT "DEB" correctly to CONC (libmariadb)
As this was missing, the libmariadb/install.cmake section for Debian
builds was not triggered at all, and existing fixes to make the
Debian paths correct did not apply.

Update libmariadb3 paths in Debian packaging after MariaDB Connector C
3.1.8 upgrade

Also now include caching_sha2_password.so which was previously omitted.

Keep old libmariadbclient.a as symbolic link for backwards compatibility.

Closes: CONC-304
Related: CONC-456, MDEV-22150
2020-04-13 14:33:50 +03:00
Sergei Golubchik
91e79dff54 cleanup: comments 2020-04-12 22:11:22 +02:00
Sergei Golubchik
fcd84da5f1 MDEV-22218 InnoDB: Failing assertion: node->pcur->rel_pos == BTR_PCUR_ON upon LOAD DATA with NO_BACKSLASH_ESCAPES in SQL_MODE and unique blob in table
`inited == NONE` at the initialization time does not always mean
that it'll be `NONE` later, at the execution time. Use a more complex
caller-specific logic to decide whether to create a cloned lookup handler.

Besides LOAD (as in the original bug report) make sure that all
prepare_for_insert() invocations are covered by tests. Add tests for
CREATE ... SELECT, multi-UPDATE, and multi-DELETE.

Don't enable write cache with long uniques.
2020-04-12 22:10:57 +02:00
Sergei Golubchik
8c0b988073 cleanup: remove unnecessary malloc 2020-04-11 12:53:25 +02:00
Sergei Golubchik
4bd8c63444 cleanup: don't repeat common code 2020-04-11 12:53:25 +02:00
Vladislav Vaintroub
364e7a9ae6 remove debugging message 2020-04-10 19:53:33 +02:00
Vladislav Vaintroub
3ab21fd4e0 Windows build - use InstallRequiredSystemLibraries for MSVC_CRT_TYPE=/MD
CMake parameter

This includes compiler-specific dependencies
- tiny part of CRT, mostly memcpy/memset in msvcr140.dll
- exception handling in msvc140_1.dll
- C++ standard library in msvcp140
not much else

CMake is smart enough to copy the dependencies into the bin directory
itself.
2020-04-10 19:05:26 +02:00
Vladislav Vaintroub
93efbc390d MDEV-22214 mariadbd.exe calls function mysqld.exe, and crashes
Stop linking plugins to the server executable on Windows.
Instead, extract whole server functionality into a large DLL, called
server.dll. Link both plugins, and small server "stub" exe to it.

This eliminates plugin dependency on the name of the server executable.
It also reduces the size of the packages (since tiny mysqld.exe
and mariadbd.exe are now both linked to one big DLL)

Also, simplify the functionality of exporing all symbols from selected
static libraries. Rely on WINDOWS_EXPORT_ALL_SYMBOLS, rather than old
self-backed solution.

fix compile error

replace GetProcAddress(GetModuleHandle(NULL), "variable_name")
for server exported data with actual variable names.

Runtime loading was never required,was error prone
, since symbols could be missing at runtime, and now it actually failed,
because we do not export symbols from executable anymore, but from a shared
library

This did require a MYSQL_PLUGIN_IMPORT decoration for the plugin,
but made the code more straightforward, and avoids missing symbols at
runtime (as mentioned before).

The audit plugin is still doing some dynamic loading, as it aims to work
cross-version. Now it won't work cross-version on Windows, as it already
uses some symbols that are *not* dynamically loaded, e.g fn_format
and those symbols now exported from server.dll , when earlier they were
exported by mysqld.exe

Windows, fixes for storage engine plugin loading
after various rebranding stuff

Create server.dll containing functionality of the whole server
make mariadbd.exe/mysqld.exe a stub that is only  calling mysqld_main()

fix build
2020-04-10 19:05:26 +02:00
Vladislav Vaintroub
38f7dbec19 CMake - clang linker on Windows does not understand /release flag 2020-04-10 19:05:26 +02:00
Vladislav Vaintroub
11cebb4ae8 CMake : Do not add compile flags, such as -Wconversion with ADD_DEFINITIONS
This broke clang-cl, apparently when compiling version resource file with
RC tool, because RC would not aceept -Wxxx at all.
2020-04-10 19:05:26 +02:00
Thirunarayanan Balathandayuthapani
6bbc0eedc6 MDEV-22193 Avoid un-necessary page initialization during recovery
- InnoDB is doing un-necessary redo log page initialisation during
recovery and unnecessary traversal of redo log during last phase.
This patch does the optimization of removing unnecessary redo log page
initialisation and detects the memory exhaust earlier.
2020-04-09 21:25:31 +05:30
Varun Gupta
ff66d38cf2 MDEV-21946: Server crash in store_length upon GROUP BY WITH ROLLUP with geometry field
overflow happening when we add suffix length to sortlength.
Make sure that sortlength is at max UINT_MAX32
2020-04-09 13:51:47 +05:30
Jan Lindström
c7ab676192 MDEV-22075 : Server crashes in wsrep_should_replicate_ddl_iterate upon CREATE VIEW
Fixed incorrect pointer reference when table is not available.
2020-04-08 18:09:28 +03:00
Sergey Vojtovich
6cf8f05fd9 Fixed centos 6 build failure 2020-04-08 15:31:57 +04:00
Daniel Black
97506bf7c3 mysql-test: add large_pages test
On by default, suppress "fallback to conventional memory"
warnings as we're never sure what the test has available.

Leave off Windows test for now. Running without SeLockMemoryPrivilege
is fatal if --large-pages is specified.
2020-04-08 14:51:14 +04:00
Daniel Black
5f5bb63b2e Add Daniel Black to authors 2020-04-08 14:51:14 +04:00
Daniel Black
a0d5894015 my_pagepages: perror -> my_error 2020-04-08 14:51:14 +04:00
Daniel Black
a535d4d1a6 my_largepage: fprintf -> my_{printf_,}error 2020-04-08 14:51:14 +04:00
Daniel Black
11aaf5c8d2 add EE_BADMEMORY_RELEASE 2020-04-08 14:51:14 +04:00
Daniel Black
5e86b2eec8 my_large_malloc: style fix 2020-04-08 14:51:14 +04:00
Daniel Black
2c00502014 my_largepage: reduce includes already in my_global.h 2020-04-08 14:51:14 +04:00
Daniel Black
96d4b228ea my_large_pages: simplify solaris constants 2020-04-08 14:51:14 +04:00
Daniel Black
7b7a9161e2 my_large_pages: remove conventional memory(my_malloc_lock) fallback
Both Windows and MMAP capable implementations fell back to a
non-MEM_LARGE_PAGES/HugeTLB allocation with the large page implementaion
failed. These can can be freed by the corresponding function.

Prior to this, if we fell back to a conventional memory, than will
results in deallocation using munmap/VirtualFree on a memory allocated
using my_malloc_lock. At worst this could succeed and the
my_malloc_lock looses its memory without knowing about it.
2020-04-08 14:51:14 +04:00
Marko Mäkelä
ccc06931c3 Merge 10.4 into 10.5 2020-04-08 10:36:41 +03:00
Marko Mäkelä
476966b3fb MDEV-21535 Unnecessarily large ha_innobase::records_in_range() scans
The bug was already fixed in
commit 1242eb3d32
but the test innodb.innodb_wl6326 had not been enabled.
2020-04-08 09:59:00 +03:00
Rasmus Johansson
9075973dbf MDEV-17812 Use MariaDB in error messages instead of MySQL
Changed wording in error messages from MySQL to MariaDB. In
cases where the word server could be used instead it was done.

Tests that have these errors recorded were updated.
2020-04-08 06:09:42 +00:00
Marko Mäkelä
0eab87cef2 MDEV-22010: Allow mariadbd in mtr suppressions 2020-04-07 17:20:38 +03:00
Marko Mäkelä
1738c0f1be MDEV-22169 Recovery fails after failing to insert into mlog_init
In a multi-batch recovery, we must ensure that INIT_PAGE and
especially the MDEV-15528 FREE_PAGE records will be taken
properly into account.

Writing a FREE_PAGE record gives the server permission to omit
a page write. If recovery insists on applying log to a page
whose page flush has been omitted, then the consistency checks
in the application of high-level redo log records (appending
an undo log record, inserting or deleting an index record)
will likely fail.

mlog_init_t::add(): Return whether the state was changed.

mlog_init_t::will_avoid_read(): Determine whether a page read
will be avoided and whether older log records can be safely
skipped.

recv_sys_t::parse(): Even if store==STORE_NO, process the records
INIT_PAGE and FREE_PAGE. While processing them, we can delete older
redo log records for the page. If store!=STORE_NO, we can directly
skip redo log recods of other types if mlog_init indicates that the
page will be freed or initialized by at a later LSN.

This fix was developed in cooperation with
Thirunarayanan Balathandayuthapani.
2020-04-07 17:18:05 +03:00
Daniel Black
d848fcad69 MDEV-22010: mtr, "mariadbd" exists in mysys error messages
Also executable can change for jemalloc.
2020-04-07 23:39:10 +10:00
Daniel Black
dcc2eaebbd MDEV-22010: mtr search for mariadbd first 2020-04-06 18:23:35 +02:00
Elena Stepanova
778a174e5e Postfix for f46917238: use colon instead of comma
Comma is reported not to work on older systems/ASAN versions
2020-04-06 10:47:11 +03:00
Otto Kekäläinen
b6eabce139 Travis-CI: Shorten deb build log to keep it under 4 MB
There is a 4 MB hard limit on Travis-CI and build output needs to be less
than that. Silencing the 'make install' step gets rid of a lot of
"Installing.." and "Missing.." and removing all mysql-test files will
make the dh_missing warnings much shorter.
2020-04-06 09:07:02 +03:00
Otto Kekäläinen
28604c349b Travis-CI: Add missing build dependency dh-exec
Backported from 30b44aaec7.
2020-04-06 09:06:51 +03:00