Commit graph

229 commits

Author SHA1 Message Date
Sergei Golubchik
4f9396b9f8 MDEV-31474 KDF() function
KDF(key_str, salt [, {info | iterations} [, kdf_name [, width ]]])

kdf_name is "hkdf" or "pbkdf2_hmac" (default).

width (in bits) can be any number divisible by 8,
by default it's taken from @@block_encryption_mode

iterations must be positive, and is 1000 by default

OpenSSL 1.0 doesn't support HKDF, so it'll return NULL.
This OpenSSL version is still used in SLES 12 and CentOS 7
2023-09-30 14:43:12 +02:00
Marko Mäkelä
588efca237 Merge 10.6 into 10.7 2022-10-13 10:05:29 +03:00
Marko Mäkelä
a992c615a6 Merge 10.5 into 10.6 2022-10-12 12:14:13 +03:00
Marko Mäkelä
977c385df3 Merge 10.4 into 10.5 2022-10-12 11:29:32 +03:00
Marko Mäkelä
7434eb566e Merge 10.3 into 10.4 2022-10-11 15:18:49 +03:00
Anel Husakovic
602124bb3b Remove redudant defines USE_MB and USE_MB_IDENT
Reviewer: <wlad@mariadb.com>
2022-10-09 17:33:07 +02:00
Marko Mäkelä
6dc157f8a6 Merge 10.5 into 10.6 2022-10-06 09:22:39 +03:00
Marko Mäkelä
de078e060e Merge 10.4 into 10.5 2022-10-06 08:29:56 +03:00
Marko Mäkelä
df97eb1432 Remove HAVE_SNPRINTF
This fixes up commit 77c184df7c
which explicitly specifies that we use ISO/IEC 9899:1999 (C99),
which includes the snprintf() function.
2022-10-05 10:09:49 +03:00
Jan Lindström
5fdbb3a72e Merge 10.6 into 10.7 2022-09-05 14:55:47 +03:00
Marko Mäkelä
3c7887a85f Merge 10.5 into 10.6 2022-09-05 10:09:03 +03:00
Daniel Black
43037a5a48 Merge branch 10.4 into 10.5 2022-08-31 11:06:14 +10:00
Daniel Black
cf1a944f5b Merge 10.3 into 10.4 2022-08-31 10:52:53 +10:00
Daniel Black
129616c70a MDEV-28592 disks plugin - getmntinfo (BSD) & getmntent (AIX)
Thanks to references from Brad Smith, BSDs use getmntinfo as
a system call for mounted filesystems.

Most BSDs return statfs structures, (and we use OSX's statfs64),
but NetBSD uses a statvfs structure.

Simplify Linux getmntent_r to just use getmntent.

AIX uses getmntent.

An attempt at writing Solaris compatibility with
a small bit of HPUX compatibility was made based on man page
entries only. Fixes welcome.

statvfs structures now use f_bsize for consistency with statfs

Test case adjusted as PATH_MAX is OS defined (e.g. 1023 on AIX)

Fixes: 0ee5cf837e

also fixes:

MDEV-27818: Disk plugin does not show zpool mounted devices

This is because zpool mounted point don't begin with /.

Due to the proliferation of multiple filesystem types since this
was written, we restrict the entries listed in the disks plugin
to excude:
* read only mount points (no point monitoring, and
  includes squash, snaps, sysfs, procfs, cgroups...)
* mount points that aren't directories (excludes /etc/hostname and
  similar mounts in containers). (getmntent (Linux/AIX) only)
* exclude systems where there is no capacity listed (excludes various
  virtual filesystem types).

Reviewer: Sergei Golubchik
2022-08-31 10:32:04 +10:00
Marko Mäkelä
bdd80e3fb1 Merge 10.6 into 10.7 2022-08-24 09:22:34 +03:00
Brad Smith
f02ca429f7 Revert aligned_alloc() addition from MDEV-28836
As pointed out with MDEV-29308 there are issues with the code as is.
MariaDB is built as C++11 / C99. aligned_alloc() is not guarenteed
to be exposed when building with any mode other than C++17 / C11.
The other *BSD's have their stdlib.h header to expose the function
with C+11 anyway, but the issue exists in the C99 code too, the
build just does not use -Werror. Linux globally defines _GNU_SOURCE
hiding the issue as well.
2022-08-22 09:10:40 +03:00
Marko Mäkelä
6680fd8d4b Merge 10.6 into 10.7 2022-06-21 18:02:41 +03:00
Marko Mäkelä
3794673111 MDEV-28836: Memory alignment cleanup
Table_cache_instance: Define the structure aligned at
the CPU cache line, and remove a pad[] data member.
Krunal Bauskar reported this to improve performance on ARMv8.

aligned_malloc(): Wrapper for the Microsoft _aligned_malloc()
and the ISO/IEC 9899:2011 <stdlib.h> aligned_alloc().
Note: The parameters are in the Microsoft order (size, alignment),
opposite of aligned_alloc(alignment, size).
Note: The standard defines that size must be an integer multiple
of alignment. It is enforced by AddressSanitizer but not by GNU libc
on Linux.

aligned_free(): Wrapper for the Microsoft _aligned_free() and
the standard free().

HAVE_ALIGNED_ALLOC: A new test. Unfortunately, support for
aligned_alloc() may still be missing on some platforms.
We will fall back to posix_memalign() for those cases.

HAVE_MEMALIGN: Remove, along with any use of the nonstandard memalign().

PFS_ALIGNEMENT (sic): Removed; we will use CPU_LEVEL1_DCACHE_LINESIZE.

PFS_ALIGNED: Defined using the C++11 keyword alignas.

buf_pool_t::page_hash_table::create(),
lock_sys_t::hash_table::create():
lock_sys_t::hash_table::resize(): Pad the allocation size to an
integer multiple of the alignment.

Reviewed by: Vladislav Vaintroub
2022-06-21 16:59:49 +03:00
Sergei Golubchik
fd132be117 Merge branch '10.6' into 10.7 2022-05-11 11:25:33 +02:00
Sergei Golubchik
3bc98a4ec4 Merge branch '10.5' into 10.6 2022-05-10 14:01:23 +02:00
Sergei Golubchik
ef781162ff Merge branch '10.4' into 10.5 2022-05-09 22:04:06 +02:00
Sergei Golubchik
a70a1cf3f4 Merge branch '10.3' into 10.4 2022-05-08 23:03:08 +02:00
Oleksandr Byelkin
9614fde1aa Merge branch '10.2' into 10.3 2022-05-03 10:59:54 +02:00
Dmitry Shulga
bc7ba7afee MDEV-27758: Errors when building Connect engine on os x 11.6.2
Added checking for support of vfork by a platform where
building being done. Set HAVE_VFORK macros in case vfork()
system call is supported. Use vfork() system call if the
macros HAVE_VFORK is set, else use fork().
2022-04-22 18:47:19 +07:00
Monty
b1d81974b2 Added support to MEM_ROOT for write protected memory
This is useful for thing like Item_true and Item_false that we
allocated and initalize once and want to ensure that nothing can
change them

Main changes:
- Memory protection is achived by allocating memory with mmap() and
  protect it from write with mprotect()
- init_alloc_root(...,MY_ROOT_USE_MPROTECT) will create a
  memroot that one can later use with protect_root() to turn it
  read only or turn it back to read-write. All allocations to this
  memroot is done with mmap() to ensure page alligned allocations.
- alloc_root() code was rearranged to combine normal and valgrind code.
- init_alloc_root() now changes block size to be power of 2's, to get less
  memory fragmentation.
- Changed MEM_ROOT structure to make it smaller. Also renamed
  MEM_ROOT m_psi_key to psi_key.
- Moved MY_THREAD_SPECIFIC marker in MEM_ROOT from block size (old hack)
  to flags.
- Added global variable my_system_page_size. This is initialized at
  startup.
2021-07-18 19:59:35 +03:00
Vladislav Vaintroub
3d6eb7afcf MDEV-25602 get rid of __WIN__ in favor of standard _WIN32
This fixed the MySQL bug# 20338 about misuse of double underscore
prefix __WIN__, which was old MySQL's idea of identifying Windows
Replace it by _WIN32 standard symbol for targeting Windows OS
(both 32 and 64 bit)

Not that connect storage engine is not fixed in this patch (must be
fixed in "upstream" branch)
2021-06-06 13:21:03 +02:00
Marko Mäkelä
6729dd894c Merge 10.5 into 10.6 2021-04-14 13:39:28 +03:00
David CARLIER
9a3cbc0541 mysqld: print status display subset of memory usage.
leveling up to some degree with linux's mallinfo* api for
 the memory usage display with debug build with malloc_zone
 Darwin api.

Closes: #1803
2021-04-14 19:21:35 +10:00
Daniel Black
460d480c74 MDEV-5536: add systemd socket activation
Systemd has a socket activation feature where a mariadb.socket
definition defines the sockets to listen to, and passes those
file descriptors directly to mariadbd to use when a connection
occurs.

The new functionality is utilized when starting as follows:

  systemctl start mariadb.socket

The mariadb.socket definition only needs to contain the network
information, ListenStream= directives, the mariadb.service
definition is still used for service instigation.

When mariadbd is started in this way, the socket, port, bind-address
backlog are all assumed to be self contained in the mariadb.socket
definition and as such the mariadb settings and command line
arguments of these network settings are ignored.
See man systemd.socket for how to limit this to specific ports.

Extra ports, those specified with extra_port in socket activation
mode, are those with a FileDescriptorName=extra. These need
to be in a separate service name like mariadb-extra.socket and
these require a Service={mariadb.service} directive to map to the
original service. Extra ports need systemd v227 or greater
(not RHEL/Centos7 - v219) when FileDescriptorName= was added,
otherwise the extra ports are treated like ordinary ports.

The number of sockets isn't limited when using systemd socket activation
(except by operating system limits on file descriptors and a minimal
amount of memory used per file descriptor). The systemd sockets passed
can include any ownership or permissions, including those the
mariadbd process wouldn't normally have the permission to create.

This implementation is compatible with mariadb.service definitions.
Those services started with:

  systemctl start mariadb.service

does actually start the mariadb.service and used all the my.cnf
settings of sockets and ports like it previously did.
2021-03-28 13:53:55 +11:00
Marko Mäkelä
80459bcbd4 Merge 10.4 into 10.5 2021-03-27 17:37:42 +02:00
Marko Mäkelä
7ae37ff74f Merge 10.3 into 10.4 2021-03-27 17:12:28 +02:00
Marko Mäkelä
3157fa182a Merge 10.2 into 10.3 2021-03-27 16:11:26 +02:00
Michael Okoko
48141f3c17 Replace mallinfo with mallinfo2 on supported systems
`mallinfo` is deprecated since glibc 2.33 and has been replaced by mallinfo2.
The deprecation causes building the server to fail if glibc version is > 2.33.

Check if mallinfo2 exist on the system and use it instead.
2021-03-27 08:40:04 +11:00
Daniel Black
3b486c28f7 MDEV-24125: linux large pages, linux/mman.h needed
Centos/RHEL7 have the MAP_HUGE_SHIFT constant
defined in linux/mman.h which needed to get included.
2020-11-19 16:30:17 +11:00
Vladislav Vaintroub
ccbe6bb6fc MDEV-19935 Create unified CRC-32 interface
Add CRC32C code to mysys. The x86-64 implementation uses PCMULQDQ in addition to CRC32 instruction
after Intel whitepaper, and is ported from rocksdb code.

Optimized ARM and POWER CRC32 were already present in mysys.
2020-09-17 16:07:37 +02:00
Marko Mäkelä
9ef36faa61 MDEV-23618 InnoDB lacks IA-32 CRC-32C acceleration on GCC 4
When MDEV-22669 introduced CRC-32C acceleration to IA-32,
it worked around a compiler bug by disabling the acceleration
on GCC 4 for IA-32 altogether, even though the compiler bug
only affects -fPIC builds that are targeting IA-32.

Let us extend the solution fe5dbfe723
and define HAVE_CPUID_INSTRUCTION that allows us to implement
a necessary and sufficient work-around of the compiler bug.
2020-08-28 14:44:36 +03:00
Marko Mäkelä
b47d61d04f MDEV-23585: Fix HAVE_CLMUL_INSTRUCTION
MDEV-22641 in commit dec3f8ca69
refactored a SIMD implementation of CRC-32 for the ISO 3309 polynomial
that uses the IA-32/AMD64 carry-less multiplication (pclmul)
instructions. The code was previously only available in Mariabackup;
it was changed to be a general replacement of the zlib crc32().

There exist AMD64 systems where CMAKE_SYSTEM_PROCESSOR matches
the pattern i[36]86 but not x86_64 or amd64. This would cause a
link failure, because mysys/checksum.c would basically assume that
the compiler support for instruction is always available on GCC-compatible
compilers on AMD64.

Furthermore, we were unnecessarily disabling the SIMD acceleration
for 32-bit executables.

Note: Until MDEV-22749 has been implemented, the PCLMUL instruction
will not be used on Microsoft Windows.

Closes: #1660
2020-08-27 09:34:53 +03:00
Oleksandr Byelkin
48b5777ebd Merge branch '10.4' into 10.5 2020-08-04 17:24:15 +02:00
Oleksandr Byelkin
57325e4706 Merge branch '10.3' into 10.4 2020-08-03 14:44:06 +02:00
Oleksandr Byelkin
c32f71af7e Merge branch '10.2' into 10.3 2020-08-03 13:41:29 +02:00
Oleksandr Byelkin
ef7cb0a0b5 Merge branch '10.1' into 10.2 2020-08-02 11:05:29 +02:00
Karthik Kamath
e6cb263ef3 MDEV-15961: Fix stacktraces under FreeBSD (aarch64)
Largely based on MySQL commit
75271e51d6

MySQL Ref:
    BUG#24566529: BACKPORT BUG#23575445 TO 5.6

    (cut)
    Also, the PTR_SANE macro which tries to check if a pointer
    is invalid (used when printing pointer values in stack traces)
    gave false negatives on OSX/FreeBSD. On these platforms we
    now simply check if the pointer is non-null. This also removes
    a sbrk() deprecation warning when building on OS X. (It was
    before only disabled with building using XCode).

Removed execinfo path of MySQL patch that was already included.

sbrk doesn't exist on FreeBSD aarch64.

Removed HAVE_BSS_START based detection and replaced with __linux__
as it doesn't exist on OSX, Solaris or Windows.  __bss_start
exists on mutiple Linux architectures.

Tested on FreeBSD and Linux x86_64. Being in FreeBSD ports for 2
years implies a good testing there on all FreeBSD architectures there
too. MySQL-8.0.21 code is functionally identical to original commit.
2020-07-28 11:10:25 +10:00
Vladislav Vaintroub
9c55f83eae Cleanup - remove HAVE_AIOWAIT and associated code from mysys
HAVE_AIOWAIT had not been disabled and unused for at least 10 years.
2020-06-05 00:16:50 +02:00
Daniel Black
9bf3a3a47a HAVE_LARGE_PAGES no longer global 2020-04-05 17:20:50 +04:00
Sergey Vojtovich
30379b487f HAVE_LARGE_PAGE_OPTION to HAVE_LARGE_PAGES
It isn't an "option" anymore.
2020-04-04 14:55:51 +04:00
Sergey Vojtovich
fc6e8b4b1b Cleanup HAVE_MMAP_ALIGNED: one cmake check less 2020-04-04 14:55:51 +04:00
Daniel Black
e8351934b6
Merge pull request #1221 from grooverdan/10.4-MDEV-18851-multiple-sized-large-page-support
MDEV-18851: multiple sized large page support (linux)
2020-04-02 23:54:08 +04:00
Oleksandr Byelkin
fad47df995 Merge branch '10.4' into 10.5 2020-03-11 17:52:49 +01:00
Oleksandr Byelkin
b8c0e49670 Merge commit '10.3' into 10.4 2020-03-11 13:27:10 +01:00
Jon Olav Hauglid
52d7980753 Bug#18913935: REMOVE SUPPORT FOR LINUXTHREADS
This patch removes support for LinuxThreads.
It was superseded by NPTL in Linux 2.6 (2003).
2020-03-10 19:24:21 +01:00