Commit graph

7273 commits

Author SHA1 Message Date
Sergei Golubchik
4adc1269cc FreeBSD compilation fixes
* FreeBSD calls amd64 what Linux calls x86_64
* signal returns void (*)(int)
* struct pam_message has char*, not const char*
* krb5_free_unparsed_name exists, but is deprecated
2020-06-04 23:52:46 +02:00
Daniel Black
463a8fc5fd MDEV-22641: postfix - crc32{,c} fixups for ppc64 2020-06-03 18:59:20 +10:00
mysqlonarm
dec3f8ca69
MDEV-22641: Provide SIMD optimized wrapper for zlib crc32() ()
Existing implementation used my_checksum (from mysys)
for calculating table checksum and binlog checksum.

This implementation was optimized for powerpc only and lacked
SIMD implementation for x86 (using clmul) and ARM
(using ACLE) instead used zlib-crc32.

mariabackup had its own copy of the crc32 implementation
using hardware optimized implementation only for x86 and lagged
hardware based implementation for powerpc and ARM.

Patch helps unifies all such calls and help aggregate all of them
using an unified interface my_checksum().

Said unification also enables hardware optimized calls for all
architecture viz. x86, ARM, POWERPC.
Default always fallback to zlib crc32.

Thanks to Daniel Black for reviewing, fixing and testing
PowerPC changes. Thanks to Marko and Daniel for early code feedback.
2020-06-01 11:34:06 +03:00
Marko Mäkelä
4a0b56f604 Merge 10.4 into 10.5 2020-05-31 10:28:59 +03:00
Marko Mäkelä
6da14d7b4a Merge 10.3 into 10.4 2020-05-30 11:04:27 +03:00
Monty
043828bdb3 Fixed wrong length in my_default.c
This couldn't cause any bugs as ptr was zero terminated, but still better
to have the length correct.
2020-05-29 22:47:37 +03:00
Marko Mäkelä
e9aaa10c11 Merge 10.2 into 10.3 2020-05-29 22:21:19 +03:00
Sergei Golubchik
e64dc07125 assert(a && b); -> assert(a); assert(b); 2020-05-27 15:56:40 +02:00
Monty
610bb1d2c0 Fixed bug thr_lock_info_init
This bug didn't cause any issues with current usage
2020-05-23 12:29:09 +03:00
Daniel Black
3ea05d0842 largepages: osx compile warning fix
my_is_2pow is only used on linux.

fixes compile warning:

mysys/my_largepage.c:48:23: warning: unused function 'my_is_2pow' [-Wunused-function]

static inline my_bool my_is_2pow(size_t n) { return !((n) & ((n) - 1)); }

                      ^

1 warning generated.
2020-05-18 19:43:50 +04:00
Marko Mäkelä
23047d3ed4 Merge 10.4 into 10.5 2020-05-18 17:30:02 +03:00
Varun Gupta
0a5668f512 MDEV-22556: Incorrect result for window function when using encrypt-tmp-files=ON
The issue here is that end_of_file for encrypted temporary IO_CACHE (used by filesort) is updated
using lseek.
Encryption adds storage overhead and hides it from the caller by recalculating offsets and lengths.
Two different IO_CACHE cannot possibly modify the same file
because the encryption key is randomly generated and stored in the IO_CACHE.
So when the tempfiles are encrypted DO NOT use lseek to change end_of_file.

Further observations about updating end_of_file using lseek
1) The end_of_file update is only used for binlog index files
2) The whole point is to update file length when the file was modified via a different file descriptor.
3) The temporary IO_CACHE files can never be modified via a different file descriptor.
4) For encrypted temporary IO_CACHE, end_of_file should not be updated with lseek
2020-05-17 15:26:18 +05:30
Marko Mäkelä
b30a013142 Merge 10.4 into 10.5 2020-05-13 14:25:06 +03:00
Marko Mäkelä
38f6c47f8a Merge 10.3 into 10.4 2020-05-13 12:52:57 +03:00
Marko Mäkelä
15fa70b840 Merge 10.2 into 10.3 2020-05-13 11:45:05 +03:00
Marko Mäkelä
6bc4444d7c Merge 10.1 into 10.2 2020-05-13 11:12:31 +03:00
Anel Husakovic
f7ba675555 MDEV-22344: Fix typos in comments 2020-05-06 18:15:32 +02:00
Marko Mäkelä
7bcaa541aa Merge 10.4 into 10.5 2020-05-05 21:16:22 +03:00
Marko Mäkelä
2c3c851d2c Merge 10.3 into 10.4 2020-05-05 20:33:10 +03:00
Oleksandr Byelkin
7fb73ed143 Merge branch '10.2' into 10.3 2020-05-04 16:47:11 +02:00
Vladislav Vaintroub
d50f776930 MDEV-22454 Allow -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON
Disable IPO (interprocedural optimization, aka /GL) on Windows
on libraries, from which server.dll exports symbols -  exporting symbols
does not work for objects compiled with /GL.
2020-05-04 14:17:06 +02:00
Oleksandr Byelkin
ca091e6372 Merge branch '10.1' into 10.2 2020-05-02 08:44:17 +02:00
Oleksandr Byelkin
23c6fb3e62 Merge branch '5.5' into 10.1 2020-04-30 17:36:41 +02:00
Sergei Golubchik
6a31aea5a1 BUG#30301356 - SOME EVENTS ARE DELAYED AFTER DROPPING EVENT
queues.c cleanup and refactoring.

Restore old version of _downhead() (from before cd483c5520)
that works well in an average case. Use it for queue_fix().

Move existing specialized version of _downhead() to queue_replace()
where it'll be handling the case it was specifically optimized for
(moving the element to the end of the queue).
And correct it to fix the heap not only down, but also up
(this fixes BUG#30301356).

Add unit tests.

Collateral cosmetic fixes.
2020-04-30 10:13:21 +02:00
Marko Mäkelä
496d0372ef Merge 10.4 into 10.5 2020-04-29 15:40:51 +03:00
Daniel Black
c238e9b96a MDEV-20685: compile fixes for Solaris/OSX/AIX
sig_return: Solaris/OSX returns different function ptr
Move defination to my_alarm.h as its the only use.

prevents compile warnings (copied from 10.3 branch)

mysys/my_sync.c:136:19: error: 'cur_dir_name' defined but not used [-Werror=unused-const-variable=]
  136 | static const char cur_dir_name[]= {FN_CURLIB, 0};
      |                   ^~~~~~~~~~~~

fix compile error (DEPRECATED) leaked from ssl headers.

In file included from /export/home/dan/mariadb-server-10.4/sql/sys_vars.cc:37:
/export/home/dan/mariadb-server-10.4/sql/sys_vars.ic:69: error: "DEPRECATED" redefined [-Werror]
   69 | #define DEPRECATED(X) X
      |
In file included from /export/home/dan/mariadb-server-10.4/include/violite.h:150,
                 from /export/home/dan/mariadb-server-10.4/sql/sql_class.h:38,
                 from /export/home/dan/mariadb-server-10.4/sql/sys_vars.cc:36:
/usr/include/openssl/ssl.h:2356: note: this is the location of the previous definition
 2356 | # define DEPRECATED __attribute__((deprecated))
      |

Avoid Werror condition on non-Linux:

plugin/server_audit/server_audit.c:2267:7: error: variable 'db_len_off' set but not used [-Werror=unused-but-set-variable]
 2267 |   int db_len_off;
      |       ^~~~~~~~~~
plugin/server_audit/server_audit.c:2266:7: error: variable 'db_off' set but not used [-Werror=unused-but-set-variable]
 2266 |   int db_off;
      |       ^~~~~~

auth_gssapi fix include path for Solaris

Consistent with the upstream packaged patch:
https://github.com/OpenIndiana/oi-userland/blob/oi/hipster/components/database/mariadb-103/patches/06-gssapi.h.patch

compile warnings on Solaris

[ 91%] Building C object plugin/server_audit/CMakeFiles/server_audit.dir/server_audit.c.o
/plugin/server_audit/server_audit.c: In function 'auditing_v8':
/plugin/server_audit/server_audit.c:2194:20: error: unused variable 'db_len_off' [-Werror=unused-variable]
 2194 |   static const int db_len_off= 128;
      |                    ^~~~~~~~~~
/plugin/server_audit/server_audit.c:2193:20: error: unused variable 'db_off' [-Werror=unused-variable]
 2193 |   static const int db_off= 120;
      |                    ^~~~~~
/plugin/server_audit/server_audit.c:2192:20: error: unused variable 'cmd_off' [-Werror=unused-variable]
 2192 |   static const int cmd_off= 4432;
      |                    ^~~~~~~
At top level:
/plugin/server_audit/server_audit.c:2192:20: error: 'cmd_off' defined but not used [-Werror=unused-const-variable=]
/plugin/server_audit/server_audit.c:2193:20: error: 'db_off' defined but not used [-Werror=unused-const-variable=]
 2193 |   static const int db_off= 120;
      |                    ^~~~~~
/plugin/server_audit/server_audit.c:2194:20: error: 'db_len_off' defined but not used [-Werror=unused-const-variable=]
 2194 |   static const int db_len_off= 128;
      |                    ^~~~~~~~~~
cc1: all warnings being treated as errors

tested on:
$ uname -a
SunOS openindiana 5.11 illumos-b97b1727bc i86pc i386 i86pc
2020-04-29 12:02:47 +03:00
Marko Mäkelä
b63446984c Merge 10.3 into 10.4 2020-04-27 17:38:17 +03:00
Marko Mäkelä
2e12d471ea Merge 10.2 into 10.3 2020-04-27 14:24:41 +03:00
Marko Mäkelä
fbe2712705 Merge 10.4 into 10.5
The functional changes of commit 5836191c8f
(MDEV-21168) are omitted due to MDEV-742 having addressed the issue.
2020-04-25 21:57:52 +03:00
Eugene Kosov
2c5067b689 cleanup THR_KEY_mysys
read TLS with my_thread_var
write TLS with set_mysys_var()

my_thread_var is no longer __attribute__ ((const)): this attribute
is simply incorrect here. Read gcc manual for more information.
sql/threadpool_generic.cc fails with that attribute.
2020-04-25 00:55:39 +03:00
Monty
eca5c2c67f Added support for more functions when using partitioned S3 tables
MDEV-22088 S3 partitioning support

All ALTER PARTITION commands should now work on S3 tables except

REBUILD PARTITION
TRUNCATE PARTITION
REORGANIZE PARTITION

In addition, PARTIONED S3 TABLES can also be replicated.
This is achived by storing the partition tables .frm and .par file on S3
for partitioned shared (S3) tables.

The discovery methods are enchanced by allowing engines that supports
discovery to also support of the partitioned tables .frm and .par file

Things in more detail

- The .frm and .par files of partitioned tables are stored in S3 and kept
  in sync.
- Added hton callback create_partitioning_metadata to inform handler
  that metadata for a partitoned file has changed
- Added back handler::discover_check_version() to be able to check if
  a table's or a part table's definition has changed.
- Added handler::check_if_updates_are_ignored(). Needed for partitioning.
- Renamed rebind() -> rebind_psi(), as it was before.
- Changed CHF_xxx hadnler flags to an enum
- Changed some checks from using table->file->ht to use
  table->file->partition_ht() to get discovery to work with partitioning.
- If TABLE_SHARE::init_from_binary_frm_image() fails, ensure that we
  don't leave any .frm or .par files around.
- Fixed that writefrm() doesn't leave unusable .frm files around
- Appended extension to path for writefrm() to be able to reuse to function
  for creating .par files.
- Added DBUG_PUSH("") to a a few functions that caused a lot of not
  critical tracing.
2020-04-19 17:33:51 +03:00
Monty
f9f33b85be Handle errors from external_unlock & mysql_unlock_tables
Other things:
- Handler errors from ha_maria::implict_commit
- Disable DBUG in safe_mutex_lock to get trace file easier to read
2020-04-19 17:33:51 +03:00
Vladislav Vaintroub
e2a932c9ea Post-fixes for MDEV-18851 for Windows
- On Windows, do not handle lack of SeLockMemory privilege as fatal error.
Just like on any other platform, there is a fallback to ordinary pages.
It is better than server that does not start, silently.

- On Windows, remove incorrect  irritating "fallback to conventional pages failed"
from the warning, when allocating large pages fails.
2020-04-17 13:11:07 +02:00
Marko Mäkelä
af91266498 Merge 10.3 into 10.4
In main.index_merge_myisam we remove the test that was added in
commit a2d24def8c because
it duplicates the test case that was added in
commit 5af12e4635.
2020-04-16 12:12:26 +03:00
Sergey Vojtovich
5679a2b6b3 Shrink my_atomic.h and my_cpu.h scope 2020-04-15 22:23:03 +04:00
Sergey Vojtovich
10cdf5230d my_file_opened to my_atomic 2020-04-15 19:14:04 +04:00
Marko Mäkelä
84db10f27b Merge 10.2 into 10.3 2020-04-15 09:56:03 +03:00
Daniel Black
a0d5894015 my_pagepages: perror -> my_error 2020-04-08 14:51:14 +04:00
Daniel Black
a535d4d1a6 my_largepage: fprintf -> my_{printf_,}error 2020-04-08 14:51:14 +04:00
Daniel Black
11aaf5c8d2 add EE_BADMEMORY_RELEASE 2020-04-08 14:51:14 +04:00
Daniel Black
5e86b2eec8 my_large_malloc: style fix 2020-04-08 14:51:14 +04:00
Daniel Black
2c00502014 my_largepage: reduce includes already in my_global.h 2020-04-08 14:51:14 +04:00
Daniel Black
96d4b228ea my_large_pages: simplify solaris constants 2020-04-08 14:51:14 +04:00
Daniel Black
7b7a9161e2 my_large_pages: remove conventional memory(my_malloc_lock) fallback
Both Windows and MMAP capable implementations fell back to a
non-MEM_LARGE_PAGES/HugeTLB allocation with the large page implementaion
failed. These can can be freed by the corresponding function.

Prior to this, if we fell back to a conventional memory, than will
results in deallocation using munmap/VirtualFree on a memory allocated
using my_malloc_lock. At worst this could succeed and the
my_malloc_lock looses its memory without knowing about it.
2020-04-08 14:51:14 +04:00
Sergey Vojtovich
22811a1c60 Fixed build failure 2020-04-05 17:32:06 +04:00
Daniel Black
113e227e26 my_largepage.c: cleanup
Order to avoid prototypes:

my_get_large_page_sizes
my_next_large_page
my_init_large_pages
my_large_malloc
my_large_free
2020-04-05 17:20:50 +04:00
Daniel Black
9bf3a3a47a HAVE_LARGE_PAGES no longer global 2020-04-05 17:20:50 +04:00
Daniel Black
71337a4452 my_large_malloc/free fall back to my_{malloc|free}_lock
For platforms != Windows and without MMAP.
2020-04-05 17:20:50 +04:00
Daniel Black
2a18e783ca my_large_page: more verbose errors on allocation fallback/failure 2020-04-05 17:20:50 +04:00
Daniel Black
d5568e7591 my_large_malloc_int consolidated into my_large_malloc
Consolidated single page size implementations by populating
my_large_page_sizes with a single page size.
2020-04-05 17:20:50 +04:00