The policy is not set for 10.2
If it is set, CMake would complain about bundled zlib for which the policy
is not set.
Fix:
- Set policy for 10.2 for the top level project.
For 10.3+ it was already set
- Cleanup zlib to remove unneeded stuff. It is an internal static library,
it needs none of PROJECT, library versioning, RC file on Windows.
The name of the library on Unix does not make any difference, since it is
static and compiled in.
-Wl,-z,relro,-z,now are linker flags and should
be checked as such.
TODO: perform module, exe shared checks separately
rather than a pure linker check.
Existing implementation used my_checksum (from mysys)
for calculating table checksum and binlog checksum.
This implementation was optimized for powerpc only and lacked
SIMD implementation for x86 (using clmul) and ARM
(using ACLE) instead used zlib-crc32.
mariabackup had its own copy of the crc32 implementation
using hardware optimized implementation only for x86 and lagged
hardware based implementation for powerpc and ARM.
Patch helps unifies all such calls and help aggregate all of them
using an unified interface my_checksum().
Said unification also enables hardware optimized calls for all
architecture viz. x86, ARM, POWERPC.
Default always fallback to zlib crc32.
Thanks to Daniel Black for reviewing, fixing and testing
PowerPC changes. Thanks to Marko and Daniel for early code feedback.
setup alternative name for rhel8/centos8
use correct C/C version when providing mariadb-connector-c
and other changes for installation over distro packages to work:
dnf install mariadb-server
dnf install --allowerasing MariaDB-{server,common,client,shared}*.rpm
Disable IPO (interprocedural optimization, aka /GL) on Windows
on libraries, from which server.dll exports symbols - exporting symbols
does not work for objects compiled with /GL.
This is a way do disable DBUG_ENTER()/DBUG_EXIT() stuff which is
needed to dbug trace. Those who doesn't need it may avoid tests
slowdown with -DWITH_DBUG_TRACE=OFF
dbug/tests.c: add define which is neede always in this test
innodb.log_file_name_debug.test: do not depend on DBUG trace stuff
in test
Benchmark results: each test eats less CPU and you can have more
parallel jobs in MTR.
patched:
./mtr -mem -par=8 -suite=innodb 185.34s user 86.85s system 133% cpu 3:23.27 total
./mtr -mem -par=8 -suite=main 80.96s user 36.01s system 182% cpu 1:04.07 total
main.select [ pass ] 1660
main.select [ pass ] 1513
main.select [ pass ] 1543
main.select [ pass ] 1660
main.select [ pass ] 1521
main.select [ pass ] 1511
main.select [ pass ] 1508
main.select [ pass ] 1520
main.select [ pass ] 1514
main.select [ pass ] 1522
vanilla:
./mtr -mem -par=8 -suite=innodb 203.61s user 92.16s system 140% cpu 3:30.16 total
./mtr -mem -par=8 -suite=main 94.11s user 35.51s system 206% cpu 1:02.69 total
main.select [ pass ] 2032
main.select [ pass ] 2017
main.select [ pass ] 2040
main.select [ pass ] 2183
main.select [ pass ] 2253
main.select [ pass ] 2075
main.select [ pass ] 2109
main.select [ pass ] 2080
main.select [ pass ] 2098
main.select [ pass ] 2114
This task deals with packing the sort key inside the sort buffer, which would
lead to efficient usage of the memory allocated for the sort buffer.
The changes brought by this feature are
1) Sort buffers would have sort keys of variable length
2) The format for sort keys inside the sort buffer would look like
|<sort_length><null_byte><key_part1><null_byte><key_part2>.......|
sort_length is the extra bytes that are required to store the variable
length of a sort key.
3) When packing of sort key is done we store the ORIGINAL VALUES inside
the sort buffer and not the STRXFRM form (mem-comparable sort keys).
4) Special comparison function packed_keys_comparison() is introduced
to compare 2 sort keys.
This patch also contains contributions from Sergei Petrunia.
Cleanup install_layout to account for multi-arch setup and remove
redundant defines in debian rules.
Signed-off-by: Vicențiu Ciorbaru <vicentiu@mariadb.org>
Also added support for MAP_SYNC. It allows to achieve decent performance
with DAX devices even when libpmem is unavailable.
Fixed Windows version of my_msync(): according to manual FlushViewOfFile()
may return before flush is actually completed. It is advised to issue
FlushFileBuffers() after FlushViewOfFile().
The library is capable of
- asynchronous execution of tasks (and optionally waiting for them)
- asynchronous file IO
This is implemented using libaio on Linux and completion ports on
Windows. Elsewhere, async io is "simulated", which means worker threads
are performing synchronous IO.
- timers, scheduling work asynchronously in some point of the future.
Also periodic timers are implemented.
The MDEV-20265 commit e746f451d5
introduces DBUG_ASSERT(right_op == r_tbl) in
st_select_lex::add_cross_joined_table(), and that assertion would
fail in several tests that exercise joins. That commit was skipped
in this merge, and a separate fix of MDEV-20265 will be necessary in 10.4.
MemorySanitizer is a compile-time instrumentation layer in clang and GCC.
Together with AddressSanitizer mostly makes the run-time instrumentation
of Valgrind redundant. It is a little more tricky to set up, because
running with uninstrumented libraries will lead into false positives.
You will need an instrumented libc++, and you should use
-stdlib=libc++ instead of the default libstdc++. To build the
instrumented library, you can refer to
https://github.com/google/sanitizers/wiki/MemorySanitizerLibcxxHowTo
or you can adapt these steps that worked for me, for clang-8 version 8.0.1:
cd /mariadb
sudo apt source libc++-8-dev
cd llvm-toolchain-8-8.0.1
mkdir libc++msan; cd libc++msan
cmake ../libcxx -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_SANITIZER=Memory \
-DCMAKE_C_COMPILER=clang-8 -DCMAKE_CXX_COMPILER=clang++-8
Then, in your MariaDB build directory, you have to compile with
libc++ and bundled libraries, such as WITH_SSL=bundled, WITH_ZLIB=bundled.
For uninstrumented system libraries, you will get false positives for
uninitialized values. Like this:
cmake -DWITH_MSAN=ON -DWITH_SSL=bundled -DWITH_ZLIB=bundled \
-DCMAKE_CXX_FLAGS='-stdlib=libc++' ..
Note: you should also add -O2 to the compiler options, or you may
get crashes due to stack overflow.
Finally, to run tests, you must replace libc++ with the instrumented one:
LD_LIBRARY_PATH=/mariadb/llvm-toolchain-8-8.0.1/libc++msan/lib \
MSAN_OPTIONS=abort_on_error=1 \
./mtr --big-test --parallel=auto --force --retry=0
Failure to do so will report numerous false positives related to
operations on std::string and the like.
This is work in progress. Some issues will still have to be fixed
for WITH_MSAN to be usable. See MDEV-20377 for details.
