This commit fixes a documentation installation
issue (for debian packaging) and generally brings
the installation control files up to date (as for
the rest of the components).
submodules.cmake: don't use "--depth 1" with old Git
Old Git may not work with "--depth 1" when the referenced commit hash
is far from HEAD.
Newer Git improves the situation. For example:
fb43e31f2b
It's safe to not use "--depth 1" with old Git.
Closes#2049
Fixed inlining flags. Remove /Ob1 added by CMake for RelWithDebInfo.
(the actual compiler default is /Ob2 if optimizations are enabled)
Allow to define custom /Ob flag with new variable MSVC_INLINE, if desired
Summary of changes
- MD_CTX_SIZE is increased
- EVP_CIPHER_CTX_buf_noconst(ctx) does not work anymore, points
to nobody knows where. The assumption made previously was that
(since the function does not seem to be documented)
was that it points to the last partial source block.
Add own partial block buffer for NOPAD encryption instead
- SECLEVEL in CipherString in openssl.cnf
had been downgraded to 0, from 1, to make TLSv1.0 and TLSv1.1 possible
(according to https://github.com/openssl/openssl/blob/openssl-3.0.0/NEWS.md
even though the manual for SSL_CTX_get_security_level claims that it
should not be necessary)
- Workaround Ssl_cipher_list issue, it now returns TLSv1.3 ciphers,
in addition to what was set in --ssl-cipher
- ctx_buf buffer now must be aligned to 16 bytes with openssl(
previously with WolfSSL only), ot crashes will happen
- updated aes-t , to be better debuggable
using function, rather than a huge multiline macro
added test that does "nopad" encryption piece-wise, to test
replacement of EVP_CIPHER_CTX_buf_noconst
The warning reads:
CMake Deprecation Warning at CMakeLists.txt:101 (CMAKE_MINIMUM_REQUIRED):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.
Add SYSTEMD_READWRITEPATH-variable to mariadb{@,}.service.in to make sure that
if one is not building RPM or DEB packages then make sure there is ReadWritePaths
directive is defined in systemd service file.
This ensures that tar-ball installation has permissions to write database default
installation path (default: /usr/local/mysql/data) even if it's located
under /usr. Writing to that location is prevented by 'ProtectSystem=full'
systemd directive by default.
Prefixing the path with "-" in systemd causes there to not be an error if the
path doesn't exist. This may occur if the user has configured a datadir
elsewhere.
Reviewer: Daniel Black
- Use corresponding entry in the manifest, as described in
https://docs.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page
- If if ANSI codepage is UTF8 (i.e for Windows 1903 and later)
Use UTF8 as default client charset
Set console codepage(s) to UTF8, in case process is using console
- Allow some previously disabled MTR tests, that used Unicode for in "exec",
for the recent Windows versions
CreateServiceA, OpenServiceA, and couple of other functions do not work
correctly with non-ASCII character, in the special case where application
has defined activeCodePage=UTF8.
Workaround by redefining affected ANSI functions to own wrapper, which
works by converting narrow(ANSI) to wide, then calling wide function.
Deprecate original ANSI service functions, via declspec, so that we can catch
their use.
If it is not enabled, build wsrep as static "stub" library from wsrep_dummy.cc
´
Allow static plugins to export symbols (on Unix)
wsrep_info relies on wsrep defined symbols (e.g LOCK_wsrep_config_state)
exported from mysqld
If it is not enabled, build wsrep as static "stub" library from wsrep_dummy.cc
´
Allow static plugins to export symbols (on Unix)
wsrep_info relies on wsrep defined symbols (e.g LOCK_wsrep_config_state)
exported from mysqld
bzip2/lz4/lzma/lzo/snappy compression is now provided via *services*
they're almost like normal services, but in include/providers/
and they're supposed to provide exactly the same interface
as original compression libraries (but not everything,
only enough of if for the code to compile).
the services are implemented via dummy functions that return
corresponding error values (LZMA_PROG_ERROR, LZO_E_INTERNAL_ERROR, etc).
the actual compression libraries are linked into corresponding
provider plugins. Providers are daemon plugins that when loaded
replace service pointers to point to actual compression functions.
That is, run-time dependency on compression libraries is now on plugins,
and the server doesn't need any compression libraries to run, but
will automatically support the compression when a plugin is loaded.
InnoDB and Mroonga use compression plugins now. RocksDB doesn't,
because it comes with standalone utility binaries that cannot
load plugins.
`mytop` and `my_print_defaults` for RPM
- Add `mytop` to client package
- Add man page of `my_print_defaults` to client package
- Add dependencies for RPMs
- Remove old comment
- Remove dead link
Reviewed by: serg@mariadb.com
libfmt 6.1 is in Ubuntu focal repositories.
it's easy to compile with 6.1 by doing
#define detail internal
but 6.1 also produces different results in main.func_sformat
and we want MariaDB to behave identically everywhere.
SFORMAT() SQL function that uses fmtlib (https://fmt.dev/)
for python-like (also Rust, C++20, etc) string formatting
Only fmtlib 7.0.0+ is supported, older fmtlib
produces different results in the test.
No native support for temporal and decimal values,
* TIME_RESULT is handled as STRING_RESULT
* DECIMAL_RESULT as REAL_RESULT
- Add new Ninja and Clang build jobs. This helps to ensure those
toolchains also work in addition to default CMake/gcc.
- Generate dependencies.dot/png to illustrate the CMake/Make/Ninja
build dependencies. Viewing this image and identifying bottle necks
in parallelism can help make the build run faster.
- Enable CUnit tests now as they are fixed on 10.6 (MDEV-25820).
- Limit parallel builds to 2 CPUs (full parallelism needs MDEV-25968) on
CMake/Make. Now only the Ninja builds run full parallel builds as only
Ninja is smart enough to prevent builds failing on resource
over-consumption.
- Enable Gitlab-CI cache for job 'centos8' for ccache so that it builds
faster. Don't use Gitlab-CI cache for other jobs, as it would too easily
use up all free tier storage on Gitlab.com and force users to get a paid
account just for MariaDB builds.
- On other jobs clean away ccache, as it only had a 5% hit rate on single
builds with no downloaded cache.
- Dump full database contents during the test install so that one can
use diff to compare the database contents at different stages and thus
track/debug potential bugs in mariadb-install-db and mariadb-upgrade
code.
Bugfixes:
- Zero out ccache stats before each run so that 'ccache -s' would actually
show the stats for the latest run.
this fix is adding alternative name to MariaDB-common on Fedora.
the fix is placed outside of IF/ELSEIF blocks to do not overwrite
existing one for Fedora.
added alternative name for MariaDB-devel package to replace
mariadb-connector-c-devel from RHEL 8 distribution.
this patch is for 10.3+ on RHEL/Centos 8
A conversion warning 4267 that we want to disable(prior to 10.3),
was suppressed with cmake VS generator for C++ and C, despite being set
only for CXX flags.
The fix is to disable the warning in C flags, too. In 10.2, this warning
is noisy, in 10.3 it is fixed.
With this patch, 4-component MSI version can be used, e.g by setting
TINY_VERSION variable in CMake, or by adding a string, e.g
MYSQL_VERSION_EXTRA=-2
which sets TINY_VERSION to 2, and also changes the package name.
The 4-component MSI versions do not support MSI major upgrades, only minor
ones, i.e do not reinstall components, just update existing ones based
on versioning rules.
To support these rules, add DefaultVersion for the files that won't
otherwise be versioned - headers, static and import libraries,
pdbs, text - xml, python and perl scripts Also silence WiX warning
that MSI won't store hashes for those files anymore.
Introduce -DFAST_BUILD parameter for a little faster build or test
if set,
- do not compile with /d2OptimizeHugeFunctions, this makes compilation
of bison output much slower on optimized build
- do not use runtime checks on debug build (RTC1). This slows down tests
considerably
replaced CPACK_RPM_PACKAGE_VERSION with SERVER_VERSION.
CPACK_* variables are empty and can't be used until INCLUDE(CPack) is
called.
SERVER_VERSION is the safest option because other variables may be
overwritten from submodules
Currently @@version_compile_machine is used by mtr
to determine if the compiled executable is 32 or 64
bits.
We extend that logic by ensuring that if the DEFAUT_MACHINE
name doesn't have "64" in its string, "-64bits" is appended
to ensure these test pass.
CMake Warning (dev) at cmake/plugin.cmake:288 (GET_TARGET_PROPERTY):
Policy CMP0045 is not set: Error on non-existent target...
Also, fix condition argument in ADD_FEATURE_INFO. It is not a string
but boolean
CMake Warning (dev) at cmake/plugin.cmake:288 (GET_TARGET_PROPERTY):
Policy CMP0045 is not set: Error on non-existent target...
Also, fix condition argument in ADD_FEATURE_INFO. It is not a string
but boolean
1. rename option DEPENDENCIES in MYSQL_ADD_PLUGIN() to DEPENDS
to be consistent with other cmake commands and macros
2. use this DEPENDS option in plugins
3. add dependencies to the plugin embedded target too
4. plugins don't need to add GenError dependency explicitly,
all plugins depend on it automatically
This gives a short overview over found/missing dependencies as well
as enabled/disabled features.
Initial author Heinz Wiesinger <heinz@m2mobi.com>
Additions by Vicențiu Ciorbaru <vicentiu@mariadb.org>
* Report all plugins enabled via MYSQL_ADD_PLUGIN
* Simplify code. Eliminate duplication by making use of WITH_xxx
variable values to set feature "ON" / "OFF" state.
Reviewed by: wlad@mariadb.com (code details) serg@mariadb.com (the idea)
mysql_config should never return -lzlib as linking flag.
Clients trying to link against mariadb that use the output of
mysql_config experience linking error because we return -lzlib
when we don't ship any libzlib.a in our binary tarballs, bundled
zlib static library is already linked.
added %config directive to /etc/logrotate.d/mysql and put it to server
package. if file is edited by customer and defaults are changed - new
.rpmnew file with defaults will be created next to old one.
Not all environments have 'diff' installed. Most notably CentOS 8
does not have diff out-of-the-box. Thus users running 'cmake .' and
'make' would fail to build MariaDB, and they would think the error
was in ABI incompatibilities due to the error message emitted by CMake
when in reality simply 'diff' was missing.
This fixes it and makes the developer experience better by simply skipping
the diffing if 'diff' is not found.
## Proof
Running in a clean CentOS 8 container.
### Before
```
$ cmake .
...
-- Looking for cpuid.h
-- Looking for cpuid.h - found
-- Looking for x86intrin.h
-- Looking for x86intrin.h - found
-- Looking for event.h
-- Looking for event.h - not found
-- Configuring done
-- Generating done
-- Build files have been written to: /build
$ make
Scanning dependencies of target abi_check
CMake Error at cmake/do_abi_check.cmake:82 (MESSAGE):
ABI check found difference between /build/include/mysql/plugin_audit.h.pp
and /build/abi_check.out
make[2]: *** [CMakeFiles/abi_check.dir/build.make:57: CMakeFiles/abi_check] Error 1
make[1]: *** [CMakeFiles/Makefile2:168: CMakeFiles/abi_check.dir/all] Error 2
make: *** [Makefile:163: all] Error 2
$ yum install -y diffutils
...
Installed:
diffutils-3.6-6.el8.x86_64
Complete!
$ make
[ 0%] Built target abi_check
Scanning dependencies of target INFO_BIN
[ 0%] Built target INFO_BIN
Scanning dependencies of target INFO_SRC
[ 0%] Built target INFO_SRC
Scanning dependencies of target wsrep_api_v26
[ 0%] Building C object wsrep-lib/wsrep-API/CMakeFiles/wsrep_api_v26.dir/v26/wsrep_dummy.c.o
[ 0%] Building C object wsrep-lib/wsrep-API/CMakeFiles/wsrep_api_v26.dir/v26/wsrep_gtid.c.o
[ 0%] Building C object wsrep-lib/wsrep-API/CMakeFiles/wsrep_api_v26.dir/v26/wsrep_loader.c.o
[ 0%] Building C object wsrep-lib/wsrep-API/CMakeFiles/wsrep_api_v26.dir/v26/wsrep_uuid.c.o
[ 0%] Linking C static library libwsrep_api_v26.a
[ 0%] Built target wsrep_api_v26
```
### After
```
$ make
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_audit.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_ftparser.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/psi/psi_abi_v1.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/psi/psi_abi_v2.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/client_plugin.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_auth.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_password_validation.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_encryption.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_data_type.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_function.h skipped.
[ 0%] Built target abi_check
[ 0%] Built target INFO_SRC
[ 0%] Built target INFO_BIN
[ 0%] Built target wsrep_api_v26
[ 0%] Building CXX object wsrep-lib/src/CMakeFiles/wsrep-lib.dir/server_state.cpp.o
```
If diff is installed, those warnings are simply not shown. Builds pass
without the need to install 'diff'.
Brad Smith made this OpenBSD file based of the
FreeBSD cmake directives in commit ab58904367
by the Monty Program Ab.
As such the Oracle Copyright header isn't really applicable.
Not all environments have 'diff' installed. Most notably CentOS 8
does not have diff out-of-the-box. Thus users running 'cmake .' and
'make' would fail to build MariaDB, and they would think the error
was in ABI incompatibilities due to the error message emitted by CMake
when in reality simply 'diff' was missing.
This fixes it and makes the developer experience better by simply
skipping the diffing if 'diff' is not found.
Closes#1846
RocksDB failed to package on fedora 32 and 33 with
*** ERROR: ambiguous python shebang in /usr/bin/myrocks_hotbackup: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
This fixed the MySQL bug# 20338 about misuse of double underscore
prefix __WIN__, which was old MySQL's idea of identifying Windows
Replace it by _WIN32 standard symbol for targeting Windows OS
(both 32 and 64 bit)
Not that connect storage engine is not fixed in this patch (must be
fixed in "upstream" branch)
- As solution `PLUGIN_CONNECT=NO` use early check to disable plugin:
Solution suggested by wlad@mariadb.com
- `JNI_FOUND` is a internal result variable and should be set with
cached library and header variables (like `JAVA_INCLUDE_PATH`) defined.
* Note: wrapper cmake/FindJNI.cmake runs first time and cmake native Find<module> returns only cached variable, like `JAVA_INCLUDE_PATH`, results variable are not cached).
Reviewed by: serg@mariadb.com
- As solution `PLUGIN_CONNECT=NO` use early check to disable plugin:
Solution suggested by wlad@mariadb.com
- `JNI_FOUND` is a internal result variable and should be set with
cached library and header variables (like `JAVA_INCLUDE_PATH`) defined.
* Note: wrapper cmake/FindJNI.cmake runs first time and cmake native Find<module> returns only cached variable, like `JAVA_INCLUDE_PATH`, results variable are not cached).
Reviewed by: serg@mariadb.com
If IPO is enabled, MSVC compiles with /GL which is not
compatible with cmake's WINDOWS_EXPORT_ALL_SYMBOLS.
Since server.dll can't autoexport symbols from /GL compiled objects,
IPO must be disabled for dbug, string,mysys, and sql targets (in case we
build "plugin" DLLs that need server.dll's symbols)
The easiest way to compile and test the server with UBSAN is to run:
./BUILD/compile-pentium64-ubsan
and then run mysql-test-run.
After this commit, one should be able to run this without any UBSAN
warnings. There is still a few compiler warnings that should be fixed
at some point, but these do not expose any real bugs.
The 'special' cases where we disable, suppress or circumvent UBSAN are:
- ref10 source (as here we intentionally do some shifts that UBSAN
complains about.
- x86 version of optimized int#korr() methods. UBSAN do not like unaligned
memory access of integers. Fixed by using byte_order_generic.h when
compiling with UBSAN
- We use smaller thread stack with ASAN and UBSAN, which forced me to
disable a few tests that prints the thread stack size.
- Verifying class types does not work for shared libraries. I added
suppression in mysql-test-run.pl for this case.
- Added '#ifdef WITH_UBSAN' when using integer arithmetic where it is
safe to have overflows (two cases, in item_func.cc).
Things fixed:
- Don't left shift signed values
(byte_order_generic.h, mysqltest.c, item_sum.cc and many more)
- Don't assign not non existing values to enum variables.
- Ensure that bool and enum values are properly initialized in
constructors. This was needed as UBSAN checks that these types has
correct values when one copies an object.
(gcalc_tools.h, ha_partition.cc, item_sum.cc, partition_element.h ...)
- Ensure we do not called handler functions on unallocated objects or
deleted objects.
(events.cc, sql_acl.cc).
- Fixed bugs in Item_sp::Item_sp() where we did not call constructor
on Query_arena object.
- Fixed several cast of objects to an incompatible class!
(Item.cc, Item_buff.cc, item_timefunc.cc, opt_subselect.cc, sql_acl.cc,
sql_select.cc ...)
- Ensure we do not do integer arithmetic that causes over or underflows.
This includes also ++ and -- of integers.
(Item_func.cc, Item_strfunc.cc, item_timefunc.cc, sql_base.cc ...)
- Added JSON_VALUE_UNITIALIZED to json_value_types and ensure that
value_type is initialized to this instead of to -1, which is not a valid
enum value for json_value_types.
- Ensure we do not call memcpy() when second argument could be null.
- Fixed that Item_func_str::make_empty_result() creates an empty string
instead of a null string (safer as it ensures we do not do arithmetic
on null strings).
Other things:
- Changed struct st_position to an OBJECT and added an initialization
function to it to ensure that we do not copy or use uninitialized
members. The change to a class was also motived that we used "struct
st_position" and POSITION randomly trough the code which was
confusing.
- Notably big rewrite in sql_acl.cc to avoid using deleted objects.
- Changed in sql_partition to use '^' instead of '-'. This is safe as
the operator is either 0 or 0x8000000000000000ULL.
- Added check for select_nr < INT_MAX in JOIN::build_explain() to
avoid bug when get_select() could return NULL.
- Reordered elements in POSITION for better alignment.
- Changed sql_test.cc::print_plan() to use pointers instead of objects.
- Fixed bug in find_set() where could could execute '1 << -1'.
- Added variable have_sanitizer, used by mtr. (This variable was before
only in 10.5 and up). It can now have one of two values:
ASAN or UBSAN.
- Moved ~Archive_share() from ha_archive.cc to ha_archive.h and marked
it virtual. This was an effort to get UBSAN to work with loaded storage
engines. I kept the change as the new place is better.
- Added in CONNECT engine COLBLK::SetName(), to get around a wrong cast
in tabutil.cpp.
- Added HAVE_REPLICATION around usage of rgi_slave, to get embedded
server to compile with UBSAN. (Patch from Marko).
- Added #ifdef for powerpc64 to avoid a bug in old gcc versions related
to integer arithmetic.
Changes that should not be needed but had to be done to suppress warnings
from UBSAN:
- Added static_cast<<uint16_t>> around shift to get rid of a LOT of
compiler warnings when using UBSAN.
- Had to change some '/' of 2 base integers to shift to get rid of
some compile time warnings.
Reviewed by:
- Json changes: Alexey Botchkov
- Charset changes in ctype-uca.c: Alexander Barkov
- InnoDB changes & Embedded server: Marko Mäkelä
- sql_acl.cc changes: Vicențiu Ciorbaru
- build_explain() changes: Sergey Petrunia
