Commit graph

157 commits

Author SHA1 Message Date
Marko Mäkelä
32062cc61c Merge 10.1 into 10.2 2018-11-06 08:41:48 +02:00
Sergei Golubchik
44f6f44593 Merge branch '10.0' into 10.1 2018-10-30 15:10:01 +01:00
Sergei Golubchik
37ab7e4596 Merge branch '5.5' into 10.0 2018-10-27 20:46:38 +02:00
Vladislav Vaintroub
acf8fc1ff8 Fix cmake warning
CMP0075 Include file check macros honor CMAKE_REQUIRED_LIBRARIES
2018-10-10 07:36:18 +01:00
Oleksandr Byelkin
0896d7ebc3 Merge branch '10.0' into bb-10.1-merge 2018-07-19 12:55:54 +02:00
Vladislav Vaintroub
0897a25c0f MDEV-16596 : Windows - redo log does not work on native 4K sector disks.
Disks with native 4K sectors need 4K alignment and size for  unbuffered IO
(i.e for files opened with FILE_FLAG_NO_BUFFERING)

Innodb opens redo log with FILE_FLAG_NO_BUFFERING, however it always does
512byte IOs. Thus, the IO on 4K native sectors will fail, rendering
Innodb non-functional.

The fix is to check whether OS_FILE_LOG_BLOCK_SIZE is multiple of logical
sector size, and if it is not, reopen the redo log without
FILE_FLAG_NO_BUFFERING flag.
2018-07-02 15:02:31 +01:00
Vladislav Vaintroub
c612a1e77c MDEV-16596 : Windows - redo log does not work on native 4K sector disks.
Disks with native 4K sectors need 4K alignment and size for  unbuffered IO
(i.e for files opened with FILE_FLAG_NO_BUFFERING)

Innodb opens redo log with FILE_FLAG_NO_BUFFERING, however it always does
512byte IOs. Thus, the IO on 4K native sectors will fail, rendering
Innodb non-functional.

The fix is to check whether OS_FILE_LOG_BLOCK_SIZE is multiple of logical
sector size, and if it is not, reopen the redo log without
FILE_FLAG_NO_BUFFERING flag.
2018-06-30 11:04:51 +01:00
Daniel Black
402c7584a8 MDEV-13785: move defination HAVE_LARGE_PAGES -> HAVE_LINUX_LARGE_PAGES
HAVE_LARGE_PAGES was always Linux but now there is
HAVE_SOLARIS_LARGE_PAGES in the code base. Innodb was using HAVE_LINUX_LARGE_PAGES
so keep this consistent everywhere.

Test plan:

$ grep Hugepagesize: /proc/meminfo
Hugepagesize:       2048 kB

$ sudo sysctl vm.nr_hugepages=1024
vm.nr_hugepages = 1024
$ sudo sysctl  kernel.shmmax=$(( 2 * 1024 *1024 * 1024 ))
kernel.shmmax = 2147483648

No errors in ouput:
$ sql/mysqld --skip-networking --datadir=/tmp/datadir --log-bin=/tmp/datadir/mysqlbin --socket /tmp/s.sock --lc-messages-dir=${PWD}/sql/share --verbose --large-pages=1
2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld (mysqld 10.2.14-MariaDB-log) starting as process 25406 ...
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Uses event mutexes
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Compressed tables use zlib 1.2.11
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using Linux native AIO
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Number of pools: 1
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using SSE2 crc32 instructions
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Completed initialization of buffer pool
2018-03-23 12:51:18 139696883590912 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Highest supported file format is Barracuda.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 128 out of 128 rollback segments are active.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Creating shared tablespace for temporary tables
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 5.7.21 started; log sequence number 1620099
2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Loading buffer pool(s) from /tmp/datadir/ib_buffer_pool
2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Buffer pool(s) load completed at 180323 12:51:18
2018-03-23 12:51:18 139697428129984 [Note] Plugin 'FEEDBACK' is disabled.
2018-03-23 12:51:18 139697428129984 [Note] Reading of all Master_info entries succeded
2018-03-23 12:51:18 139697428129984 [Note] Added new Master_info '' to hash table
2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld: ready for connections.
Version: '10.2.14-MariaDB-log'  socket: '/tmp/s.sock'  port: 0  Source distribution

$ grep -i huge /proc/25406/smaps | grep -v ' 0 kB'
Private_Hugetlb:    8192 kB
Private_Hugetlb:    2048 kB
$ grep huge /proc/25406/numa_maps
7f0d74400000 default file=/SYSV00000000\040(deleted) huge
7f0dbd200000 default file=/SYSV00000000\040(deleted) huge dirty=4 N0=4 kernelpagesize_kB=2048
7f0dc5600000 default file=/SYSV00000000\040(deleted) huge
7f0dd1200000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048

$ grep Huge  /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     940
HugePages_Free:      935
HugePages_Rsvd:      177
HugePages_Surp:        0
Hugepagesize:       2048 kB

Ran again with --memlock
(note needs ulimit -l > size)

$ grep Huge  /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     940
HugePages_Free:      758
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

$ grep huge /proc/26020/numa_maps
7fe870400000 default file=/SYSV00000000\040(deleted) huge dirty=62 N0=62 kernelpagesize_kB=2048
7fe8b3a00000 default file=/SYSV00000000\040(deleted) huge dirty=66 N0=66 kernelpagesize_kB=2048
7fe8bd600000 default file=/SYSV00000000\040(deleted) huge dirty=53 N0=53 kernelpagesize_kB=2048
7fe8c8400000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048

$ grep -i huge /proc/26020/smaps | grep -v ' 0 kB'
Private_Hugetlb:  126976 kB
Private_Hugetlb:  135168 kB
Private_Hugetlb:  108544 kB
Private_Hugetlb:    2048 kB
2018-03-29 18:55:18 +04:00
Vladislav Vaintroub
24462cece4 Disable noisy warning in old compiler (VS2015) 2018-02-14 19:13:08 +00:00
Vladislav Vaintroub
2dd8a732f3 Windows, compiling - disable pkg_config
pkg_config usually comes with Strawberry perl, and tends to find packages
that might work in mingw compilation, but not with MSVC.

Thus disable PKG_CONFIG, otherwise any FIND_PACKAGE() that is using PkgConfig
can find something (like LibXml2 from connect), can potentially find something
that is not going to compile.
2018-02-14 17:00:13 +00:00
Vladislav Vaintroub
067b90c7a9 Fix MinSizeRel build on Windows.
do not include test suite in release zip anymore.
2018-01-26 10:37:46 +00:00
Vladislav Vaintroub
477a1bc42b Windows : fix compile warnings C4267, on 32bit first 2018-01-26 10:37:46 +00:00
Vladislav Vaintroub
9891ee5a2a Fix and reenable Windows compiler warning C4800 (size_t conversion). 2018-01-26 10:37:46 +00:00
Vladislav Vaintroub
b132d4d749 Windows, compilation : Treat warning as error, if MYSQL_MAINTAINER_MODE
is set to ERR

This matches gcc/clang handling.
2018-01-09 22:52:01 +00:00
Vladislav Vaintroub
7354dc6773 MDEV-13384 - misc Windows warnings fixed 2017-09-28 17:20:46 +00:00
Vladislav Vaintroub
eba44874ca MDEV-13844 : Fix Windows warnings. Fix DBUG_PRINT.
- Fix win64 pointer truncation warnings
(usually coming from misusing 0x%lx and long cast in DBUG)

- Also fix printf-format warnings

Make the above mentioned warnings fatal.

- fix pthread_join on Windows to set return value.
2017-09-28 17:20:46 +00:00
Marko Mäkelä
f192b48d62 Merge 10.1 into 10.2 2017-08-29 10:07:33 +03:00
Marko Mäkelä
11352d52cd Merge 10.0 into 10.1 2017-08-28 15:05:46 +03:00
Vladislav Vaintroub
dd229430a9 Windows compile : make compilation fail on "uninitialized variable used" warning C4700
This is a genuine error, and will crash debug buildd in runtime checks if not fixed.
it is better to fail during compile.
2017-08-24 08:05:11 +00:00
Vladislav Vaintroub
15878ee41c Windows, compiling : Remove _DEBUG preprocessor constant, to fix
debug build with older cmake.

The constant is implicitely defined by VS when chosen C runtime is Debug
(/MTd, MDd).

CMake does not define it since https://public.kitware.com/Bug/view.php?id=15777
was fixed.

We remove it from compile flags, to be able to build Debug with /MT runtime
using older cmakes.
2017-04-04 18:55:18 +00:00
Vladislav Vaintroub
0cca5bdf0b Allow to specify C runtime library used for compilation.
Default to static release (previously static debug was used in debug
builds, but not is appears to be too slow)
2017-04-02 17:29:04 +00:00
Marko Mäkelä
ec4cf111c0 MDEV-11520 after-merge fix for 10.1: Use sparse files.
If page_compression (introduced in MariaDB Server 10.1) is enabled,
the logical action is to not preallocate space to the data files,
but to only logically extend the files with zeroes.

fil_create_new_single_table_tablespace(): Create smaller files for
ROW_FORMAT=COMPRESSED tables, but adhere to the minimum file size of
4*innodb_page_size.

fil_space_extend_must_retry(), os_file_set_size(): On Windows,
use SetFileInformationByHandle() and FILE_END_OF_FILE_INFO,
which depends on bumping _WIN32_WINNT to 0x0600.
FIXME: The files are not yet set up as sparse, so
this will currently end up physically extending (preallocating)
the files, wasting storage for unused pages.

os_file_set_size(): Add the parameter "bool sparse=false" to declare
that the file is to be extended logically, instead of being preallocated.
The only caller with sparse=true is
fil_create_new_single_table_tablespace().
(The system tablespace cannot be created with page_compression.)

fil_space_extend_must_retry(), os_file_set_size(): Outside Windows,
use ftruncate() to extend files that are supposed to be sparse.
On systems where ftruncate() is limited to files less than 4GiB
(if there are any), fil_space_extend_must_retry() retains the
old logic of physically extending the file.
2017-02-22 22:29:56 +02:00
Sergei Golubchik
4a5d25c338 Merge branch '10.1' into 10.2 2016-12-29 13:23:18 +01:00
Sergei Golubchik
2f20d297f8 Merge branch '10.0' into 10.1 2016-12-11 09:53:42 +01:00
Sergei Golubchik
3e8155c637 Merge branch '5.5' into 10.0 2016-12-09 16:33:48 +01:00
Vladislav Vaintroub
d8cb6822bc VS2015 build fixes
- new location of signtool
- silence a nonsensical warning from stl  header
(complain about noexcept() function attribute, if /EHsc is not set)
2016-10-27 19:45:44 +00:00
Vladislav Vaintroub
3a7bc23a16 MDEV-9154 : Remove workarounds (mainly dynamic function loading)
for running obsolete versions of Windows
2016-06-01 20:29:10 +02:00
Sergei Golubchik
c4cb240061 MDEV-9024 Build fails with VS2015
cherry-pick f1daf9ce from 10.0 branch
-------------------------------------

Fix build failures caused by new C runtime library
- isnan, snprintf, struct timespec are now defined, attempt to
redefine them leads
- P_tmpdir, tzname are no more defined
-  lfind() and lsearch() in lf_hash.c had to be renamed,  declaration
conflicts with some C runtime functions with the same name declared in
a header included by stdlib.h

 Also fix couple of annoying warnings :
- remove #define NOMINMAX from config.h to avoid "redefined" compiler
warnings(NOMINMAX is already in compile flags)

- disable incremental linker in Debug as well (feature not used much
and compiler crashes often)

Also simplify package building with Wix, require Wix 3.9 or later
(VS2015 is not compatible with old Wix 3.5/3.6)
2016-02-06 22:41:58 +01:00
Sergei Golubchik
65d69c84cf cmake: remove unused checks, options, and symbols 2015-11-23 16:02:56 +01:00
Sergei Golubchik
beded7d9c9 Merge branch '10.0' into 10.1 2015-11-19 15:52:14 +01:00
Sergei Golubchik
29dd634a4c dbug: correct trace for DBUG_RETURN(func()); -- gcc only
when func1 calls func2 from DBUG_RETURN, dbug shows the trace as
| > func1
| < func1
| > func2
| < func2
because DBUG_LEAVE happens before func2(). Change that to invoke
DBUG_LEAVE when the local variable goes out of scope. This uses
gcc specific __attribute__((cleanup)).
2015-11-16 07:55:55 +01:00
Vladislav Vaintroub
f1daf9ced4 MDEV-9024 Build fails with VS2015
Fix build failures caused by new C runtime library
- isnan, snprintf, struct timespec are now defined, attempt to
redefine them leads
- P_tmpdir, tzname are no more defined
-  lfind() and lsearch() in lf_hash.c had to be renamed,  declaration
conflicts with some C runtime functions with the same name declared in
a header included by stdlib.h

 Also fix couple of annoying warnings :
- remove #define NOMINMAX from config.h to avoid "redefined" compiler
warnings(NOMINMAX is already in compile flags)

- disable incremental linker in Debug as well (feature not used much
and compiler crashes often)


Also simplify package building with Wix, require Wix 3.9 or later
(VS2015 is not compatible with old Wix 3.5/3.6)
2015-11-06 17:27:24 +01:00
Sergei Golubchik
dfb74dea30 Merge branch '10.0' into 10.1 2015-10-12 00:37:58 +02:00
Sergei Golubchik
cfeedbfd3e Merge branch '5.5' into 10.0 2015-10-09 17:12:26 +02:00
Sergei Golubchik
82e9f6d948 Merge remote-tracking branch 'mysql/5.5' into 5.5 2015-10-08 22:54:24 +02:00
Tor Didriksen
067ae38c0a Bug #20168526 YASSL: CORRUPT SSL-KEY CRASHES CLIENT
Post-push fix: broken build on windows.
The problem is min/max macros from windows.h
which interfere with a template function callex max.

Solution: ADD_DEFINITIONS(-DNOMINMAX)
2015-07-13 10:10:12 +02:00
Sergei Golubchik
658992699b Merge tag 'mariadb-10.0.20' into 10.1 2015-06-27 20:35:26 +02:00
Sergei Golubchik
66fd45afce MDEV-7398 mysqld segfaults on FreeBSD 10.1 i386 when built with clang 3.4
in cmake tests let's treat clang like gcc (same options,
same builtins) in many cases.

* don't check the compiler when
  * testing for -fvisibility=hidden support
  * testing for HAVE_ABI_CXA_DEMANGLE
  * testing for HAVE_GCC_ATOMIC_BUILTINS
  * when removing options with string(replace)
  * when running ${CC} --version (ignore the error instead)
* run ABI checks for clang
* use "canonical" gcc flags for clang
* fix groonga too

Also:

* add cmake detection for gcc __atomic_* builtins. they might be
  supported (__ATOMIC_SEQ_CST is defined), but not for all operand
  sizes. In particular, 64-bit atomic load is problematic on i386
* cache check results for Windows
* remove the test for HAVE_CXXABI_H (HAVE_ABI_CXA_DEMANGLE is
  suffifient)
2015-06-16 23:58:21 +02:00
Sergei Golubchik
4b21cd21fe Merge branch '10.0' into merge-wip 2015-01-31 21:48:47 +01:00
Sergei Golubchik
d9c01e4b4a 5.5 merge 2015-01-21 12:03:02 +01:00
Sergei Golubchik
a978bdda1e mysql-5.5.41 merge 2014-12-19 11:35:44 +01:00
Sergei Golubchik
f62c12b405 Merge 10.0.14 into 10.1 2014-10-15 12:59:13 +02:00
Jon Olav Hauglid
57eec7bc29 Bug#17633291: SET PROPER COMPILE OPTIONS FOR CLANG
Bug#17959689: MAKE GCC AND CLANG GIVE CONSISTENT COMPILATION WARNINGS
Bug#18313717: ENABLE -WERROR IN MAINTANER MODE WHEN COMPILING WITH CLANG
Bug#18510941: REMOVE CMAKE WORKAROUNDS FOR OLDER VERSIONS OF OS X/XCODE
  
Backport from mysql-5.6 to mysql-5.5
2014-10-13 09:52:28 +02:00
Jon Olav Hauglid
8b64f82505 Bug#17633291: SET PROPER COMPILE OPTIONS FOR CLANG
Bug#17959689: MAKE GCC AND CLANG GIVE CONSISTENT COMPILATION WARNINGS
Bug#18313717: ENABLE -WERROR IN MAINTANER MODE WHEN COMPILING WITH CLANG
Bug#18510941: REMOVE CMAKE WORKAROUNDS FOR OLDER VERSIONS OF OS X/XCODE
  
Backport from mysql-5.6 to mysql-5.5
2014-10-13 09:52:28 +02:00
Venkatesh Duggirala
0827d75627 Bug #18808072 MYSQLBINLOG USES LOCALTIME() TO PRINT EVENTS, CAUSES KERNEL MUTEX CONTENTION
Problem: For every event read, mysqlbinlog calls localtime() which in turn
calls stat(/etc/localtime) which is causing kernel mutex contention.

Analysis and Fix:
localtime() calls stat(/etc/localtime) for every instance of the call
where as localtime_r() the reentrant version was optimized to store
the read only tz internal structure. Hence it will not call
stat(/etc/localtime). It will call only once at the beginning.
The mysql server is calling localtime_r() and mysqlbinlog tool is
one place where we are still using localtime().

Once the process (mysqlbinlog) is started if timezone is changed
it will be not picked up the the process and it will continue
with the same values as the beginning of the process. This
behavior is in-lined with mysql server.

Also adding localtime_r() and gmtime_r() support for windows.
2014-10-08 21:54:35 +05:30
Venkatesh Duggirala
a3cc647dbd Bug #18808072 MYSQLBINLOG USES LOCALTIME() TO PRINT EVENTS, CAUSES KERNEL MUTEX CONTENTION
Problem: For every event read, mysqlbinlog calls localtime() which in turn
calls stat(/etc/localtime) which is causing kernel mutex contention.

Analysis and Fix:
localtime() calls stat(/etc/localtime) for every instance of the call
where as localtime_r() the reentrant version was optimized to store
the read only tz internal structure. Hence it will not call
stat(/etc/localtime). It will call only once at the beginning.
The mysql server is calling localtime_r() and mysqlbinlog tool is
one place where we are still using localtime().

Once the process (mysqlbinlog) is started if timezone is changed
it will be not picked up the the process and it will continue
with the same values as the beginning of the process. This
behavior is in-lined with mysql server.

Also adding localtime_r() and gmtime_r() support for windows.
2014-10-08 21:54:35 +05:30
Sergei Golubchik
2156f62d2e portability: use getifaddrs()
instead of exec'ing /usr/sbin/ifconfig|grep|sed|awk
2014-10-01 23:48:36 +02:00
Sergei Golubchik
1c6ad62a26 mysql-5.5.39 merge
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
2014-08-02 21:26:16 +02:00
Sergei Golubchik
6fb17a0601 5.5.39 merge 2014-08-07 18:06:56 +02:00
Sergei Golubchik
6c0e3ef450 cmake cleanup: use MY_CHECK_AND_SET_COMPILER_FLAG.
move MY_CHECK_C_COMPILER_FLAG and MY_CHECK_CXX_COMPILER_FLAG
to a separate file, introduce MY_CHECK_AND_SET_COMPILER_FLAG,
use it where possible.
2014-06-26 11:54:13 +02:00