Commit graph

6,601 commits

Author SHA1 Message Date
Vladislav Vaintroub
8ba0eea65c Fix warning VS2017 15.7 update.
This previously unreported warning comes from casting size_t to ulong
in sql_hset.h in Hash_Set::at().

Change my_hash_element to accept size_t index parameter.
2018-05-09 23:04:18 +01:00
Marko Mäkelä
4cd7979c56 Merge 10.1 into 10.2 2018-04-24 09:39:45 +03:00
Marko Mäkelä
9c34a4124d Merge 10.0 into 10.1 2018-04-24 09:26:40 +03:00
Sergei Golubchik
587568b72a Merge branch '5.5' into 10.0 2018-04-20 14:33:24 +02:00
Sergei Golubchik
1a019d0801 Merge branch 'mysql/5.5' into 5.5 2018-04-19 22:31:26 +02:00
Daniel Black
88ac368fea defaults-group-suffix in print_defaults
Also clarify which --{no-,}default* options, must be first.

Sample output:
$  client/mysql --help
client/mysql  Ver 15.1 Distrib 5.5.59-MariaDB, for Linux (x86_64) using readline 5.1
Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Usage: client/mysql [OPTIONS] [database]

Default options are read from the following files in the given order:
/etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf
The following groups are read: mysql client client-server client-mariadb
The following options may be given as the first argument:
--print-defaults          Print the program argument list and exit.
--no-defaults             Don't read default options from any option file.
The following specify which files/groups are read (specified before other options):
--defaults-file=#         Only read default options from the given file #.
--defaults-extra-file=#   Read this file after the global files are read.
--defaults-group-suffix=# Additionally read default groups with # appended as a suffix.

tests running from build directory:
TEST: print defaults ignored as not first
$ sql/mysqld  --no-defaults --print-defaults --lc-messages-dir=${PWD}/sql/share

TEST: no startup occurs as --print-defaults specified
$ sql/mysqld  --print-defaults --lc-messages-dir=${PWD}/sql/share
sql/mysqld would have been started with the following arguments:
--lc-messages-dir=/home/dan/repos/build-mariadb-5.5/sql/share

TEST: default args can't be anywhere
$ client/mysql --user=bob  --defaults-file=/etc/my.cnf
client/mysql: unknown variable 'defaults-file=/etc/my.cnf'
$ client/mysql --user=bob  --defaults-group-suffix=.group
client/mysql: unknown variable 'defaults-group-suffix=.group'

/etc/my.cnf:
[client-server.group]
socket=/var/lib/mysql-multi/group/mysqld.sock
user=bob

/etc/my.other.cnf:
socket=/var/lib/mysql-other/mysqld.sock

TEST: defaults file read and suffix also applied
$  client/mysql  --defaults-file=/etc/my.other.cnf  --defaults-group-suffix=.group
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql-other/mysqld.sock' (2)

TEST: defaults extra file
$ client/mysql  --defaults-extra-file=/etc/my.other.cnf  --defaults-group-suffix=.group
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql-other/mysqld.sock' (2)
2018-04-13 14:56:50 +04:00
Vicențiu Ciorbaru
45e6d0aebf Merge branch '10.1' into 10.2 2018-04-10 17:43:18 +03:00
Vicențiu Ciorbaru
4c89cff558 Merge branch '10.0' into 10.1 2018-04-07 17:11:22 +03:00
Vicențiu Ciorbaru
6a72b9096a Merge branch '5.5' into 10.0 2018-04-03 18:08:30 +03:00
Sergei Golubchik
df6197c8b9 compiler warning
warning: format '%p' expects argument of type 'void *', but argument 4 has type 'long int'
2018-04-03 16:19:10 +02:00
Daniel Black
402c7584a8 MDEV-13785: move defination HAVE_LARGE_PAGES -> HAVE_LINUX_LARGE_PAGES
HAVE_LARGE_PAGES was always Linux but now there is
HAVE_SOLARIS_LARGE_PAGES in the code base. Innodb was using HAVE_LINUX_LARGE_PAGES
so keep this consistent everywhere.

Test plan:

$ grep Hugepagesize: /proc/meminfo
Hugepagesize:       2048 kB

$ sudo sysctl vm.nr_hugepages=1024
vm.nr_hugepages = 1024
$ sudo sysctl  kernel.shmmax=$(( 2 * 1024 *1024 * 1024 ))
kernel.shmmax = 2147483648

No errors in ouput:
$ sql/mysqld --skip-networking --datadir=/tmp/datadir --log-bin=/tmp/datadir/mysqlbin --socket /tmp/s.sock --lc-messages-dir=${PWD}/sql/share --verbose --large-pages=1
2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld (mysqld 10.2.14-MariaDB-log) starting as process 25406 ...
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Uses event mutexes
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Compressed tables use zlib 1.2.11
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using Linux native AIO
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Number of pools: 1
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using SSE2 crc32 instructions
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Completed initialization of buffer pool
2018-03-23 12:51:18 139696883590912 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Highest supported file format is Barracuda.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 128 out of 128 rollback segments are active.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Creating shared tablespace for temporary tables
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 5.7.21 started; log sequence number 1620099
2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Loading buffer pool(s) from /tmp/datadir/ib_buffer_pool
2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Buffer pool(s) load completed at 180323 12:51:18
2018-03-23 12:51:18 139697428129984 [Note] Plugin 'FEEDBACK' is disabled.
2018-03-23 12:51:18 139697428129984 [Note] Reading of all Master_info entries succeded
2018-03-23 12:51:18 139697428129984 [Note] Added new Master_info '' to hash table
2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld: ready for connections.
Version: '10.2.14-MariaDB-log'  socket: '/tmp/s.sock'  port: 0  Source distribution

$ grep -i huge /proc/25406/smaps | grep -v ' 0 kB'
Private_Hugetlb:    8192 kB
Private_Hugetlb:    2048 kB
$ grep huge /proc/25406/numa_maps
7f0d74400000 default file=/SYSV00000000\040(deleted) huge
7f0dbd200000 default file=/SYSV00000000\040(deleted) huge dirty=4 N0=4 kernelpagesize_kB=2048
7f0dc5600000 default file=/SYSV00000000\040(deleted) huge
7f0dd1200000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048

$ grep Huge  /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     940
HugePages_Free:      935
HugePages_Rsvd:      177
HugePages_Surp:        0
Hugepagesize:       2048 kB

Ran again with --memlock
(note needs ulimit -l > size)

$ grep Huge  /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     940
HugePages_Free:      758
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

$ grep huge /proc/26020/numa_maps
7fe870400000 default file=/SYSV00000000\040(deleted) huge dirty=62 N0=62 kernelpagesize_kB=2048
7fe8b3a00000 default file=/SYSV00000000\040(deleted) huge dirty=66 N0=66 kernelpagesize_kB=2048
7fe8bd600000 default file=/SYSV00000000\040(deleted) huge dirty=53 N0=53 kernelpagesize_kB=2048
7fe8c8400000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048

$ grep -i huge /proc/26020/smaps | grep -v ' 0 kB'
Private_Hugetlb:  126976 kB
Private_Hugetlb:  135168 kB
Private_Hugetlb:  108544 kB
Private_Hugetlb:    2048 kB
2018-03-29 18:55:18 +04:00
Marko Mäkelä
84129fb1b5 After-merge fix for commit 98eb9518db
The merge only covered 10.1 up to
commit 4d248974e0.

Actually merge the changes up to
commit 0a534348c7.

Also, remove the unused InnoDB field trx_t::abort_type.
2018-03-16 15:49:53 +02:00
Sergey Vojtovich
0a534348c7 MDEV-14265 - RPMLint warning: shared-lib-calls-exit
Eliminated last exit() call from libmysqld.
2018-03-16 13:26:52 +04:00
Sergei Golubchik
e92cc09765 MDEV-15345 Compilation fails to build my_addr_resolve.c
fix the compilation error.
no support for plugins yet.
2018-02-22 16:46:11 +01:00
Sergei Golubchik
49bcc82686 Merge branch '10.1' into 10.2 2018-02-11 13:47:16 +01:00
Sergei Golubchik
60dfe12be3 MDEV-14868 MariaDB server crashes after using ROLLBACK TO when encrypt_tmp_files=ON
Fix reinit_io_cache(WRITE_CACHE) with non-zero seek_offset.
Run encryption.tempfiles with and without binlog checksums.
2018-02-08 22:54:37 +01:00
Marko Mäkelä
be6307c0fa Merge 10.1 into 10.2 2018-02-08 13:53:21 +02:00
Marko Mäkelä
871f2a6ee2 Merge 10.0 into 10.1 2018-02-08 13:29:08 +02:00
Oleksandr Byelkin
b6455479e5 MDEV-15230: column_json breaks cyrillic in 10.1.31
Use unsigned comparison.
2018-02-07 18:14:45 +01:00
Sergei Golubchik
4771ae4b22 Merge branch 'github/10.1' into 10.2 2018-02-06 14:50:50 +01:00
Marko Mäkelä
8812a2f858 Merge 10.0 into 10.1 2018-02-03 12:53:30 +02:00
Vicențiu Ciorbaru
0765caa073 Merge remote-tracking branch 'origin/5.5' into 10.0 2018-02-02 18:14:35 +01:00
Sergei Golubchik
29031fe391 my_addr_resolve fixes
* use a longer timeout, 10ms is not always enough
* stop waiting as long as the data is read, do not unconditionally
  wait for timeout every time
2018-02-02 11:42:07 +01:00
Sergei Golubchik
d4df7bc9b1 Merge branch 'github/10.0' into 10.1 2018-02-02 10:09:44 +01:00
Vicențiu Ciorbaru
7a63ffab71 Fix an out of scope bzero 2018-01-30 21:13:24 +02:00
Vladislav Vaintroub
9891ee5a2a Fix and reenable Windows compiler warning C4800 (size_t conversion). 2018-01-26 10:37:46 +00:00
Vicențiu Ciorbaru
d833bb65d5 Merge remote-tracking branch '5.5' into 10.0 2018-01-24 12:29:31 +02:00
Oleksandr Byelkin
a4663af05c MDEV-7533: COLUMN_JSON() doesn't escape control characters in string values
escape all charecters less or equal 0x1F (control symbols)
(shorter sequence are not used to make code simple, long encoding is always legal according to the rfc4627)
2018-01-23 13:06:39 +01:00
Sergei Golubchik
dc28b6d180 improve ASAN instrumentation: MEM_ROOT
more complete TRASH-ing of memroots
2018-01-22 11:39:54 +01:00
Sergei Golubchik
22ae3843db Correct TRASH() macro usage
TRASH was mapped to TRASH_FREE and was supposed to be used for memory
that should not be accessed anymore, while TRASH_ALLOC() is to be
used for uninitialized but to-be-used memory.

But sometimes TRASH() was used in the latter sense.

Remove TRASH() macro, always use explicit TRASH_ALLOC() or TRASH_FREE().
2018-01-22 11:39:54 +01:00
Vicențiu Ciorbaru
26e5f9dda1 MDEV-14229: Stack trace is not resolved for shared objects
Resolving a stacktrace including functions in dynamic libraries requires
us to look inside the libraries for the symbols. Addr2line needs to be
started with the correct binary for each address on the stack. To do this,
figure out which library it is using dladdr, then if the addr2line
binary was started with a different binary, fork it again with the
correct one.

We only have one addr2line process running at any point during the
stacktrace resolving step. The maximum number of forks for addr2line should
generally be around 6.

One for server stacktrace code, one for plugin code, one when going back
into server code, one for pthread library, one for libc, one for the
_start function in the server. More can come up if plugin calls server
function which goes back to a plugin, etc.
2018-01-19 16:11:34 +02:00
Sergei Golubchik
2d52d3c1bf Merge branch 'mysql/5.5' into 5.5 2018-01-18 17:54:48 +01:00
Sergei Golubchik
d8001106c9 MDEV-14469 build with cmake -DMYSQL_MAINTAINER_MODE=ON fails: 'readdir_r' is deprecated
1. test readdir_r() availability under -Werror
2. don't protect readdir() with mutexes, it's not needed for the way
   we use readdir()
2018-01-15 15:57:04 +01:00
Vicențiu Ciorbaru
985d2d393c Merge remote-tracking branch 'origin/10.1' into 10.2 2017-12-22 12:23:39 +02:00
Sergey Vojtovich
2cd3169113 MDEV-14265 - RPMLint warning: shared-lib-calls-exit
find_type_or_exit() client helper did exit(1) on error, exit(1) moved to
clients.

mysql_read_default_options() did exit(1) on error, error is passed through and
handled now.

my_str_malloc_default() did exit(1) on error, replaced my_str_ allocator
functions with normal my_malloc()/my_realloc()/my_free().

sql_connect.cc did many exit(1) on hash initialisation failure. Removed error
check since my_hash_init() never fails.

my_malloc() did exit(1) on error. Replaced with abort().

my_load_defaults() did exit(1) on error, replaced with return 2.

my_load_defaults() still does exit(0) when invoked with --print-defaults.
2017-12-19 19:10:54 +04:00
Marko Mäkelä
ece9c54e10 Merge 10.1 into 10.2 2017-12-14 08:40:01 +02:00
Marko Mäkelä
e9cc486c97 Fix a typo: schedule, scheduling 2017-12-13 20:43:09 +02:00
Marko Mäkelä
ce07d09fd3 Merge 10.0 into 10.1 2017-12-12 19:28:26 +02:00
Karthik Kamath
8bc828b982 BUG#26502135: MYSQLD SEGFAULTS IN
MDL_CONTEXT::TRY_ACQUIRE_LOCK_IMPL

ANALYSIS:
=========
Server sometimes exited when multiple threads tried to
acquire and release metadata locks simultaneously (for
example, necessary to access a table). The same problem
could have occurred when new objects were registered/
deregistered in Performance Schema.

The problem was caused by a bug in LF_HASH - our lock free
hash implementation which is used by metadata locking
subsystem in 5.7 branch. In 5.5 and 5.6 we only use LF_HASH
in Performance Schema Instrumentation implementation. So
for these versions, the problem was limited to P_S.

The problem was in my_lfind() function, which searches for
the specific hash element by going through the elements
list. During this search it loads information about element
checked such as key pointer and hash value into local
variables. Then it confirms that they are not corrupted by
concurrent delete operation (which will set pointer to 0)
by checking if element is still in the list. The latter
check did not take into account that compiler (and
processor) can reorder reads in such a way that load of key
pointer will happen after it, making result of the check
invalid.

FIX:
====
This patch fixes the problem by ensuring that no such
reordering can take place. This is achieved by using
my_atomic_loadptr() which contains compiler and processor
memory barriers for the check mentioned above and other
similar places.

The default (for non-Windows systems) implementation of
my_atomic*() relies on old __sync intrisics and implements
my_atomic_loadptr() as read-modify operation. To avoid
scalability/performance penalty associated with addition of
my_atomic_loadptr()'s we change the my_atomic*() to use
newer __atomic intrisics when available. This new default
implementation doesn't have such a drawback.
2017-11-27 19:59:29 +05:30
Oleksandr Byelkin
2913f615f0 MDEV-8949: COLUMN_CREATE unicode name breakage
Use utf-mb4 if it is possible.
2017-11-14 10:49:46 +01:00
Sergei Golubchik
e0a1c745ec Merge branch '10.1' into 10.2 2017-10-24 14:53:18 +02:00
Sergei Golubchik
9d2e2d7533 Merge branch '10.0' into 10.1 2017-10-22 13:03:41 +02:00
Sergei Golubchik
da4503e956 Merge branch '5.5' into 10.0 2017-10-18 15:14:39 +02:00
Sergei Golubchik
d76f5774fe MDEV-13459 Warnings, when compiling with gcc-7.x
mostly caused by -Wimplicit-fallthrough
2017-10-17 07:37:39 +02:00
Vladislav Vaintroub
7354dc6773 MDEV-13384 - misc Windows warnings fixed 2017-09-28 17:20:46 +00:00
Vladislav Vaintroub
eba44874ca MDEV-13844 : Fix Windows warnings. Fix DBUG_PRINT.
- Fix win64 pointer truncation warnings
(usually coming from misusing 0x%lx and long cast in DBUG)

- Also fix printf-format warnings

Make the above mentioned warnings fatal.

- fix pthread_join on Windows to set return value.
2017-09-28 17:20:46 +00:00
Tor Didriksen
d52edb75bb Bug#23072792 MYSQL_GROUP_SUFFIX DOES NOT WORK
Reintroduce environment variable MYSQL_GROUP_SUFFIX to be used as
--default-group-suffix value if not already set.

The environment variable was accidentally renamed to DEFAULT_GROUP_SUFFIX_ENV
in MySQL server 5.5.
2017-09-08 16:48:15 +02:00
Marko Mäkelä
5660c061fa Merge 10.1 into 10.2 2017-08-31 18:06:11 +03:00
Marko Mäkelä
be45f083e6 Merge 10.0 into 10.1 2017-08-31 18:04:44 +03:00
Vladislav Vaintroub
66eb9774e4 MDEV-13691 : my_write() sets inappropriate errno for ERROR_FILE_SYSTEM_LIMITATON
ERROR_FILE_SYSTEM_LIMITATION was seen by support when backing up large
file. However mariabackup error message was not very helpful,
since it mapped the error to generic catch-all EINVAL.

With the patch, ERROR_FILE_SYSTEM_LIMITATION will be mapped to more
appropriate EFBIG. Also add mapping from ERROR_NO_SYSTEM_RESOURCES
to ENOMEM.
2017-08-31 14:57:31 +00:00