Commit graph

9295 commits

Author SHA1 Message Date
Oleksandr Byelkin
28f08d3753 Merge branch '10.1' into 10.2 2018-09-14 08:47:22 +02:00
Alexey Botchkov
c8bb43a938 MDEV-17121 JSON_ARRAY_APPEND.
Extra comma added to the result when an json array is empty.
2018-09-11 14:37:45 +04:00
Sergei Golubchik
db947b7599 Merge branch '10.0-galera' into 10.1 2018-09-07 15:25:27 +02:00
Sergei Golubchik
a6246cab16 fix failures of innodb_plugin tests in --embedded
Post-fix for 7e8ed15b95

Also, apply the same innodb fix to xtradb.
2018-09-04 09:19:50 +02:00
Sergei Golubchik
aec54fb938 MDEV-9627 clang builds fail on i386
don't use C11 atomics unless HAVE_GCC_C11_ATOMICS is defined
2018-09-04 09:19:48 +02:00
Jan Lindström
c5a8583b31 Merge tag 'mariadb-10.0.36' into 10.0-galera 2018-08-02 11:44:02 +03:00
Marko Mäkelä
ef3070e997 Merge 10.1 into 10.2 2018-08-02 08:19:57 +03:00
Oleksandr Byelkin
865e807125 Merge branch '10.0' into 10.1 2018-07-31 11:58:29 +02:00
Marko Mäkelä
91181b225c Merge 5.5 into 10.0 2018-07-30 15:09:25 +03:00
Oleksandr Byelkin
fceda2dab6 Merge remote-tracking branch 'mysql/5.5' into 5.5
We do not accept:
1. We did not have this problem (fixed earlier and better)
 d982e717ab Bug#27510150: MYSQLDUMP FAILS FOR SPECIFIC --WHERE CLAUSES
2. We do not have such options (an DBUG_ASSERT put just in case)
 bbc2e37fe4 Bug#27759871: BACKRONYM ISSUE IS STILL IN MYSQL 5.7
3. Serg fixed it in other way in this release:
 e48d775c6f Bug#27980823: HEAP OVERFLOW VULNERABILITIES IN MYSQL CLIENT LIBRARY
2018-07-29 13:10:29 +02:00
Oleksandr Byelkin
cb5952b506 Merge branch '10.0' into bb-10.1-merge-sanja 2018-07-25 22:24:40 +02:00
Alexander Barkov
e2ac4098ed Simplify caseup() and casedn() in charsets
After the MDEV-13118 fix there's no code in the server that
wants caseup/casedn to change the argument in place for simple
charsets.  Let's remove this logic and always return the result in a
new string for all charsets, both simple and complex.

1. Removing the optimization that *some* character sets used in casedn()
  and caseup(), which allowed (and required) to change the case in-place,
  overwriting the string passed as the "src" argument.
  Now all CHARSET_INFO's work in the same way:
  non of them change the source string in-place, all of them now convert
  case from the source string to the destination string, leaving
  the source string untouched.

2. Adding "const" qualifier to the "char *src" parameter
   to caseup() and casedn().

3. Removing duplicate implementations in ctype-mb.c.
  Now both caseup() and casedn() implementations for all CJK character sets
  use internally the same function my_casefold_mb()
  (the former my_casefold_mb_varlen()).

4. Removing the "unused" attribute from parameters of some my_case{up|dn}_xxx()
   implementations, as the affected parameters are now *used* in the code.
   Previously these parameters were used only in DBUG_ASSERT().
2018-07-19 13:02:14 +04:00
Sergei Golubchik
b942aa34c1 Merge branch '10.1' into 10.2 2018-06-21 23:47:39 +02:00
Georg Richter
b4db59ba47 MDEV-15596 10.2 doesn't work with openssl 1.1.1 2018-06-21 10:15:27 +02:00
Sergei Golubchik
bb24663f5a MDEV-13577 slave_parallel_mode=optimistic should not report the mode's specific temporary errors
Revert 7bbe324fc1
Fix the bug differently (in log_event.cc)
Fix the test case to actually fail without the bug fix
2018-06-20 11:10:27 +02:00
Monty
831df10981 Add PART_INDIRECT_KEY_FLAG
This is to mark that a field is indirectly part of a key, which simplifes
checking if we need to have this field up to date to evaluate a key.

For example:
CREATE TABLE t1 (a int, b int as (a) virtual,
                 c int as (b) virtual, index(c))
would mark a and b with PART_INDIRECT_KEY_FLAG.
c is marked with PART_KEY_FLAG as before.
2018-06-19 16:23:34 +03:00
Vicențiu Ciorbaru
6e55236c0a Merge branch '10.0-galera' into 10.1 2018-06-12 19:39:37 +03:00
Andrei Elkin
7bbe324fc1 MDEV-13577 slave_parallel_mode=optimistic should not report the mode's
specific temporary errors

The optimistic parallel slave's worker thread could face a run-time error due to
the algorithm's specifics which allows for conflicts like the reported
"Can't find record in 'table'".
A typical stack is like

{noformat}
#0  handler::print_error (this=0x61c00008f8a0, error=149, errflag=0) at handler.cc:3650
#1  0x0000555555e95361 in write_record (thd=thd@entry=0x62a0000a2208, table=table@entry=0x61f00008ce88, info=info@entry=0x7fffdee356d0) at sql_insert.cc:1944
#2  0x0000555555ea7767 in mysql_insert (thd=thd@entry=0x62a0000a2208, table_list=0x61b00012ada0, fields=..., values_list=..., update_fields=..., update_values=..., duplic=<optimized out>, ignore=<optimized out>) at sql_insert.cc:1039
#3  0x0000555555efda90 in mysql_execute_command (thd=thd@entry=0x62a0000a2208) at sql_parse.cc:3927
#4  0x0000555555f0cc50 in mysql_parse (thd=0x62a0000a2208, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at sql_parse.cc:7449
#5  0x00005555566d4444 in Query_log_event::do_apply_event (this=0x61200005b9c8, rgi=<optimized out>, query_arg=<optimized out>, q_len_arg=<optimized out>) at log_event.cc:4508
#6  0x00005555566d639e in Query_log_event::do_apply_event (this=<optimized out>, rgi=<optimized out>) at log_event.cc:4185
#7  0x0000555555d738cf in Log_event::apply_event (rgi=0x61d0001ea080, this=0x61200005b9c8) at log_event.h:1343
#8  apply_event_and_update_pos_apply (ev=ev@entry=0x61200005b9c8, thd=thd@entry=0x62a0000a2208, rgi=rgi@entry=0x61d0001ea080, reason=<optimized out>) at slave.cc:3479
#9  0x0000555555d8596b in apply_event_and_update_pos_for_parallel (ev=ev@entry=0x61200005b9c8, thd=thd@entry=0x62a0000a2208, rgi=rgi@entry=0x61d0001ea080) at slave.cc:3623
#10 0x00005555562aca83 in rpt_handle_event (qev=qev@entry=0x6190000fa088, rpt=rpt@entry=0x62200002bd68) at rpl_parallel.cc:50
#11 0x00005555562bd04e in handle_rpl_parallel_thread (arg=arg@entry=0x62200002bd68) at rpl_parallel.cc:1258
{noformat}

Here {{handler::print_error}} computes whether to error log the
current error when --log-warnings > 1. The decision flag is consulted
bu {{my_message_sql()}} which can be eventually called.
In the bug case the decision is to log.
However in the optimistic mode slave applier case any conflict is
attempted to resolve with rollback and retry to success. Hence the
logging is at least extraneous.

The case is fixed with adding a new flag {{ME_LOG_AS_WARN}} which
{{handler::print_error}} may propagate further on through {{my_error}}
when the error comes from an optimistically running slave worker thread.

The new flag effectively requests the warning level for the errlog record,
while the thread's DA records the actual error (which is regarded as temporary one
by the parallel slave error handler).
2018-06-12 15:29:16 +03:00
Arun Kuruvila
bbc2e37fe4 Bug#27759871: BACKRONYM ISSUE IS STILL IN MYSQL 5.7
Description:- Client applications establishes connection to
server, which does not support SSL, via TCP even when SSL is
enforced via MYSQL_OPT_SSL_MODE or MYSQL_OPT_SSL_ENFORCE or
MYSQL_OPT_SSL_VERIFY_SERVER_CERT.

Analysis:- There exist no error handling for catching client
applications which enforces SSL connection to connect to a
server which does not support SSL.

Fix:- Error handling is done to catch above mentioned
scenarios.
2018-05-14 11:28:13 +05:30
Vladislav Vaintroub
8ba0eea65c Fix warning VS2017 15.7 update.
This previously unreported warning comes from casting size_t to ulong
in sql_hset.h in Hash_Set::at().

Change my_hash_element to accept size_t index parameter.
2018-05-09 23:04:18 +01:00
Jan Lindström
2bbc868c50
Merge pull request #710 from grooverdan/10.0-galera-MDEV-8743-mysqld-socket-o_cloexec
MDEV-8743: mysqld port/socket - FD_CLOEXEC if no SOCK_CLOEXEC
2018-05-09 10:05:14 +03:00
Jan Lindström
648cf7176c Merge remote-tracking branch 'origin/5.5-galera' into 10.0-galera 2018-05-07 13:49:14 +03:00
Jan Lindström
a5001a2ad7 Merge tag 'mariadb-5.5.60' into 5.5-galera 2018-04-24 13:34:57 +03:00
Marko Mäkelä
39a4985520 Remove most 'register' use in C++
Modern compilers (such as GCC 8) emit warnings that the
'register' keyword is deprecated and not valid C++17.

Let us remove most use of the 'register' keyword.
Code in 'extra/' is not touched.
2018-04-24 12:48:27 +03:00
Marko Mäkelä
9c34a4124d Merge 10.0 into 10.1 2018-04-24 09:26:40 +03:00
Sergei Golubchik
587568b72a Merge branch '5.5' into 10.0 2018-04-20 14:33:24 +02:00
Sergei Golubchik
1a019d0801 Merge branch 'mysql/5.5' into 5.5 2018-04-19 22:31:26 +02:00
Daniel Black
ccd566af20 MDEV-8743: mysqld port/socket - FD_CLOEXEC if no SOCK_CLOEXEC
In MDEV-8743, the port/socket of mysqld was changed to set FD_CLOEXEC.
The existing mysql_socket_socket function already set that with
SOCK_CLOEXEC when the socket was created. So here we move the fcntl
functionality to the mysql_socket_socket as port/socket are the only
callers.

Preprocessor checks of SOCK_CLOEXEC cannot be done as its a 0 if not
there and SOCK_CLOEXEC (being the value of the enum in bits/socket_type.h)
Preprocesssor logic for arithmetic and non-arithmetic defines are
hard/nonportable/ugly to read. As such we just check in my_global.h
and define HAVE_SOCK_CLOEXEC if we have it.

There was a disparity in behaviour between defined(WITH_WSREP) and
not depending on the OS, so the WITH_WSREP condition was removed
from setting calling fcntl.

All sockets are now maked SOCK_CLOEXEC/FD_CLOEXEC.

strace of mysqld with SOCK_CLOEXEC:

socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_TCP) = 10
write(2, "180419 14:52:40 [Note] Server socket created on IP: '127.0.0.1'.\n", 65180419 14:52:40 [Note] Server socket created on IP: '127.0.0.1'.
) = 65
setsockopt(10, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
bind(10, {sa_family=AF_INET, sin_port=htons(16020), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
listen(10, 150)                         = 0
socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_TCP) = 11
write(2, "180419 14:52:40 [Note] Server socket created on IP: '127.0.0.1'.\n", 65180419 14:52:40 [Note] Server socket created on IP: '127.0.0.1'.
) = 65
setsockopt(11, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
bind(11, {sa_family=AF_INET, sin_port=htons(16021), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
listen(11, 150)                         = 0
socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0) = 12
unlink("/home/dan/repos/build-mariadb-server-10.0/mysql-test/var/tmp/mysqld.1.sock") = -1 ENOENT (No such file or directory)
setsockopt(12, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
umask(000)                              = 006
bind(12, {sa_family=AF_UNIX, sun_path="/home/dan/repos/build-mariadb-server-10.0/mysql-test/var/tmp/mysqld.1.sock"}, 110) = 0
umask(006)                              = 000
listen(12, 150)                         = 0
2018-04-19 14:34:46 +10:00
Vicențiu Ciorbaru
45e6d0aebf Merge branch '10.1' into 10.2 2018-04-10 17:43:18 +03:00
Daniel Black
1479273cdb MDEV-14705: slow innodb startup/shutdown can exceed systemd timeout
Use systemd EXTEND_TIMEOUT_USEC to advise systemd of progress

Move towards progress measures rather than pure time based measures.

Progress reporting at numberious shutdown/startup locations incuding:
* For innodb_fast_shutdown=0 trx_roll_must_shutdown() for rolling back incomplete transactions.
* For merging the change buffer (in srv_shutdown(bool ibuf_merge))
* For purging history, srv_do_purge

Thanks Marko for feedback and suggestions.
2018-04-06 09:58:14 +03:00
Daniel Black
402c7584a8 MDEV-13785: move defination HAVE_LARGE_PAGES -> HAVE_LINUX_LARGE_PAGES
HAVE_LARGE_PAGES was always Linux but now there is
HAVE_SOLARIS_LARGE_PAGES in the code base. Innodb was using HAVE_LINUX_LARGE_PAGES
so keep this consistent everywhere.

Test plan:

$ grep Hugepagesize: /proc/meminfo
Hugepagesize:       2048 kB

$ sudo sysctl vm.nr_hugepages=1024
vm.nr_hugepages = 1024
$ sudo sysctl  kernel.shmmax=$(( 2 * 1024 *1024 * 1024 ))
kernel.shmmax = 2147483648

No errors in ouput:
$ sql/mysqld --skip-networking --datadir=/tmp/datadir --log-bin=/tmp/datadir/mysqlbin --socket /tmp/s.sock --lc-messages-dir=${PWD}/sql/share --verbose --large-pages=1
2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld (mysqld 10.2.14-MariaDB-log) starting as process 25406 ...
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Uses event mutexes
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Compressed tables use zlib 1.2.11
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using Linux native AIO
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Number of pools: 1
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using SSE2 crc32 instructions
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Completed initialization of buffer pool
2018-03-23 12:51:18 139696883590912 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Highest supported file format is Barracuda.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 128 out of 128 rollback segments are active.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Creating shared tablespace for temporary tables
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 5.7.21 started; log sequence number 1620099
2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Loading buffer pool(s) from /tmp/datadir/ib_buffer_pool
2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Buffer pool(s) load completed at 180323 12:51:18
2018-03-23 12:51:18 139697428129984 [Note] Plugin 'FEEDBACK' is disabled.
2018-03-23 12:51:18 139697428129984 [Note] Reading of all Master_info entries succeded
2018-03-23 12:51:18 139697428129984 [Note] Added new Master_info '' to hash table
2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld: ready for connections.
Version: '10.2.14-MariaDB-log'  socket: '/tmp/s.sock'  port: 0  Source distribution

$ grep -i huge /proc/25406/smaps | grep -v ' 0 kB'
Private_Hugetlb:    8192 kB
Private_Hugetlb:    2048 kB
$ grep huge /proc/25406/numa_maps
7f0d74400000 default file=/SYSV00000000\040(deleted) huge
7f0dbd200000 default file=/SYSV00000000\040(deleted) huge dirty=4 N0=4 kernelpagesize_kB=2048
7f0dc5600000 default file=/SYSV00000000\040(deleted) huge
7f0dd1200000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048

$ grep Huge  /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     940
HugePages_Free:      935
HugePages_Rsvd:      177
HugePages_Surp:        0
Hugepagesize:       2048 kB

Ran again with --memlock
(note needs ulimit -l > size)

$ grep Huge  /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     940
HugePages_Free:      758
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

$ grep huge /proc/26020/numa_maps
7fe870400000 default file=/SYSV00000000\040(deleted) huge dirty=62 N0=62 kernelpagesize_kB=2048
7fe8b3a00000 default file=/SYSV00000000\040(deleted) huge dirty=66 N0=66 kernelpagesize_kB=2048
7fe8bd600000 default file=/SYSV00000000\040(deleted) huge dirty=53 N0=53 kernelpagesize_kB=2048
7fe8c8400000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048

$ grep -i huge /proc/26020/smaps | grep -v ' 0 kB'
Private_Hugetlb:  126976 kB
Private_Hugetlb:  135168 kB
Private_Hugetlb:  108544 kB
Private_Hugetlb:    2048 kB
2018-03-29 18:55:18 +04:00
Marko Mäkelä
3d7915f000 Merge 10.1 into 10.2 2018-03-21 22:58:52 +02:00
Vicențiu Ciorbaru
82aeb6b596 Merge branch '10.1' into 10.2 2018-03-21 10:36:49 +02:00
Marko Mäkelä
613be24b7a Merge 10.0 into 10.1 2018-03-20 19:25:08 +02:00
Marko Mäkelä
0492100059 Merge 5.5 into 10.0 2018-03-20 18:36:03 +02:00
Eugene Kosov
5a8f8f89d6 honor alignment rules and xtradb too 2018-03-20 10:46:57 +03:00
Vicențiu Ciorbaru
24b353162f Merge branch '10.0-galera' into 10.1 2018-03-19 15:21:01 +02:00
Marko Mäkelä
84129fb1b5 After-merge fix for commit 98eb9518db
The merge only covered 10.1 up to
commit 4d248974e0.

Actually merge the changes up to
commit 0a534348c7.

Also, remove the unused InnoDB field trx_t::abort_type.
2018-03-16 15:49:53 +02:00
Sergey Vojtovich
0a534348c7 MDEV-14265 - RPMLint warning: shared-lib-calls-exit
Eliminated last exit() call from libmysqld.
2018-03-16 13:26:52 +04:00
Daniel Black
8b54c31486 MDEV-8743: where O_CLOEXEC is available, use for innodb buf_dump
As this is the only moderately critical fopened for writing file,
create an alternate path to use open and fdopen for non-glibc platforms
that support O_CLOEXEC (BSDs).

Tested on Linux (by modifing the GLIBC defination) to take this
alternate path:

$ cd /proc/23874
$ more fdinfo/71
pos:    0
flags:  02100001
mnt_id: 24
$ ls -la fd/71
l-wx------. 1 dan dan 64 Mar 14 13:30 fd/71 -> /dev/shm/var_auto_i7rl/mysqld.1/data/ib_buffer_pool.incomplete
2018-03-15 12:07:43 +02:00
Nirbhay Choubey
c54c490c59 MDEV-8743: O_CLOEXEC/SOCK_CLOEXEC defines for non-unix compatibility
From 0f4478105f
2018-03-02 11:17:39 +11:00
Sergei Golubchik
e4a73acc63 Merge branch '10.1' into 10.2 2018-02-22 16:46:02 +01:00
Sergei Golubchik
a04e4f531a Merge branch '10.0' into 10.1 2018-02-22 14:12:02 +01:00
Sergei Golubchik
b728641e86 Merge branch '5.5' into 10.0 2018-02-22 09:22:03 +01:00
Arun Kuruvila
e4784703ee Bug#25471090: MYSQL USE AFTER FREE
Description:- Mysql client crashes when trying to connect
to a fake server which is sending incorrect packets.

Analysis:- Mysql client crashes when it tries to read
server version details.

Fix:- A check is added in "red_one_row()".
2018-02-12 15:19:43 +05:30
Sergei Golubchik
7c6cf7fefe bug: ha_heap was unilaterally increasing reclength
proper fix replacing the hack from b80fa4000d

don't confuse length of the data area (reclength) with the
offset to the "deleted" mark.
2018-02-11 18:36:54 +01:00
Sergei Golubchik
775a8a0e4b compilation error on windows
get_rec_bits() must return uchar, it's result is often
assigned to uchar. And len is always less than 8.
2018-02-06 14:51:44 +01:00
Sergei Golubchik
4771ae4b22 Merge branch 'github/10.1' into 10.2 2018-02-06 14:50:50 +01:00
Sergei Golubchik
d4df7bc9b1 Merge branch 'github/10.0' into 10.1 2018-02-02 10:09:44 +01:00
Jan Lindström
c7e5feb259 Merge tag 'mariadb-10.0.34' into 10.0-galera
Conflicts:
	storage/innobase/lock/lock0lock.cc
	storage/xtradb/lock/lock0lock.cc
	storage/xtradb/lock/lock0wait.cc
	support-files/mysql.server.sh
2018-02-01 14:09:48 +02:00