Commit graph

9379 commits

Author SHA1 Message Date
Marko Mäkelä
b2c4740034 Fix some -Wsign-conversion
InnoDB was using int64_t instead of ha_rows (unsigned 64-bit).
2018-04-29 17:53:21 +03:00
Monty
2ccd6716fc Fix a lot of compiler warnings found by -Wunused 2018-04-26 17:35:12 +03:00
Marko Mäkelä
7396dfcca7 Merge 10.2 into 10.3 2018-04-24 20:59:57 +03:00
Marko Mäkelä
39a4985520 Remove most 'register' use in C++
Modern compilers (such as GCC 8) emit warnings that the
'register' keyword is deprecated and not valid C++17.

Let us remove most use of the 'register' keyword.
Code in 'extra/' is not touched.
2018-04-24 12:48:27 +03:00
Vicențiu Ciorbaru
65eefcdc60 Merge remote-tracking branch '10.2' into 10.3 2018-04-12 12:41:19 +03:00
Vicențiu Ciorbaru
45e6d0aebf Merge branch '10.1' into 10.2 2018-04-10 17:43:18 +03:00
Sergei Golubchik
479bd5a6fe improve strmake_buf() to detect wrong usage reliably
strmake_buf() macro should only be used with char[] arrays,
and never with char* pointers. To distinguish between the two
we create a new variable of the same type and initialize it
using array initializer, this causes compilation failure
with pointers. The variable is unused and will be removed by the
compiler. It's enough to do this check only with gcc, so
it doesn't have to be portable.
2018-04-10 13:12:36 +02:00
Daniel Black
1479273cdb MDEV-14705: slow innodb startup/shutdown can exceed systemd timeout
Use systemd EXTEND_TIMEOUT_USEC to advise systemd of progress

Move towards progress measures rather than pure time based measures.

Progress reporting at numberious shutdown/startup locations incuding:
* For innodb_fast_shutdown=0 trx_roll_must_shutdown() for rolling back incomplete transactions.
* For merging the change buffer (in srv_shutdown(bool ibuf_merge))
* For purging history, srv_do_purge

Thanks Marko for feedback and suggestions.
2018-04-06 09:58:14 +03:00
luz.paz
3dd01669b4 Misc. typos
Found via `codespell -i 3 -w --skip="./debian/po" -I ../mariadb-server-word-whitelist.txt  ./cmake/ ./debian/ ./Docs/ ./include/ ./man/ ./plugin/ ./strings/`
2018-04-05 15:26:57 +04:00
Vladislav Vaintroub
19bb7fdcd6 MDEV-15694 Windows : use GetSystemTimePreciseAsFileTime if available for high resolution time
Use high accuracy timer on Windows 8.1+ for system versioning,it needs
accurate high resoution start query time.

Continue to use the inaccurate (but much faster timer function)
GetSystemTimeAsFileTime() where accuracy does not matter, e.g in
set_timespec_time_nsec(),or my_time()
2018-04-01 14:38:45 +00:00
Marko Mäkelä
e895041ba9 Merge 10.2 into 10.3 2018-03-30 18:27:02 +03:00
Daniel Black
402c7584a8 MDEV-13785: move defination HAVE_LARGE_PAGES -> HAVE_LINUX_LARGE_PAGES
HAVE_LARGE_PAGES was always Linux but now there is
HAVE_SOLARIS_LARGE_PAGES in the code base. Innodb was using HAVE_LINUX_LARGE_PAGES
so keep this consistent everywhere.

Test plan:

$ grep Hugepagesize: /proc/meminfo
Hugepagesize:       2048 kB

$ sudo sysctl vm.nr_hugepages=1024
vm.nr_hugepages = 1024
$ sudo sysctl  kernel.shmmax=$(( 2 * 1024 *1024 * 1024 ))
kernel.shmmax = 2147483648

No errors in ouput:
$ sql/mysqld --skip-networking --datadir=/tmp/datadir --log-bin=/tmp/datadir/mysqlbin --socket /tmp/s.sock --lc-messages-dir=${PWD}/sql/share --verbose --large-pages=1
2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld (mysqld 10.2.14-MariaDB-log) starting as process 25406 ...
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Uses event mutexes
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Compressed tables use zlib 1.2.11
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using Linux native AIO
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Number of pools: 1
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using SSE2 crc32 instructions
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Completed initialization of buffer pool
2018-03-23 12:51:18 139696883590912 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Highest supported file format is Barracuda.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 128 out of 128 rollback segments are active.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Creating shared tablespace for temporary tables
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 5.7.21 started; log sequence number 1620099
2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Loading buffer pool(s) from /tmp/datadir/ib_buffer_pool
2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Buffer pool(s) load completed at 180323 12:51:18
2018-03-23 12:51:18 139697428129984 [Note] Plugin 'FEEDBACK' is disabled.
2018-03-23 12:51:18 139697428129984 [Note] Reading of all Master_info entries succeded
2018-03-23 12:51:18 139697428129984 [Note] Added new Master_info '' to hash table
2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld: ready for connections.
Version: '10.2.14-MariaDB-log'  socket: '/tmp/s.sock'  port: 0  Source distribution

$ grep -i huge /proc/25406/smaps | grep -v ' 0 kB'
Private_Hugetlb:    8192 kB
Private_Hugetlb:    2048 kB
$ grep huge /proc/25406/numa_maps
7f0d74400000 default file=/SYSV00000000\040(deleted) huge
7f0dbd200000 default file=/SYSV00000000\040(deleted) huge dirty=4 N0=4 kernelpagesize_kB=2048
7f0dc5600000 default file=/SYSV00000000\040(deleted) huge
7f0dd1200000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048

$ grep Huge  /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     940
HugePages_Free:      935
HugePages_Rsvd:      177
HugePages_Surp:        0
Hugepagesize:       2048 kB

Ran again with --memlock
(note needs ulimit -l > size)

$ grep Huge  /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     940
HugePages_Free:      758
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

$ grep huge /proc/26020/numa_maps
7fe870400000 default file=/SYSV00000000\040(deleted) huge dirty=62 N0=62 kernelpagesize_kB=2048
7fe8b3a00000 default file=/SYSV00000000\040(deleted) huge dirty=66 N0=66 kernelpagesize_kB=2048
7fe8bd600000 default file=/SYSV00000000\040(deleted) huge dirty=53 N0=53 kernelpagesize_kB=2048
7fe8c8400000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048

$ grep -i huge /proc/26020/smaps | grep -v ' 0 kB'
Private_Hugetlb:  126976 kB
Private_Hugetlb:  135168 kB
Private_Hugetlb:  108544 kB
Private_Hugetlb:    2048 kB
2018-03-29 18:55:18 +04:00
Sergei Golubchik
b1818dccf7 Merge branch '10.2' into 10.3 2018-03-28 17:31:57 +02:00
Marko Mäkelä
3d7915f000 Merge 10.1 into 10.2 2018-03-21 22:58:52 +02:00
Marko Mäkelä
865cec928a MDEV-15505 Fix wsrep XID storage byte order 2018-03-21 16:29:30 +02:00
Teemu Ollakka
33aad1d273 MDEV-15505 Fixes to compilation without -DWITH_WSREP:BOOL=ON
Removed including wsrep_api.h from service_wsrep.h. This caused
various kinds of collisions with definitions when wsrep is
not supposed to be built in. Defined functions wsrep_xid_seqno()
and wsrep_xid_uuid() in wsrep_dummy.cc. Replaced wsrep_seqno_t
with long long where wsrep_api.h is not included.

Removed wsrep_xid_seqno() macro from wsrep_mysqld.h and made
wsrep code using wsrep_xid_seqno() in handler.cc to be compiled
in only if WITH_WSREP is ON.

Included wsrep_api.h for mariabackup if WITH_WSREP is ON.
2018-03-21 12:02:09 +02:00
Vicențiu Ciorbaru
82aeb6b596 Merge branch '10.1' into 10.2 2018-03-21 10:36:49 +02:00
Marko Mäkelä
613be24b7a Merge 10.0 into 10.1 2018-03-20 19:25:08 +02:00
Marko Mäkelä
0492100059 Merge 5.5 into 10.0 2018-03-20 18:36:03 +02:00
Eugene Kosov
5a8f8f89d6 honor alignment rules and xtradb too 2018-03-20 10:46:57 +03:00
Vicențiu Ciorbaru
24b353162f Merge branch '10.0-galera' into 10.1 2018-03-19 15:21:01 +02:00
Marko Mäkelä
84129fb1b5 After-merge fix for commit 98eb9518db
The merge only covered 10.1 up to
commit 4d248974e0.

Actually merge the changes up to
commit 0a534348c7.

Also, remove the unused InnoDB field trx_t::abort_type.
2018-03-16 15:49:53 +02:00
Sergey Vojtovich
0a534348c7 MDEV-14265 - RPMLint warning: shared-lib-calls-exit
Eliminated last exit() call from libmysqld.
2018-03-16 13:26:52 +04:00
Daniel Black
8b54c31486 MDEV-8743: where O_CLOEXEC is available, use for innodb buf_dump
As this is the only moderately critical fopened for writing file,
create an alternate path to use open and fdopen for non-glibc platforms
that support O_CLOEXEC (BSDs).

Tested on Linux (by modifing the GLIBC defination) to take this
alternate path:

$ cd /proc/23874
$ more fdinfo/71
pos:    0
flags:  02100001
mnt_id: 24
$ ls -la fd/71
l-wx------. 1 dan dan 64 Mar 14 13:30 fd/71 -> /dev/shm/var_auto_i7rl/mysqld.1/data/ib_buffer_pool.incomplete
2018-03-15 12:07:43 +02:00
Teemu Ollakka
b125ae0a84 MDEV-15505 New wsrep XID format for backwards compatibility
A new wsrep XID format was added to keep the XID implementation
backwards compatible. Original version always reads XID seqno
part in host byte order, the new version in little endian byte
order. Wsrep XID will always be written in the new format.

Included wsrep_api.h from service_wsrep.h for wsrep type definitions.
Removed redundant wsrep XID code from mariabackup and included
service_wsrep.h in order to use
2018-03-12 14:51:49 +02:00
Teemu Ollakka
dd74b94823 MDEV-15505 Fix wsrep XID seqno byte order
The problem is that the seqno part of wsrep XID is always
stored in host byte order. This may cause issues when a physical
backup is restored on a host with different architecture, the
seqno part with XID may have incorrect value.

In order to fix this, wsrep XID seqno is always written into
XID data buffer in little endian byte order using int8store()
and read from data buffer using sint8korr(). For backwards
compatibility the seqno is read from TRX_SYS page in host
byte order during upgrade.

This patch implements byte ordering in wsrep_xid_init(),
wsrep_xid_seqno(), and exposes functions to read wsrep
XID uuid and seqno in wsrep_service_st. Backwards compatibility
for upgrade is provided in trx_rseg_init_wsrep_xid().
2018-03-12 14:46:20 +02:00
Daniel Black
0a63c91ab0 MDEV-8743: fopen mode e (glibc only) to prevent galera SST scripts accessing server files
For RemoteDatafile::read_link_file and buffer poool dumps

Note: STR_O_CLOEXEC needs to be at the end of the fopen otherwise
fopen will return EINVAL.
2018-03-05 12:00:09 +11:00
Sergei Golubchik
7961bc4b89 helper append_interval(String*) 2018-02-23 15:33:24 +01:00
Sergei Golubchik
2732fcc608 Merge branch 'bb-10.2-ext' into 10.3 2018-02-23 08:43:34 +01:00
Sergei Golubchik
b8af22af15 Merge branch '10.2' into bb-10.2-ext 2018-02-22 19:29:52 +01:00
Sergei Golubchik
e4a73acc63 Merge branch '10.1' into 10.2 2018-02-22 16:46:02 +01:00
Sergei Golubchik
a04e4f531a Merge branch '10.0' into 10.1 2018-02-22 14:12:02 +01:00
Sergei Golubchik
b728641e86 Merge branch '5.5' into 10.0 2018-02-22 09:22:03 +01:00
Vladislav Vaintroub
23d7b77358 Fix truncation warning on Windows. 2018-02-21 18:22:43 +00:00
Vladislav Vaintroub
56e7b7eaed Make possible to use clang on Windows (clang-cl)
-DWITH_ASAN can be used as well now, on x64

Fix many clang-cl warnings.
2018-02-20 21:17:36 +00:00
Marko Mäkelä
cc3b5d1fe7 Merge bb-10.2-ext into 10.3 2018-02-15 11:48:30 +02:00
Marko Mäkelä
b006d2ead4 Merge bb-10.2-ext into 10.3 2018-02-15 10:22:03 +02:00
Alexander Barkov
c17a06abf8 MDEV-15310 Range optimizer does not work well for "WHERE temporal_column NOT IN (const_list)"
There were two problems related to the bug report:
1. Item_datetime::get_date() was not implemented.
   So execution went through val_int() followed
   by int-to-datetime or int-to-time conversion.
   This was the reason why the optimizer did not
   work well on data with fractional seconds.
2. Item_datetime::set() did not have a TIME specific code
   to mix months and days to hours after unpack_time().
   This is why the optimizer did not work well with negative
   TIME values, as well as huge time values.

Changes:

1. Overriding Item_datetime::get_date(), to return ltime.
   This fixes the problem N1.
2. Cleanup: Moving pack_time() and unpack_time() from
   sql-common/my_time.c and include/my_time.h to
   sql/sql_time.cc and sql/sql_time.h, as they are not needed
   on the client side.
3. Adding a new "enum_mysql_timestamp_type ts_type" parameter
   to unpack_time() and moving the TIME specific code to mix
   months and days with hours inside unpack_time().
   Adding a new "ts_type" parameter to Item_datetime::set(),
   to pass it from the caller down to unpack_time().
   So now the TIME specific code is automatically called
   from Item_datetime::set(). This fixes the problem N2.
   This change also helped to get rid of duplicate TIME specific code
   from other three places, where mixing month/days to hours
   was done immediately after unpack_time().
   Moving the DATE specific code to zero hhmmssff
   from Item_func_min_max::get_date_native to inside unpack_time(),
   for symmetry.
4. Removing the virtual method in_vector::result_type(),
   adding in_vector::type_handler() instead.
   This helps to get result_type(), field_type(),
   mysql_timestamp_type() of an in_vector easier.
   Passing type_handler()->mysql_timestamp_type() as
   a new parameter to Item_datetime::set() inside
   in_temporal::value_to_item().
5. Cleaup: Removing separate implementations of in_datetime::get_value()
   and in_time::get_value(). Adding a single implementation
   in_temporal::get_value() instead.
   Passing type_handler()->field_type() to get_value_internal().
2018-02-14 22:58:34 +04:00
Sergei Golubchik
7c6cf7fefe bug: ha_heap was unilaterally increasing reclength
proper fix replacing the hack from b80fa4000d

don't confuse length of the data area (reclength) with the
offset to the "deleted" mark.
2018-02-11 18:36:54 +01:00
Alexander Barkov
3cad31f2a7 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2018-02-08 19:06:25 +04:00
Sergei Golubchik
775a8a0e4b compilation error on windows
get_rec_bits() must return uchar, it's result is often
assigned to uchar. And len is always less than 8.
2018-02-06 14:51:44 +01:00
Sergei Golubchik
4771ae4b22 Merge branch 'github/10.1' into 10.2 2018-02-06 14:50:50 +01:00
Vladislav Vaintroub
6c279ad6a7 MDEV-15091 : Windows, 64bit: reenable and fix warning C4267 (conversion from 'size_t' to 'type', possible loss of data)
Handle string length as size_t, consistently (almost always:))
Change function prototypes to accept size_t, where in the past
ulong or uint were used. change local/member variables to size_t
when appropriate.

This fix excludes rocksdb, spider,spider, sphinx and connect for now.
2018-02-06 12:55:58 +00:00
Alexander Barkov
217fc122c8 Merge remote-tracking branch 'origin/bb-10.2-ext' into 10.3 2018-02-04 18:40:06 +04:00
Sergei Golubchik
d4df7bc9b1 Merge branch 'github/10.0' into 10.1 2018-02-02 10:09:44 +01:00
Monty
d69642dedd Added name to MEM_ROOT for esier debugging
This will make it easier to how memory allocation is done when debugging
with either DBUG or gdb.

Will especially help when debugging stored procedures

Main change is a name argument as second argument to init_alloc_root()
init_sql_alloc()

Other things:
- Added DBUG_ENTER/EXIT to some Virtual_tmp_table functions
2018-02-02 11:08:36 +02:00
Jan Lindström
c7e5feb259 Merge tag 'mariadb-10.0.34' into 10.0-galera
Conflicts:
	storage/innobase/lock/lock0lock.cc
	storage/xtradb/lock/lock0lock.cc
	storage/xtradb/lock/lock0wait.cc
	support-files/mysql.server.sh
2018-02-01 14:09:48 +02:00
Monty
a7e352b54d Changed database, tablename and alias to be LEX_CSTRING
This was done in, among other things:
- thd->db and thd->db_length
- TABLE_LIST tablename, db, alias and schema_name
- Audit plugin database name
- lex->db
- All db and table names in Alter_table_ctx
- st_select_lex db

Other things:
- Changed a lot of functions to take const LEX_CSTRING* as argument
  for db, table_name and alias. See init_one_table() as an example.
- Changed some function arguments from LEX_CSTRING to const LEX_CSTRING
- Changed some lists from LEX_STRING to LEX_CSTRING
- threads_mysql.result changed because process list_db wasn't always
  correctly updated
- New append_identifier() function that takes LEX_CSTRING* as arguments
- Added new element tmp_buff to Alter_table_ctx to separate temp name
  handling from temporary space
- Ensure we store the length after my_casedn_str() of table/db names
- Removed not used version of rename_table_in_stat_tables()
- Changed Natural_join_column::table_name and db_name() to never return
  NULL (used for print)
- thd->get_db() now returns db as a printable string (thd->db.str or "")
2018-01-30 21:33:55 +02:00
Marko Mäkelä
921c5e9314 Merge bb-10.2-ext into 10.3
MDEV-11415 Remove excessive undo logging during ALTER TABLE…ALGORITHM=COPY

Move a test from innodb.rename_table_debug to innodb.alter_copy.

ha_innobase::extra(HA_EXTRA_BEGIN_ALTER_COPY): Register id-versioned
tables so that mysql.transaction_registry will be updated, even for
empty tables that are subjected to ALTER TABLE…ALGORITHM=COPY.
2018-01-30 21:26:53 +02:00
Marko Mäkelä
0c1f220611 Merge 10.2 into bb-10.2-ext 2018-01-30 20:47:12 +02:00