Commit graph

188920 commits

Author SHA1 Message Date
Alexander Barkov
535c284aed MDEV-21966 Bind REPLICATION SLAVE ADMIN to a number of global system variables 2020-03-18 07:11:18 +04:00
Monty
dbde95d912 Updated aria_pack to support transactional tables
Added options:
--datadir
--ignore-control-file
--require-control-file

- Improved error messages if open fails
- If control file can't be found/opened, assume that all rows in the
  tables are commited.
2020-03-17 19:13:17 +02:00
Alexander Barkov
90b7ac28a9 MDEV-21963 Bind BINLOG ADMIN to a number of global system variables 2020-03-17 19:08:28 +04:00
Marko Mäkelä
ab0034a789 MDEV-20370 Crash after OPTIMIZE TABLE on TEMPORARY TABLE
Temporary tables are typically short-lived, and temporary tables
are assumed to be accessed only by the thread that is handling
the owning connection. Hence, they must not be subject to
defragmenting.

ha_innobase::optimize(): Do not add temporary tables to
the defragment_table() queue.
2020-03-17 16:28:16 +02:00
Thirunarayanan Balathandayuthapani
e61b99073f MDEV-14808 innodb_fts.sync fails in buildbot with wrong result
AUTO_INCREMENT values are nondeterministic after crash recovery.
While MDEV-6076 guarantees that the AUTO_INCREMENT values of committed
transactions will not roll back, it is possible that the AUTO_INCREMENT
values will be durably incremented for incomplete transactions. So
changing the test case to avoid showing the result of AUTO_INCREMENT value.
2020-03-17 19:40:13 +05:30
Monty
517f659e6d Fixed that caused failure in --ps binlog_encryption.rpl_gtid_basic
Problem was that replace_dynstr_append_mem() assumed strings are
null terminated which is not always the case.
2020-03-17 12:37:56 +02:00
Alexander Barkov
dec14dcffe MDEV-21961 Bind CONNECTION ADMIN to a number of global system variables 2020-03-17 14:27:52 +04:00
Alexey Botchkov
e64a3df4dc MDEV-21959 GIS error message doesn't show the wrong value, just the type.
Error message now shows the whole value.
2020-03-17 13:11:46 +04:00
Alexander Barkov
513cfd046d MDEV-21960 Bind READ_ONLY ADMIN to @@read_only 2020-03-17 12:44:06 +04:00
Marko Mäkelä
89698cef72 MDEV-10570: Fix -Wmaybe-uninitialized
Rows_log_event::change_to_flashback_event(): Reduce the scope
of the variable swap_buff2, and do not duplicate conditions.

GCC 9.3.0 flagged the -Wmaybe-uninitialized when compiling the
10.5 branch using cmake -DWITH_ASAN=ON -DCMAKE_CXX_FLAGS=-O2
2020-03-17 10:35:56 +02:00
Alexander Barkov
b602584183 MDEV-21957 Bind BINLOG ADMIN to @@binlog_format, @@binlog_direct_.., @@sql_log_bin 2020-03-17 11:09:19 +04:00
Marko Mäkelä
c7ba92372b Merge 10.4 into 10.5 2020-03-17 07:58:41 +02:00
Marko Mäkelä
023d986732 perfschema: remove unused variables 2020-03-17 07:56:40 +02:00
Marko Mäkelä
7b5aaaa554 Fix the build on big-endian systems 2020-03-17 07:51:53 +02:00
Alexander Barkov
bd6afd8b5e MDEV-21956 Add class Sys_var_charptr_fscs
Simplifying definitions on Sys_var_charptr*:

- Removing sys_var::is_os_charset
- Adding a new class Sys_var_charptr_fscs, to handle system
  variables with character_set_filesystem.
2020-03-17 07:44:23 +04:00
Igor Babaev
a2d24def8c MDEV-21932 A fast plan with ROR index-merge is ignored when
'index_merge_sort_union=off'

When index_merge_sort_union is set to 'off' and  index_merge_union is set
to 'on' then any evaluated index merge scan must consist only of ROR scans.
The cheapest out of such index merges must be chosen. This index merge
might not be the cheapest index merge.
2020-03-17 02:16:49 +02:00
Monty
1242eb3d32 Removed double records_in_range calls from multi_range_read_info_const
This was to remove a performance regression between 10.3 and 10.4
In 10.5 we will have a better implementation of records_in_range
that will enable us to get more statistics.
This change was not done in 10.4 because the 10.5 will be part of
a larger change that is not suitable for the GA 10.4 version

Other things:
- Changed default handler block_size to 8192 to fix things statistics
  for engines that doesn't set the block size.
- Fixed a bug in spider when using multiple part const ranges
  (Patch from Kentoku)
2020-03-17 02:16:48 +02:00
Daniel Black
c6db115ce6 Fix compile on all big endian related to innodb:ut_crc32_swap_byteorder
Move function ut_crc32_swap_byteorder to a non-x86 #ifdef area.

As its only used in BIGENDIAN, use #ifdefs around
ut_crc32_swap_byteorder.

Travis CI and Debian both include s390x in builds/test, which is big endian.

Fixes commit: 1312b4ebb6
2020-03-17 11:07:38 +11:00
Vladislav Vaintroub
96b472c0ae MDEV-18439 Windows builds should have core_file=1 by default
It did not work, eventhough the default for Windows was changed to 1
for the corresponding system variable.
This be because  test_flags was zeroed mysql_init_variables.

The patch fixes this glitch.
2020-03-17 00:56:49 +01:00
Marko Mäkelä
9cc7edb1cf Cleanup: Remove an unused variable
log_write_buf(): Remove the unused variable write_header.
The dependent code was removed in
commit 0c2365c4e3.
That was dead or unnecessary code at least ever since
commit 9ef2d29ff4
removed the support for innodb_log_files_in_group>1.
2020-03-16 21:57:25 +02:00
Petr Vaněk
b1d7ba472e innodb: fix typo in function description
Function os_file_get_last_error_low returns error number +
OS_FILE_ERROR_MAX (which is currently set to 200) for unknown numbers to
this program.
2020-03-16 17:36:16 +01:00
Monty
2e0277373d Fixed multi_update_debug.test
This should fix the following failure:

+ Warnings:
+ Warning       1639    debug sync point wait timed out
2020-03-16 17:49:44 +02:00
Marko Mäkelä
097e2f9d0a MDEV-16188: Fix clang 10 -Wimplicit-int-float-conversion
mi_records_in_range(): Because HA_POS_ERROR cannot be accurately
represented in double (it will be off by one), add an explicit
cast to silence the warning.
2020-03-16 16:51:35 +02:00
Marko Mäkelä
b7f0644710 MDEV-5313: Fix GCC 10 -Wenum-conversion 2020-03-16 16:32:11 +02:00
Eugene Kosov
7dafab7569 MDEV-21949 key rotation for innodb_encrypt_log is not working in 10.5
log_t::has_encryption_key_rotation(): checks whether
key rotation is supported.

In a subsequent redo log format version, this key rotation
may be broken again.
2020-03-16 17:27:51 +03:00
Eugene Kosov
0c2365c4e3 cleanup redo log
Write log header just ones when file is created, instead of
writing to it on every log file wrap around.

log_t::file::write_header_durable(): this one writes to log header

log_write_buf(): this one stops writing to log header
2020-03-16 17:27:51 +03:00
Eugene Kosov
ce496d4f9e cleanup redo log
move statistics modification into one place
2020-03-16 17:27:51 +03:00
Marko Mäkelä
e5e95a287e Merge 10.3 into 10.4 2020-03-16 16:24:36 +02:00
Vladislav Vaintroub
56402e84b5 MDEV-21824 Crash in convert_error_message
restore check for client charset_info == NULL, which was previously
removed in MDEV-8844
2020-03-16 11:43:30 +01:00
Vladislav Vaintroub
92d61c2229 fix typo on non-Linux/Windows 2020-03-16 11:19:41 +01:00
Sergei Golubchik
b66745e9b9 fix a copy-paste error in 8fd654ce0e
that by a (un)lucky coincidence did not prevent 8fd654ce0e
from working properly
2020-03-16 10:47:39 +01:00
Sergei Golubchik
b55d960e43 fix a nondeterminism in perfschema.statement_program_nesting_event_check test
when selecting from perfschema, filter out statements
used by the test istself in wait_condition.inc, because they,
by design, can be repeated unpredictable number of times.
2020-03-16 10:33:38 +01:00
Marko Mäkelä
c7daabdb05 MDEV-13134/MDEV-21855: Add a test case 2020-03-16 10:10:33 +02:00
Marko Mäkelä
17080cbcf0 MDEV-21945 Assertion w==OPT failed in trx_purge_add_undo_to_history()
By default, when redo log is being written for modifying a persistent
data page, the data page must actually be changed. If the write can
sometimes be optimized away, then the template parameter w=mtr_t::OPT
should be passed in order to silence the debug assertion failure.

InnoDB undo log pages can be reused without properly freeing or
initializing them in between. In particular, the undo log header
page field TRX_UNDO_TRX_NO could have been part of an undo log
record page, and those bytes could accidentally have the desired
value when the page is reused as an undo log header page of
another transaction.

Because the function trx_undo_set_state_at_finish() always changes
the TRX_UNDO_STATE of the page, and because recovery is only reading
TRX_UNDO_TRX_NO for pages that either have the correct TRX_UNDO_STATE
or, in trx_rseg_array_init(), are attached to the TRX_SYS page, the
garbage values in TRX_UNDO_TRX_NO do not seem to cause a problem.

This assertion failure affects debug builds only.
2020-03-16 08:58:54 +02:00
Sergei Golubchik
9bd583ce1f MDEV-21942 Building 10.5 requires Internet access
Force correct PCRE2 linking in the official binaries.
System pcre2 in RPM/DEB, bundled in bintars.
2020-03-16 01:13:01 +01:00
Sergei Golubchik
b6b6980686 MDEV-21942 Building 10.5 requires Internet access
fix build requirements for debian
2020-03-16 01:13:01 +01:00
Sergei Golubchik
79499b597a update the test result for new perfschema 2020-03-16 01:13:01 +01:00
Sergei Golubchik
8fd654ce0e MDEV-21943 reduce the binary tarball size
strip all client binaries (that is, not mysqld) in bintars
2020-03-16 01:13:01 +01:00
Sergei Golubchik
47e220a3a7 MDEV-21943 reduce the binary tarball size
in mysql_release builds only build embedded for rpm and deb
but not for bintar
2020-03-16 01:13:01 +01:00
Sergei Golubchik
0afccbf7b8 restore stack traces that were broken by ebfe8c4e0e 2020-03-16 01:13:01 +01:00
Kentoku SHIBA
2fde97119e Merge branch '10.5' of github.com:MariaDB/server into 10.5 2020-03-16 08:42:50 +09:00
Kentoku SHIBA
5929e222e4 fix evaluating bitmap issue in spider 2020-03-16 08:39:49 +09:00
Otto Kekäläinen
c8388de2fd Fix various spelling errors
e.g.
- dont -> don't
- occurence -> occurrence
- succesfully -> successfully
- easyly -> easily

Also remove trailing space in selected files.

These changes span:
- server core
- Connect and Innobase storage engine code
- OQgraph, Sphinx and TokuDB storage engines

Related to MDEV-21769.
2020-03-16 00:10:50 +02:00
Vladislav Vaintroub
3c57693ff1 MDEV-21534 - Improve innodb redo log group commit performance
Instrument new synchronization primitive with thd_wait_begin/end
to inform threadpool about waits.

This considerably improve performance on write benchmarks
(e.g sysbench update_index) with generic threadpool, of course the cost is
possibility of many newly created threads.
2020-03-15 21:40:11 +01:00
Varun Gupta
637c17588a MDEV-21922: Allow packing addon fields even if they don't honour max_length_for_sort_data
Addon fields will be packed if the length of addon fields is greater
than max_length_for_sort_data.
2020-03-15 23:48:51 +05:30
Andrei Elkin
345e21d2eb MDEV-742 test correction.
Fixed XA-ROLLBACK not to write into binlog when XA-prepare run
in sql_log_bin-OFF environment. (Two tests affected).
2020-03-15 14:14:35 +02:00
Sergey Vojtovich
8e5ae4e4df Test case fix: sort XA RECOVER result
Part of
MDEV-742 - XA PREPAREd transaction survive disconnect/server restart
2020-03-15 12:52:53 +04:00
Andrei Elkin
c8ae357341 MDEV-742 XA PREPAREd transaction survive disconnect/server restart
Lifted long standing limitation to the XA of rolling it back at the
transaction's
connection close even if the XA is prepared.

Prepared XA-transaction is made to sustain connection close or server
restart.
The patch consists of

    - binary logging extension to write prepared XA part of
      transaction signified with
      its XID in a new XA_prepare_log_event. The concusion part -
      with Commit or Rollback decision - is logged separately as
      Query_log_event.
      That is in the binlog the XA consists of two separate group of
      events.

      That makes the whole XA possibly interweaving in binlog with
      other XA:s or regular transaction but with no harm to
      replication and data consistency.

      Gtid_log_event receives two more flags to identify which of the
      two XA phases of the transaction it represents. With either flag
      set also XID info is added to the event.

      When binlog is ON on the server XID::formatID is
      constrained to 4 bytes.

    - engines are made aware of the server policy to keep up user
      prepared XA:s so they (Innodb, rocksdb) don't roll them back
      anymore at their disconnect methods.

    - slave applier is refined to cope with two phase logged XA:s
      including parallel modes of execution.

This patch does not address crash-safe logging of the new events which
is being addressed by MDEV-21469.

CORNER CASES: read-only, pure myisam, binlog-*, @@skip_log_bin, etc

Are addressed along the following policies.
1. The read-only at reconnect marks XID to fail for future
   completion with ER_XA_RBROLLBACK.

2. binlog-* filtered XA when it changes engine data is regarded as
   loggable even when nothing got cached for binlog.  An empty
   XA-prepare group is recorded. Consequent Commit-or-Rollback
   succeeds in the Engine(s) as well as recorded into binlog.

3. The same applies to the non-transactional engine XA.

4. @@skip_log_bin=OFF does not record anything at XA-prepare
   (obviously), but the completion event is recorded into binlog to
   admit inconsistency with slave.

The following actions are taken by the patch.

At XA-prepare:
   when empty binlog cache - don't do anything to binlog if RO,
   otherwise write empty XA_prepare (assert(binlog-filter case)).

At Disconnect:
   when Prepared && RO (=> no binlogging was done)
     set Xid_cache_element::error := ER_XA_RBROLLBACK
     *keep* XID in the cache, and rollback the transaction.

At XA-"complete":
   Discover the error, if any don't binlog the "complete",
   return the error to the user.

Kudos
-----
Alexey Botchkov took to drive this work initially.
Sergei Golubchik, Sergei Petrunja, Marko Mäkelä provided a number of
good recommendations.
Sergei Voitovich made a magnificent review and improvements to the code.
They all deserve a bunch of thanks for making this work done!
2020-03-14 22:45:48 +02:00
Monty
5754ea2eca Fixed compiler failures with gcc 7.4.1 and new my_malloc code 2020-03-14 15:24:13 +02:00
Sergei Golubchik
5c1ed707a3 mtr: update heuristics for --parallel=auto
to work better for CPUs with more than 2000 bogomips.
old behavior is preserved if less than 2500 bogomips.
2020-03-14 12:38:26 +01:00