Replace calls to `sprintf` and `strcpy` by the safer options `snprintf`
and `safe_strcpy` in the following directories:
- libmysqld
- mysys
- sql-common
- strings
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
Problem:
========
If a primary is shutdown during an active semi-sync connection
during the period when the primary is awaiting an ACK, the primary
hard kills the active communication thread and does not ensure the
transaction was received by a replica. This can lead to an
inconsistent replication state.
Solution:
========
During shutdown, the primary should wait for an ACK or timeout
before hard killing a thread which is awaiting a communication. We
extend the `SHUTDOWN WAIT FOR SLAVES` logic to identify and ignore
any threads waiting for a semi-sync ACK in phase 1. Then, before
stopping the ack receiver thread, the shutdown is delayed until all
waiting semi-sync connections receive an ACK or time out. The
connections are then killed in phase 2.
Notes:
1) There remains an unresolved corner case that affects this
patch. MDEV-28141: Slave crashes with Packets out of order when
connecting to a shutting down master. Specifically, If a slave is
connecting to a master which is actively shutting down, the slave
can crash with a "Packets out of order" assertion error. To get
around this issue in the MTR tests, the primary will wait a small
amount of time before phase 1 killing threads to let the replicas
safely stop (if applicable).
2) This patch also fixes MDEV-28114: Semi-sync Master ACK Receiver
Thread Can Error on COM_QUIT
Reviewed By
============
Andrei Elkin <andrei.elkin@mariadb.com>
This change removed 68 explict strlen() calls from the code.
The following renames was done to ensure we don't use the old names
when merging code from earlier releases, as using the new variables
for print function could result in crashes:
- charset->csname renamed to charset->cs_name
- charset->name renamed to charset->coll_name
Almost everything where mechanical changes except:
- Changed to use the new Protocol::store(LEX_CSTRING..) when possible
- Changed to use field->store(LEX_CSTRING*, CHARSET_INFO*) when possible
- Changed to use String->append(LEX_CSTRING&) when possible
Other things:
- There where compiler issues with ensuring that all character set names
points to the same string: gcc doesn't allow one to use integer constants
when defining global structures (constant char * pointers works fine).
To get around this, I declared defines for each character set name
length.
This patch changes the main name of 3 byte character set from utf8 to
utf8mb3. New old_mode UTF8_IS_UTF8MB3 is added and set TRUE by default,
so that utf8 would mean utf8mb3. If not set, utf8 would mean utf8mb4.
Shifting a 16-bit type by 16 bits is undefined behaviour.
The result is at least 32 bits, so let us cast the shift operand
to a wider type before shifting.
As an additional improvement, let's store string representations of
the numbers using an array uint16[256] instead of char[512].
This allows to use int2store(), which copies two bytes at a time on x86,
instead of copying the two bytes with digits one-by-one.
This change gives an additional 7% to 26% query time reduce for:
SELECT BENCHMARK(10*1000*1000,CONCAT(TIME'10:20:30'));
SELECT BENCHMARK(10*1000*1000,CONCAT(TIME'10:20:30.123456'));
SELECT BENCHMARK(10*1000*1000,CONCAT(DATE'2001-01-01'));
SELECT BENCHMARK(10*1000*1000,CONCAT(TIMESTAMP'2001-01-01 10:20:30'));
SELECT BENCHMARK(10*1000*1000,CONCAT(TIMESTAMP'2001-01-01 10:20:30.123456'));
The total time reduce (part1 + part2) is now between 15% to 38%
for these queries.
It is already in libmariadb, and server (also that client in server)
does not need it.
It does not work in embedded either since it relies on non-blocking sockets
This patch improves functions my_xxx_to_str() using the idea
introduced in this change in MySQL-8.0:
commit 8d10f2fff6bbdea7f436b868ebb5fd811defc68a
Author: Knut Anders Hatlen <knut.hatlen@oracle.com>
Date: Thu Oct 10 13:55:07 2019 +0200
Bug#30472888: IMPROVE THE PERFORMANCE OF INTEGER HANDLING IN THE TEXT PROTOCOL
The new way prints 2 digits at a time and demonstrates a very impressing query time reduce:
10% to 38%, depending on the exact data type and the number of fractional digits:
SELECT BENCHMARK(10*1000*1000,CONCAT(TIME'10:20:30'));
SELECT BENCHMARK(10*1000*1000,CONCAT(TIME'10:20:30.123456'));
SELECT BENCHMARK(10*1000*1000,CONCAT(DATE'2001-01-01'));
SELECT BENCHMARK(10*1000*1000,CONCAT(TIMESTAMP'2001-01-01 10:20:30'));
SELECT BENCHMARK(10*1000*1000,CONCAT(TIMESTAMP'2001-01-01 10:20:30.123456'));
See MDEV for details on the benchmark results.
Protocol_local fixed so it can be used now.
Some Protocol:: methods made virtual so they can adapt.
as well as net_ok and net_send_error functions.
execute_sql_string function is exported to the plugins.
To be changed with the mysql_use_result.
Problem:
=======
rpl.rpl_slave_load_tmpdir_not_exist 'stmt' w3 [ fail ] Found warnings/errors
in server log file!
Test ended at 2017-09-27 20:34:55
[Warning] Master is configured to log replication events with checksum, but
will not send such events to slaves that cannot process them
^ Found warnings in /mnt/buildbot/build/mariadb-10.2.10/mysql-test/var/3/log/mysqld.1.err
ok
Analysis:
========
When slave tries to connect to master 'get_master_version_and_clock' function
is invoked to perform elaborated slave-master handshake. During this process
slave server queries master server, to know if it is checksum aware and at the
same time master is notified about its CRC-awareness. The master's side
instant value of @@global.binlog_checksum is stored in the dump thread's
uservar area as well as cached locally to become known in consensus by master
and slave.
Post hand-shake slave requests master for binlog dump. It sends
'COM_BINLOG_DUMP'. This command is sent to master by 'cli_advanced_command'
call. If there is some temporary network failure during this request_dump
call, 'end_server' is invoked to close the current connection between master
and slave. Upon connection close the dump thread on the master gets terminated
and it clears the 'uservar' data it got through master-slave handshake.
The 'COM_BINLOG_DUMP' command is sent once again without master-slave
handshake. Since the checksum data is not available with new dump thread a
warning gets reported.
Fix:
===
Upon network write error donot attempt reconnect, proceed to master-slave
handshake. This ensures that master is aware of slave's capability to use
checksums.
DESCRIPTION:
============
There can be issues if the packets sent by the server
are not proper. Certain checks should be performed at the
client side while unpacking fields data.
FIX:
====
Check for the appropriate fields data and error out if it
is not present.
RB: 23601
Changed wording in error messages from MySQL to MariaDB. In
cases where the word server could be used instead it was done.
Tests that have these errors recorded were updated.
only my_getopt should use it, because it changes my_getopt's behavior.
If one simply wants to skip the separator - don't ask it to be added
in the first place
The MDEV-20265 commit e746f451d5
introduces DBUG_ASSERT(right_op == r_tbl) in
st_select_lex::add_cross_joined_table(), and that assertion would
fail in several tests that exercise joins. That commit was skipped
in this merge, and a separate fix of MDEV-20265 will be necessary in 10.4.
