Analysis
Mysqlbinlog output for encrypted binary log
#Q> insert into tab1 values (3,'row 003')
#190912 17:36:35 server id 10221 end_log_pos 980 CRC32 0x53bcb3d3 Table_map: `test`.`tab1` mapped to number 19
# at 940
#190912 17:36:35 server id 10221 end_log_pos 1026 CRC32 0xf2ae5136 Write_rows: table id 19 flags: STMT_END_F
Here we can see Table_map_log_event ends at 980 but Next event starts at 940.
And the reason for that is we do not send START_ENCRYPTION_EVENT to the slave
Solution:-
Send Start_encryption_log_event as Ignorable_log_event to slave(mysqlbinlog),
So that mysqlbinlog can update its log_pos.
Since Slave can request multiple FORMAT_DESCRIPTION_EVENT while master does not
have so We only update slave master pos when master actually have the
FORMAT_DESCRIPTION_EVENT. Similar logic should be applied for START_ENCRYPTION_EVENT.
Also added the test case when new server reads the data from old server which
does not send START_ENCRYPTION_EVENT to slave.
Master Slave Upgrade Scenario.
When Slave is updated first, Slave will have extra logic of handling
START_ENCRYPTION_EVENT But master willnot be sending START_ENCRYPTION_EVENT.
So there will be no issue.
When Master is updated first, It will send START_ENCRYPTION_EVENT to
slave , But slave will ignore this event in queue_event.
This patch allows the server to open old tables that have
"bad" generated columns (i.e. indexed virtual generated columns,
persistent generated columns) that depend on sql_mode,
for general things like SELECT, INSERT, DROP, etc.
Warning are issued in such cases.
Only these commands are now disallowed and return an error:
- CREATE TABLE introducing a "bad" generated column
- ALTER TABLE introducing a "bad" generated column
- CREATE INDEX introdicing a "bad" generated column
(i.e. adding an index on a virtual generated column
that depends on sql_mode).
Note, these commands are allowed:
- ALTER TABLE removing a "bad" generate column
- ALTER TABLE removing an index from a "bad" virtual generated column
- DROP INDEX removing an index from a "bad" virtual generated column
but only if the table does not have any "bad" columns as a result.
for passing ones.
Changes to be committed:
new file: mysql-test/std_data/galera-cert.pem
new file: mysql-test/std_data/galera-key.pem
new file: mysql-test/std_data/galera-upgrade-ca-cert.pem
new file: mysql-test/std_data/galera-upgrade-server-cert.pem
new file: mysql-test/std_data/galera-upgrade-server-key.pem
modified: mysql-test/suite/galera/disabled.def
modified: mysql-test/suite/galera/r/MW-416.result
modified: mysql-test/suite/galera/r/MW-44.result
modified: mysql-test/suite/galera/r/galera_sst_mysqldump_with_key,debug.rdiff
modified: mysql-test/suite/galera/r/galera_sst_mysqldump_with_key.result
modified: mysql-test/suite/galera/t/MW-416.test
modified: mysql-test/suite/galera/t/galera_kill_applier.test
* fix CRL tests to work
* regenerate certificates to be at least 2048 bit
(fixes buster and rhel8 in buildbot)
* update generate-ssl-cert.sh to generate crl files
* make all SSL tests to use certificates generated
in generate-ssl-cert.sh, remove unused certificates
Backport from 10.4 9c60535f86
* fix CRL tests to work
* regenerate certificates to be at least 2048 bit
(fixes buster and rhel8 in buildbot)
* update generate-ssl-cert.sh to generate crl files
* make all SSL tests to use certificates generated
in generate-ssl-cert.sh, remove unused certificates
use frm_version, not mysql_version when parsing frm
In particular, virtual columns are stored according to
frm_version. And CHECK TABLE will overwrite mysql_version
to the current server version, so it cannot correctly
describe frm format.
The previous patch 269da4bf19 was
actually for MDEV-8894 (not for MDEV-5377). It was erroneously
pushed with a wrong title.
This patch is a small cleanup for MDEV-8894.
CREATE TABLE is now not a part of binary logs recorded with MySQL,
only INSERT statements are. This will allow to reuse the same binary
logs in combinations with different CREATE TABLE statements,
to tests different data types.
Test galera checks that a `GRA_x_x.log` file is created whenever
wsrep applier fails to apply some replication event. The file
contains the corresponding binlog event that failed to apply.
The test creates a new file by concatenating a pre-recorded
file containing the binlog header (see `std-data/binlog-header.log`)
and the `GRA_x_x.log` file. The test then checks that the resulting
file, containing the binlog header and the event that failed to
apply, is correctly read by `mysqlbinlog` program.
The test fails in MariaDB because the GRA_x_x.log file created
by MariaDB already contains the binlog header (see MDEV-7867).
This patch fixes/simplifies test `galera.galera_gra_log` so that
it doesn't concatenate `std-data/binlog-header.log` with the
`GRA_x_x.log` file. File `std-data/binlog-header.log` is deleted
altoghether, because not used by any other test.
- Adding a new virtual method Field::load_data_set_no_data().
- Overriding Field_timestamp::load_data_set_no_data() and moving
the TIMESTAMP specific code there.
- Overriding Field_geom::load_data_set_no_data() and implementing
GEOMETRY specific behavior, to prevent writing empty strings
when the loaded file ends unexpectedly. This fixes the bug.
- Adding a new test gis-loaddaata.test.
- The test in loaddata.test for CHAR was added simply to record behavior.
The CHAR data type did not change its behaviour (only GEOMRYRY did).
- Additionally, moving duplicate code into a new method
Field::load_data_set_value() and reusing it in three places.
Now we don't open partitions if it was explicitly cpecified.
ha_partition::m_opened_partition bitmap added to track
partitions that were actually opened.
The fixes for these bugs:
Bug#27586 Wrong autoinc value assigned by LOAD DATA in the NO_AUTO_VALUE_ON_ZERO mode
Bug#22372 Disable spatial key, load data, enable spatial key, crashes table
fixed only LOAD DATA INFILE, but did not fix LOAD XML INFILE.
This patch does for LOAD XML FILE what patches for Bug#27586 and Bug#22372
earlier did for LOAD DATA INFILE.
1. Fixing the auto_increment problem:
a. table->auto_increment_field_not_null is not set to TRUE
anymore when a column does not have a corresponding XML tag.
b. Adding "table->auto_increment_field_not_null= false"
in the end of read_xml_field().
These two changes resemble the patch for Bug#27586.
2. Fixing the GEOMETRY problem:
The result for "reset()" was not tested for errors in read_xml_field(),
which made it possible for empty string to sneak into a "GEOMETRY NOT NULL"
column when this column does not have a corresponding XML tag with data.
After this patch the result of reset() is tested and and an error is
returned in such cases.
This change effectively resembles the patch for Bug#22372
3. Spliting the code into a new virtual method Field::load_data_set_null().
Rationale:
a. To avoid duplicate code in read_sep_field() and read_xml_field():
Changes #1 and #2 made the code handling NULL values for Field
exactly the same in read_sep_field() and read_xml_field().
b. To avoid tests for field_type(), which is not friendly to
upcoming data type plugins.
This change makes it possible for data type plugins
to implement their own special way for handling NULL values in LOAD DATA
by overriding Field_xxx::load_data_set_null(),
like Field_geom and Field_timestamp do.