Commit graph

1887 commits

Author SHA1 Message Date
Marko Mäkelä
2c1067166d Merge bb-10.2-ext into 10.3 2017-10-04 08:24:06 +03:00
Alexander Barkov
8ae8cd6348 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2017-10-02 22:35:13 +04:00
Vladislav Vaintroub
7354dc6773 MDEV-13384 - misc Windows warnings fixed 2017-09-28 17:20:46 +00:00
Vladislav Vaintroub
eba44874ca MDEV-13844 : Fix Windows warnings. Fix DBUG_PRINT.
- Fix win64 pointer truncation warnings
(usually coming from misusing 0x%lx and long cast in DBUG)

- Also fix printf-format warnings

Make the above mentioned warnings fatal.

- fix pthread_join on Windows to set return value.
2017-09-28 17:20:46 +00:00
Marko Mäkelä
e3d44f5d62 Merge bb-10.2-ext into 10.3 2017-09-21 08:12:19 +03:00
Marko Mäkelä
fc3b1a7d2f Merge 10.2 into bb-10.2-ext 2017-09-20 17:47:49 +03:00
Vicențiu Ciorbaru
246d321f67 Merge branch '10.2' of github.com:MariaDB/server into 10.2 2017-09-20 10:33:13 +03:00
Marko Mäkelä
5792b016f1 Fix a typo
The function check_mdl_lock_works() is for debug builds only.
2017-09-20 08:44:40 +03:00
Vicențiu Ciorbaru
22c322c649 Merge branch '10.1' into 10.2 2017-09-19 12:43:02 +03:00
Vicențiu Ciorbaru
ec6042bda0 Merge branch '10.0' into 10.1 2017-09-19 12:06:50 +03:00
Marko Mäkelä
e17a282da9 Merge bb-10.2-ext into 10.3 2017-09-18 11:38:07 +03:00
Sergei Golubchik
bba169b984 MDEV-12763 10.2 uses deprecated openssl 1.0 apis even with 1.1
Use OpenSSL 1.1 when applicable.
Create compatibility macros for OpenSSL 1.0- and YaSSL.
2017-09-18 10:12:23 +02:00
Marko Mäkelä
fe949504f0 Merge 10.2 into bb-10.2-ext 2017-09-17 14:03:51 +03:00
Vladislav Vaintroub
d1253e19a1 Fix compilation in mariabackup
Compilation got confused  about 2 wsrep.h headers in include path
Rename backup's wsrep.h to backup_wsrep.h to fixO
2017-09-16 22:19:16 +02:00
Vladislav Vaintroub
ad17e8e518 MDEV-13821 : mariabackup sometimes could lose ib_logf(FATAL) messages,
The messages are getting lost because they are written with fprintf()
but without fflush(), so abort() would lose buffered text.

Applied fix from 10.2, which makes in_logf() use
sql_print_information(), which in turn does proper flush after each message
2017-09-16 09:45:38 +00:00
Vladislav Vaintroub
74f677fcc2 MDEV-13802 mariabackup --lock-ddl-per-table fails when table names contain backticks
use ut_get_name() for formatting database/table names.
2017-09-15 15:49:53 +00:00
Vladislav Vaintroub
bb7ab40521 mariabackup : Fix enumerate_ibd_files() to include .isl 2017-09-15 15:49:53 +00:00
Marko Mäkelä
348eaf4252 Merge bb-10.2-ext into 10.3 2017-09-14 09:12:47 +03:00
Marko Mäkelä
2bd6ccae72 Merge 10.2 into bb-10.2-ext 2017-09-14 09:00:10 +03:00
Marko Mäkelä
3f17f51132 Follow-up to MDEV-13563 mariabackup --lock-ddl-per-table
Fix memory leaks, and add a missing newline to a message.
2017-09-14 08:58:31 +03:00
Alexander Barkov
178540b904 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2017-09-13 12:12:05 +04:00
Marko Mäkelä
66a09bd6ab MDEV-13318 Crash recovery failure after the server is killed during innodb_encrypt_log startup
This fixes several InnoDB bugs related to innodb_encrypt_log and
two Mariabackup --backup bugs.

log_crypt(): Properly derive the initialization vector from the
start LSN of each block. Add a debug assertion.

log_crypt_init(): Note that the function should only be used when
creating redo log files and that the information is persisted in
the checkpoint pages.

xtrabackup_copy_log(): Validate data_len.

xtrabackup_backup_func(): Always use the chosen checkpoint buffer.

log_group_write_buf(), log_write_up_to(): Only log_crypt() the redo
log payload, not the padding bytes.

innobase_start_or_create_for_mysql(): Do not invoke log_crypt_init()
or initiate a redo log checkpoint.

recv_find_max_checkpoint(): Return the contents of LOG_CHECKPOINT_NO
to xtrabackup_backup_func() in log_sys->next_checkpoint_no.
2017-09-12 11:32:49 +03:00
Vladislav Vaintroub
31774f0ede MDEV-13563 lock DDL for mariabackup in 10.2
Implement lock-ddl-per-table option that locks tables before it
is copied to backup, and helds the lock until backup finished

The "DDL-lock" itself is implemented as "SELECT * from <table> LIMIT 0",
inside a transaction, and "COMMIT" of this transaction is the DDL-unlock.
2017-09-12 05:57:05 +00:00
Vladislav Vaintroub
a46679dcf7 Build improvements and cleanups.
- the probably ultimate fix for dependencies on VS
- remove some GET_TARGET_PROPERTY(LOCATION ...), they are deprecated in
cmake 3.9
- simplify signing targets on Windows.
- remove INSTALL_DEBUG_TARGET, we do not mix binaries from different builds
in the same package
2017-09-08 18:22:15 +00:00
Marko Mäkelä
474f51711b Merge bb-10.2-ext into 10.3 2017-09-08 15:59:06 +03:00
Marko Mäkelä
fb14761d66 Merge 10.2 into bb-10.2-ext 2017-09-08 15:44:34 +03:00
Vladislav Vaintroub
18e17f134c Windows : Fix MTR's misuse of servers --console parameter 2017-09-08 08:33:41 +02:00
Vladislav Vaintroub
d471469bd2 MDEV-13466 Implement --export option for MariaDB Backup
full server recovery is performed .
We start "mysqld" with  --bootstrap
and pass bootstrap script consisting of several FLUSH TABLES FOR export/
UNLOCK TABLES
2017-09-07 22:54:06 +00:00
Marko Mäkelä
ae02407ce3 Merge bb-10.2-ext into 10.3 2017-09-07 12:30:56 +03:00
Marko Mäkelä
a82ed92a6a Merge 10.2 into bb-10.2-ext 2017-09-07 12:23:58 +03:00
Marko Mäkelä
b429e8cada Mariabackup: Detach the threads at exit 2017-09-06 19:10:12 +03:00
Marko Mäkelä
4e1fa7f63d Merge bb-10.2-ext into 10.3 2017-09-01 11:33:45 +03:00
Marko Mäkelä
6749d39a95 Merge 10.2 into bb-10.2-ext 2017-09-01 08:47:55 +03:00
Jan Lindström
eca238aea7 MDEV-13557: Startup failure, unable to decrypt ibdata1
Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though
encryption is not enabled.

Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on
shutdown after failed startup.

Problem was that we created encryption metadata (crypt_data) for
system tablespace even when no encryption was enabled and too early.
System tablespace can be encrypted only using key rotation.

Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables
require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION
contain row only if tablespace really has encryption metadata.

xb_load_single_table_tablespace(): Do not call
fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data
has been removed.

fil_crypt_realloc_iops(): Avoid divide by zero.

fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if
encryption threads exist. This is required to find tablespaces
requiring key rotation if no other changes happen.

fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting
when nothing happens to better enable key rotation on startup.

fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first
page.

class Datafile, class SysTablespace : remove m_crypt_info field.

Datafile::get_first_page(): Return a pointer to first page buffer.

fsp_header_init(): Write encryption metadata to page 0 only if
tablespace is encrypted or encryption is disabled by table option.

i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not
contain encryption metadata. This is required to avoid too early
wait condition trigger in encrypted -> unencrypted state transfer.
2017-08-31 08:36:56 +03:00
Marko Mäkelä
f192b48d62 Merge 10.1 into 10.2 2017-08-29 10:07:33 +03:00
Sergei Golubchik
bb8e99fdc3 Merge branch 'bb-10.2-ext' into 10.3 2017-08-26 00:34:43 +02:00
Sergei Golubchik
27412877db Merge branch '10.2' into bb-10.2-ext 2017-08-25 10:25:48 +02:00
Michael Widenius
458d5ed8aa Lots of small cleanups
- Simplified use_trans_cache() to return at once if is_transactional is set
- Indentation and spelling errors fixed
- Don't call signal_update() if update_binlog_end_pos() is called as the
  function already calls signal_update()
- Removed not used function wait_for_update_bin_log(), which would cause
  errors if ever used.
- Simplified handler::clone() by always allocating 'ref' in ha_open(). To do
  this I added an optional MEM_ROOT argument to ha_open() to be used when
  allocating 'ref'
- Changed arguments to get_system_var() from LEX_CSTRING to LEX_CSTRING*
- Added THD as argument to create_select_for_variable(). Changed also char*
  argument to LEX_CSTRING to avoid strlen() call.
- Change calls to append() to use LEX_CSTRING
2017-08-24 01:05:49 +02:00
Sergei Golubchik
d5a6bae7c0 Fix compilation errors 2017-08-24 01:05:48 +02:00
Sergei Golubchik
4040a17ea2 Compile mariabackup with its own copy of net_serv.cc
Don't use the server's version, that expects a valid THD.
Modify net_serv.cc not not use any THD if MYSQL_SERVER isn't defined.

This reverts commit aaddac5cd7.
2017-08-23 19:05:13 +02:00
Vladislav Vaintroub
aaddac5cd7 fix compile errors 2017-08-23 08:27:46 +00:00
Vladislav Vaintroub
9af7561eb4 MDEV-13608 : set client plugin directory with mysql_options()
if plugin_dir is specified. Also, allow to specify protocol (e.g pipe)
2017-08-21 17:16:12 +00:00
Marko Mäkelä
8a3e2970ad MDEV-13575 On failure, Mariabackup --backup --safe-slave-backup may forget to START SLAVE SQL_THREAD
backup_release(): New function, refactored from backup_finish().
Release some resources that may have been acquired by backup_startup()
and should be released even after a failed operation.

xtrabackup_backup_low(): Refactored from xtrabackup_backup_func().

xtrabackup_backup_func(): Always call backup_release() after calling
backup_start().
2017-08-18 21:42:33 +03:00
Marko Mäkelä
605b835220 MDEV-13754 Memory leak in mariabackup.incremental_backup
The test mariabackup.incremental_backup revealed a memory leak
in have_queries_to_wait_for(). The problem is that
xb_mysql_query() is being invoked with bool use_result=true
but the result is not being freed by mysql_store_result().
There are similar leaks in other functions.

have_queries_to_wait_for(): Invoke mysql_free_result() to
clean up after the mysql_store_result() that was invoked
by xb_mysql_query().

select_incremental_lsn_from_history(): Plug the leak on failure.

kill_long_queries(): Plug the memory leak.
(This function always leaked memory when it was called.)
2017-08-18 10:31:09 +03:00
Marko Mäkelä
74ce0cf148 MDEV-13574 related Mariabackup code cleanup (non-functional change)
have_queries_to_wait_for(), kill_long_queries(): Declare and initialize
variables in one go.
2017-08-18 10:18:26 +03:00
Marko Mäkelä
e9e051d297 Follow-up fix to MDEV-12988 backup fails if innodb_undo_tablespaces>0
The fix broke mariabackup --prepare --incremental.

The restore of an incremental backup starts up (parts of) InnoDB twice.
First, all data files are discovered for applying .delta files. Then,
after the .delta files have been applied, InnoDB will be restarted
more completely, so that the redo log records will be applied via the
buffer pool.

During the first startup, the buffer pool is not initialized, and thus
trx_rseg_get_n_undo_tablespaces() must not be invoked. The apply of
the .delta files will currently assume that the --innodb-undo-tablespaces
option correctly specifies the number of undo tablespace files, just
like --backup does.

The second InnoDB startup of --prepare for applying the redo log will
properly invoke trx_rseg_get_n_undo_tablespaces().

enum srv_operation_mode: Add SRV_OPERATION_RESTORE_DELTA for
distinguishing the apply of .delta files from SRV_OPERATION_RESTORE.

srv_undo_tablespaces_init(): In mariabackup --prepare --incremental,
in the initial SRV_OPERATION_RESTORE_DELTA phase, do not invoke
trx_rseg_get_n_undo_tablespaces() because the buffer pool or the
redo logs are not available. Instead, blindly rely on the parameter
--innodb-undo-tablespaces.
2017-08-18 09:12:04 +03:00
Sergei Golubchik
cb1e76e4de Merge branch '10.1' into 10.2 2017-08-17 11:38:34 +02:00
Marko Mäkelä
5a43c8bae7 MDEV-13416 mariabackup --backup fails to copy log if LSN is above 4294967295
xtrabackup_copy_log(): Use a 64-bit bitmask (lsn_t), not 32-bit (int).
2017-08-11 16:42:27 +03:00
Marko Mäkelä
620ba97cfc Merge remote-tracking branch 'origin/bb-10.2-ext' into 10.3 2017-08-09 12:59:39 +03:00
Marko Mäkelä
1f0a22acbd Merge 10.2 into bb-10.2-ext 2017-08-08 09:47:00 +03:00