Commit graph

177011 commits

Author SHA1 Message Date
Vladislav Vaintroub
66905f6dcb Fix several compile warnings on Windows 2017-03-17 08:56:57 +00:00
Sergei Petrunia
131d858206 Temporarily disable rocksdb.blind_delete_without_tx_api test 2017-03-17 04:46:01 +03:00
Sergei Petrunia
dd743dae32 Update test results for rocksdb.misc
This .result file is not a statement of which storage engine
should be used for any particular table in mysql database.

This is just a check that a query against I_S doesn't crash.
2017-03-17 04:05:03 +03:00
Sergei Petrunia
c707997e15 MariaRocks: run rocksdb testsuite with --default-storage-engine=rocksdb
Most tests use CREATE TABLE ... ENGINE=ROCKSB, but there are some
exceptions: rpl_savepoint, rpl_row_stats.
In order to avoid any "oh we are using the wrong storage engine"
surprises, set the default for the whole testsuite.
2017-03-17 01:21:11 +03:00
Sergei Petrunia
46a78868fe MariaRocks: make rocksdb.rocksdb_datadir test pass
The test runs $MYSQLD_BOOTSTRAP_CMD but that command does not include
arguments for loading ha_rocksdb.so plugin. Add them.
2017-03-17 00:05:48 +03:00
Sergei Petrunia
23f9bb966b MDEV-12285: MariaRocks: "[ERROR] mysqld: Deadlock ..." messages in server stderr
The mssages are caused by log_warnings=2.
Set log_warnings=1 for the rocksdb test suite.
2017-03-16 21:28:42 +03:00
Jan Lindström
b1ec35b903 Add assertions when key rotation list is used. 2017-03-16 17:30:13 +02:00
Sergei Golubchik
8971286a3c compiler warning 2017-03-16 14:03:17 +01:00
Monty
2d0c579a86 Wait for slave threads to start during startup
- Before this patch during startup all slave threads was started without
  any check that they had started properly.
- If one did a START SLAVE, STOP SLAVE or CHANGE MASTER as first command to the server
  there was a chance that server could access structures that where not
  properly  initialized which could lead to crashes in
  Log_event::read_log_event
- Fixed by waiting for slave threads to start up properly also during
  server startup, like we do with START SLAVE.
2017-03-16 14:21:33 +02:00
Monty
e7f55fde88 Removed wrong assert
The following is an updated commit message for the following commit
that was pushed before I had a chance to update the commit message:
c5e25c8b40

Fixed dead locks when doing stop slave while slave was starting.

- Added a separate lock for protecting start/stop/reset of a specific slave.
  This solves some possible dead locks when one calls stop slave while
  the slave is starting as the old run_locks was over used for other things.
- Set hash->records to 0 before calling free of all hash elements.
  This was set to stop concurrent threads to loop over hash elements and
  access members that was already freed.
  This was a problem especially in start_all_slaves/stop_all_slaves
  as the mutex protecting the hash was temporarily released while a slave
  was started/stopped.
- Because of change to hash->records during hash_reset(),
  any_slave_sql_running() will return 1 during shutdown as one can't
  loop over master_info_index->master_info_hash while hash_reset() of it
  is in progress.
  This also fixes a potential old bug in any_slave_sql_running() where
  during shutdown and ~Master_info_index(), my_hash_free() we could
  potentially try to access elements that was already freed.
2017-03-16 14:21:32 +02:00
=Ian Gilfillan
c333cae652 MDEV-11964 Add more stub missing man pages 2017-03-16 15:42:13 +04:00
Marko Mäkelä
7668a79a88 MDEV-12269 Port Bug#22996442 INNODB: MAKE UNIV_DEBUG DEPEND ON DBUG_OFF
This is a partial port of my patch in MySQL 8.0.
In MySQL 8.0, all InnoDB references to DBUG_OFF were replaced
with UNIV_DEBUG. We will not do that in MariaDB.

InnoDB used two independent compile-time flags that distinguish
debug and non-debug builds, which is confusing.

Also, make ut_ad() and alias of DBUG_ASSERT().
2017-03-16 10:24:53 +02:00
Marko Mäkelä
105f46ffb8 MDEV-12273 Remove dict_table_t::does_not_fit_in_memory
In the InnoDB internal SQL parser, there is the keyword
DOES_NOT_FIT_IN_MEMORY that is never specified in any CREATE TABLE
statement that is passed to the InnoDB SQL parser
(que_eval_sql() or pars_sql() or yyparse()). If this keyword were
ever present, it would set the flag dict_table_t::does_not_fit_in_memory
which is only present in debug builds.

Let us remove all traces of this.

Also, fix storage/innobase/pars/make_flex.sh so that no the generated
file storage/innobase/pars/lexyy.cc works as is.

FIXME: Always generate the InnoDB Bison files at build time, similar
to how sql/sql_yacc.yy is handled. (This would still leave the
generated scanner files, unless we want to add a build-time dependency
for Flex.)
2017-03-16 10:24:53 +02:00
Vicențiu Ciorbaru
49de95679d Revert "Make rocksdb build as a deb package too"
This reverts commit 38919f68a1.

Temporary revert to be able to see other failures until builders are
updated.
2017-03-16 09:50:26 +02:00
Sergei Petrunia
17d7cc731e MDEV-12277: rocksdb.rocksdb fails with Sort Aborted error in server stderr
Add a suppression
2017-03-16 09:46:01 +03:00
Varun Gupta
aad15eae89 Test result for MDEV-10766 fixed 2017-03-16 10:18:02 +05:30
Sergei Petrunia
c5a20553c0 More testsuite fixes
- Disable rocksdb.show_enge
- Disable rocksdb.rpl_row_not_found

- Run rocksdb.blind_delete_without_tx_api only with binlog_format=row
  (like its .cnf file specifies)
2017-03-16 01:12:01 +03:00
Vicențiu Ciorbaru
38919f68a1 Make rocksdb build as a deb package too 2017-03-15 23:20:20 +02:00
Sergei Petrunia
adb7470742 Disable rocksdb.rpl_row_triggers, rocksdb.trx_info_rpl
These are not expected to work yet.
2017-03-15 23:44:16 +03:00
Sergei Petrunia
6dc2d581d4 Make rocksdb.rocksdb_range pass: MariaDB doesnt support ICP over reverse index scans atm 2017-03-15 23:36:20 +03:00
Sergei Petrunia
c010f06380 MariaRocks: Run rocksdb testsuite with @@rocksdb_flush_log_at_trx_commit=0
The default value of 1 causes many tests to time out (primary reason is
that many tests populate tables with one-row INSERT statements that
run with autocommit=1).
2017-03-15 23:20:28 +03:00
Sergei Golubchik
2c2bd8c155 MDEV-12261 build failure without P_S
restore mysql_file_delete_with_symlink() but let it use
new my_handler_delete_with_symlink() mysys helper.
2017-03-15 17:48:30 +01:00
Sergei Petrunia
bf578ff920 Add missing source include/have_rocksdb.inc 2017-03-15 16:02:37 +00:00
Varun Gupta
cd7e6d8b53 MDEV-11645: archive.archive fails in buildbot with valgrind (Use of uninitialised value)
The fix is about filling the space beyond the end of VARCHAR values with zeroes.
VARCHAR NULLs already has its buffer filled with zeros. So when the VARCHAR fields
are not NULL, then we explicitly fill the buffer with zeros.
2017-03-15 21:26:39 +05:30
Varun Gupta
6ac754163c MDEV-10766: Queries which start with WITH clause do not get inserted into query cache
Added conditions so that the WITH queries are also added to the query cache
2017-03-15 20:15:31 +05:30
Oleksandr Byelkin
122d0701f7 MDEV-11761: CLIENT_DEPRECATE_EOF : Client must identify a "stored procedure output resultset"
fix proposed by Diego
2017-03-15 13:45:14 +01:00
Alexey Botchkov
5dd4d663fa Test result fixed. 2017-03-15 15:51:44 +04:00
Vladislav Vaintroub
854359ffc5 Fix AWS KMS plugin's compile error
Do not include base64.h, it was recently removed.
2017-03-14 20:44:25 +00:00
Marko Mäkelä
b5285bd7e2 MDEV-11873 Unnecessary InnoDB warnings upon bootstrap
dict_create_or_check_foreign_constraint_tables(): Change the warning
about the foreign key metadata table creation to a note.

Remove messages after metadata table creation. If the creation fails,
startup will abort with a message. Normally the creation succeeds on
bootstrap, and the messages would only be noise.

Remove the related suppressions from the tests.
2017-03-14 17:11:46 +02:00
Jan Lindström
50eb40a2a8 MDEV-11738: Mariadb uses 100% of several of my 8 cpus doing nothing
MDEV-11581: Mariadb starts InnoDB encryption threads
when key has not changed or data scrubbing turned off

Background: Key rotation is based on background threads
(innodb-encryption-threads) periodically going through
all tablespaces on fil_system. For each tablespace
current used key version is compared to max key age
(innodb-encryption-rotate-key-age). This process
naturally takes CPU. Similarly, in same time need for
scrubbing is investigated. Currently, key rotation
is fully supported on Amazon AWS key management plugin
only but InnoDB does not have knowledge what key
management plugin is used.

This patch re-purposes innodb-encryption-rotate-key-age=0
to disable key rotation and background data scrubbing.
All new tables are added to special list for key rotation
and key rotation is based on sending a event to
background encryption threads instead of using periodic
checking (i.e. timeout).

fil0fil.cc: Added functions fil_space_acquire_low()
to acquire a tablespace when it could be dropped concurrently.
This function is used from fil_space_acquire() or
fil_space_acquire_silent() that will not print
any messages if we try to acquire space that does not exist.
fil_space_release() to release a acquired tablespace.
fil_space_next() to iterate tablespaces in fil_system
using fil_space_acquire() and fil_space_release().
Similarly, fil_space_keyrotation_next() to iterate new
list fil_system->rotation_list where new tables.
are added if key rotation is disabled.
Removed unnecessary functions fil_get_first_space_safe()
fil_get_next_space_safe()

fil_node_open_file(): After page 0 is read read also
crypt_info if it is not yet read.

btr_scrub_lock_dict_func()
buf_page_check_corrupt()
buf_page_encrypt_before_write()
buf_merge_or_delete_for_page()
lock_print_info_all_transactions()
row_fts_psort_info_init()
row_truncate_table_for_mysql()
row_drop_table_for_mysql()
    Use fil_space_acquire()/release() to access fil_space_t.

buf_page_decrypt_after_read():
    Use fil_space_get_crypt_data() because at this point
    we might not yet have read page 0.

fil0crypt.cc/fil0fil.h: Lot of changes. Pass fil_space_t* directly
to functions needing it and store fil_space_t* to rotation state.
Use fil_space_acquire()/release() when iterating tablespaces
and removed unnecessary is_closing from fil_crypt_t. Use
fil_space_t::is_stopping() to detect when access to
tablespace should be stopped. Removed unnecessary
fil_space_get_crypt_data().

fil_space_create(): Inform key rotation that there could
be something to do if key rotation is disabled and new
table with encryption enabled is created.
Remove unnecessary functions fil_get_first_space_safe()
and fil_get_next_space_safe(). fil_space_acquire()
and fil_space_release() are used instead. Moved
fil_space_get_crypt_data() and fil_space_set_crypt_data()
to fil0crypt.cc.

fsp_header_init(): Acquire fil_space_t*, write crypt_data
and release space.

check_table_options()
	Renamed FIL_SPACE_ENCRYPTION_* TO FIL_ENCRYPTION_*

i_s.cc: Added ROTATING_OR_FLUSHING field to
information_schema.innodb_tablespace_encryption
to show current status of key rotation.
2017-03-14 16:23:10 +02:00
Daniel Bartholomew
a2f34809e5 bump the VERSION 2017-03-14 09:56:05 -04:00
Alexey Botchkov
af6eee1fc5 MDEV-11833 JSON functions don't seem to respect max_allowed_packet.
Now let's check JSON length to fit the max_allowed packet.
2017-03-14 17:31:14 +04:00
Alexey Botchkov
d0e8b427a1 MDEV-12078 Using spatial index changes type from point to geometry
In get_mm_tree we have to change Field_geom::geom_type to
        GEOMETRY as we have to let storing all types of the spatial features
        in the field. So now we restore the original geom_type as it's
        done.
2017-03-14 16:35:39 +04:00
Varun Gupta
adbe1c5fe9 MDEV-6486: Assertion `!table || (!table->read_set || bitmap_is_set(table->read_set, field_index))'
failed with SELECT SQ, TEXT field

The functon find_all_keys does call Item_subselect::walk, which calls walk() for the subquery
The issue is that when a field is represented by Item_outer_ref(Item_direct_ref(Item_copy_string( ...))).
Item_copy_string does have a pointer to an Item_field in Item_copy::item but does not implement Item::walk method, so we are not
able to set the bitmap for that field. This is the reason why the assert fails.

Fixed by adding the walk method to Item_copy class.
2017-03-14 17:31:29 +05:30
Alexey Botchkov
7c7c0696e7 MDEV-11856 json_search doesn't search for values with double quotes
character (").

        The my_wildcmp function doesn't expect the string parameter to
        have escapements, only the template. So the string
        should be unescaped if necessary.
2017-03-14 15:25:02 +04:00
Marko Mäkelä
a77c2ea78f Post-fix MDEV-12219 Discard temporary undo logs at transaction commit
row_purge_remove_sec_if_poss_leaf(): Add a debug assertion.

row_purge_parse_undo_rec(): Add a debug assertion that the table cannot
be a temporary table. Remove a dead condition.
2017-03-14 12:15:49 +02:00
Sergei Petrunia
20c085a4b7 MariaRocks: disable rocksdb.mysqldump2 also (needs --print-ordering-key) 2017-03-14 12:23:36 +03:00
Sergei Petrunia
1a3065b51a MariaRocks: make rocksdb.issue495 declare it uses partitioning 2017-03-14 12:23:08 +03:00
Vladislav Vaintroub
3990e55fef Windows : Fix packaging when building with VS2017 2017-03-13 23:31:03 +00:00
Vladislav Vaintroub
98be67266d Fix truncation of affected rows and insert id in select_insert::send_ok_packet 2017-03-13 23:13:24 +00:00
Vicențiu Ciorbaru
3eb8bc7408 Make rocksdb not be compiled on x86 architectures 2017-03-14 01:01:11 +02:00
Vicențiu Ciorbaru
06f1f1aa6e Make ELOOP be considered a File Not Found error when it comes from handlerton
Fix symlink-aria && symlink-myisam to account for this possibility.
2017-03-14 00:25:26 +02:00
Sergei Petrunia
57672a85e3 MariaRocks: make partition.test work on any platform 2017-03-13 22:04:07 +00:00
Vladislav Vaintroub
f0a2f4bbb9 Windows : Remove the option for creating anonymous account from the MSI in 10.2 2017-03-13 20:14:28 +00:00
Vladislav Vaintroub
b7ffe9b81e MDEV-11903 : correction - min innodb pagesize is 4K 2017-03-13 19:58:04 +00:00
Vladislav Vaintroub
a8715884a4 MDEV-11903 Windows : Support innodb page sizes in the installer/mysql_install_db.exe
- add PAGESIZE property to the MSI installer
- add combobox  to the MSI UI to select innodb page size
- add new parameter --innodb_page_size for mysql_install_db.exe.
this is passed down to bootstrap and also stored in my.ini.

MSI will call mysql_install_db.exe with --innodb_page_size set to the
PAGESIZE property
2017-03-13 18:39:11 +00:00
Marko Mäkelä
9dc10d5851 Merge 10.0 into 10.1 2017-03-13 19:17:34 +02:00
Oleksandr Byelkin
bfef611a22 MDEV-12244: C API header file contains C++ comment for COM_BINLOG_DUMP_GTID
fixed C++ comment in C code
2017-03-13 18:03:05 +01:00
Marko Mäkelä
13e5c9de80 MDEV-12219 Discard temporary undo logs at transaction commit
Starting with MySQL 5.7, temporary tables in InnoDB are handled
differently from persistent tables. Because temporary tables are
private to a connection, concurrency control and multi-versioning
(MVCC) are not applicable. For performance reasons, purge is
disabled as well. Rollback is supported for temporary tables;
that is why we have the temporary undo logs in the first place.

Because MVCC and purge are disabled for temporary tables, we should
discard all temporary undo logs already at transaction commit,
just like we discard the persistent insert_undo logs. Before this
change, update_undo logs were being preserved.

trx_temp_undo_t: A wrapper for temporary undo logs, comprising
a rollback segment and a single temporary undo log.

trx_rsegs_t::m_noredo: Use trx_temp_undo_t.
(Instead of insert_undo, update_undo, there will be a single undo.)

trx_is_noredo_rseg_updated(), trx_is_rseg_assigned(): Remove.

trx_undo_add_page(): Remove the parameter undo_ptr.
Acquire and release the rollback segment mutex inside the function.

trx_undo_free_last_page(): Remove the parameter trx.

trx_undo_truncate_end(): Remove the parameter trx, and add the
parameter is_temp. Clean up the code a bit.

trx_undo_assign_undo(): Split the parameter undo_ptr into rseg, undo.

trx_undo_commit_cleanup(): Renamed from trx_undo_insert_cleanup().
Replace the parameter undo_ptr with undo.
This will discard the temporary undo or insert_undo log at
commit/rollback.

trx_purge_add_update_undo_to_history(), trx_undo_update_cleanup():
Remove 3 parameters. Always operate on the persistent update_undo.

trx_serialise(): Renamed from trx_serialisation_number_get().

trx_write_serialisation_history(): Simplify the code flow.
If there are no persistent changes, do not update MONITOR_TRX_COMMIT_UNDO.

trx_commit_in_memory(): Simplify the logic, and add assertions.

trx_undo_page_report_modify(): Keep a direct reference to the
persistent update_undo log.

trx_undo_report_row_operation(): Simplify some code.
Always assign TRX_UNDO_INSERT for temporary undo logs.

trx_prepare_low(): Keep only one parameter. Prepare all 3 undo logs.

trx_roll_try_truncate(): Remove the parameter undo_ptr.
Try to truncate all 3 undo logs of the transaction.

trx_roll_pop_top_rec_of_trx_low(): Remove.

trx_roll_pop_top_rec_of_trx(): Remove the redundant parameter
trx->roll_limit. Clear roll_limit when exhausting the undo logs.
Consider all 3 undo logs at once, prioritizing the persistent
undo logs.

row_undo(): Minor cleanup. Let trx_roll_pop_top_rec_of_trx()
reset the trx->roll_limit.
2017-03-13 18:57:17 +02:00
Marko Mäkelä
056ec4ab24 Fix some compilation warnings. 2017-03-13 18:11:36 +02:00