post-review fixes:
* move all ssl implementation related ifdefs/defines to one file
(ssl_compat.h)
* work around OpenSSL-1.1 desire to malloc every EVP context by
run-time checking that context allocated on the stack is big enough
(openssl.c)
* use newer version of the AWS SDK for OpenSSL 1.1
* use get_dh2048() function as generated by openssl 1.1
(viosslfactories.c)
Also, include fixes by Vladislav Vaintroub to the
aws_key_management plugin. The AWS C++ SDK specifically depends on
OPENSSL_LIBRARIES, not generic SSL_LIBRARIES (such as YaSSL).
Do not exporting mysqld entry points directly.
This is needed for mariabackup, to load encryption plugins on Windows.
All plugins are "pure" by default. To mark plugin "impure"
it should use RECOMPILE_FOR_EMBEDDED or STORAGE_ENGINE keyword.
Use older version of the SDK generally, because the newer ones break
with older cmake.
On Macs, use newer version, to fix mac specific the build error.
Update AWS SDK version from 1.0.8 to 1.0.100
Commit b64910ce27 (MDEV-12453)
enabled AWS_SDK to build correctly on buildbot.
Travis still had build faults like below despite many common elements
between the builds;
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstring:79:9: error: no member named 'strcoll' in the global namespace; did you mean 'strtoll'?
[ 24%] Building CXX object storage/rocksdb/CMakeFiles/rocksdblib.dir/rocksdb/db/internal_stats.cc.o
using ::strcoll;
~~^
/usr/include/stdlib.h:169:3: note: 'strtoll' declared here
strtoll(const char *__str, char **__endptr, int __base);
^
use CMAKE_CXX_STANDARD to set C++11 flags with CMake 3.1+ (apples flags are somehow different from standard clang)
port htonbe16/32/64 macros for rocksdb
use reinterpret_cast<size_t> to cast macOS's pthread_t (pointer type) to size_t , for rocksdb
* define MYSQL_DYNAMIC_PLUGIN only for server plugins
* don't typedef my_bool in mysql.h if plugin.h has already done it
* fix the include guard in plugin.h
Also, implement MDEV-11027 a little differently from 5.5 and 10.0:
recv_apply_hashed_log_recs(): Change the return type back to void
(DB_SUCCESS was always returned).
Report progress also via systemd using sd_notifyf().
Also, implement MDEV-11027 a little differently from 5.5:
recv_sys_t::report(ib_time_t): Determine whether progress should
be reported.
recv_apply_hashed_log_recs(): Rename the parameter to last_batch.
Implementation of MDEV-7660 introduced unwanted incompatible change:
modifications under LOCK TABLES with autocommit enabled are rolled back on
disconnect. Previously everything was committed, because LOCK TABLES didn't
adjust autocommit setting.
This patch restores original behavior by reverting some changes done in
MDEV-7660:
- sql/sql_parse.cc: do not reset autocommit on LOCK TABLES
- sql/sql_base.cc: do not set autocommit on UNLOCK TABLES
- test cases: main.lock_tables_lost_commit, main.partition_explicit_prune,
rpl.rpl_switch_stm_row_mixed, tokudb.nested_txn_implicit_commit,
tokudb_bugs.db806
But it makes InnoDB tables under LOCK TABLES ... READ [LOCAL] not protected
against DML. To restore protection some changes from WL#6671 were merged,
specifically MDL_SHARED_READ_ONLY and test cases.
WL#6671 merge highlights:
- Not all tests merged.
- In MySQL LOCK TABLES ... READ acquires MDL_SHARED_READ_ONLY for all engines,
in MariaDB MDL_SHARED_READ is always acquired first and then upgraded to
MDL_SHARED_READ_ONLY for InnoDB only.
- The above allows us to omit MDL_SHARED_WRITE_LOW_PRIO implementation in
MariaDB, which is rather useless with InnoDB. In MySQL it is needed to
preserve locking behavior between low priority writes and LOCK TABLES ... READ
for non-InnoDB engines (covered by sys_vars.sql_low_priority_updates_func).
- Omitted HA_NO_READ_LOCAL_LOCK, we rely on lock_count() instead.
- Omitted "piglets": in MariaDB stream of DML against InnoDB table may lead to
concurrent LOCK TABLES ... READ starvation.
- HANDLER ... OPEN acquires MDL_SHARED_READ instead of MDL_SHARED in MariaDB.
- Omitted SNRW->X MDL lock upgrade for IMPORT/DISCARD TABLESPAECE under LOCK
TABLES.
- Omitted strong locks for views, triggers and SP under LOCK TABLES.
- Omitted IX schema lock for LOCK TABLES READ.
- Omitted deadlock weight juggling for LOCK TABLES.
Full WL#6671 merge status:
- innodb.innodb-lock: fully merged
- main.alter_table: not merged due to different HANDLER solution
- main.debug_sync: fully merged
- main.handler_innodb: not merged due to different HANDLER solution
- main.handler_myisam: not merged due to different HANDLER solution
- main.innodb_mysql_lock: fully merged
- main.insert_notembedded: fully merged
- main.lock: not merged (due to no strong locks for views)
- main.lock_multi: not merged
- main.lock_sync: fully merged (partially in MDEV-7660)
- main.mdl_sync: not merged
- main.partition_debug_sync: not merged due to different HANDLER solution
- main.status: fully merged
- main.view: fully merged
- perfschema.mdl_func: not merged (no such test in MariaDB)
- perfschema.table_aggregate_global_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_global_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_global_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_global_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_4u_3t: not merged (didn't fail in MariaDB)
- sys_vars.sql_low_priority_updates_func: not merged
- include/thr_rwlock.h: not merged, rw_pr_lock_assert_write_owner and
rw_pr_lock_assert_not_write_owner are macros in MariaDB
- sql/handler.h: not merged (HA_NO_READ_LOCAL_LOCK)
- sql/mdl.cc: partially merged (MDL_SHARED_READ_ONLY only)
- sql/mdl.h: partially merged (MDL_SHARED_READ_ONLY only)
- sql/lock.cc: fully merged
- sql/sp_head.cc: not merged
- sql/sp_head.h: not merged
- sql/sql_base.cc: partially merged (MDL_SHARED_READ_ONLY only)
- sql/sql_base.h: not merged
- sql/sql_class.cc: fully merged
- sql/sql_class.h: fully merged
- sql/sql_handler.cc: merged partially (different solution in MariaDB)
- sql/sql_parse.cc: partially merged, mostly omitted low priority write part
- sql/sql_reload.cc: not merged comment change
- sql/sql_table.cc: not merged SNRW->X upgrade for IMPORT/DISCARD TABLESPACE
- sql/sql_view.cc: not merged
- sql/sql_yacc.yy: not merged (MDL_SHARED_WRITE_LOW_PRIO, MDL_SHARED_READ_ONLY)
- sql/table.cc: not merged (MDL_SHARED_WRITE_LOW_PRIO)
- sql/table.h: not merged (MDL_SHARED_WRITE_LOW_PRIO)
- sql/trigger.cc: not merged
- storage/innobase/handler/ha_innodb.cc: merged store_lock()/lock_count()
changes (in MDEV-7660), didn't merge HA_NO_READ_LOCAL_LOCK
- storage/innobase/handler/ha_innodb.h: fully merged in MDEV-7660
- storage/myisammrg/ha_myisammrg.cc: not merged comment change
- storage/perfschema/table_helper.cc: not merged (no MDL support in MariaDB PFS)
- unittest/gunit/mdl-t.cc: not merged
- unittest/gunit/mdl_sync-t.cc: not merged
MariaDB specific changes:
- handler.heap: different HANDLER solution, MDEV-7660
- handler.innodb: different HANDLER solution, MDEV-7660
- handler.interface: different HANDLER solution, MDEV-7660
- handler.myisam: different HANDLER solution, MDEV-7660
- main.mdl_sync: MDEV-7660 specific changes
- main.partition_debug_sync: removed test due to different HANDLER solution,
MDEV-7660
- main.truncate_coverage: removed test due to different HANDLER solution,
MDEV-7660
- mysql-test/include/mtr_warnings.sql: additional cleanup, MDEV-7660
- mysql-test/lib/v1/mtr_report.pl: additional cleanup, MDEV-7660
- plugin/metadata_lock_info/metadata_lock_info.cc: not in MySQL
- sql/sql_handler.cc: MariaDB specific fix for mysql_ha_read(), MDEV-7660
MySQL has a bug failing to handle
MYSQL_THDVAR_STR(... PLUGIN_VAR_NOSYSVAR |
PLUGIN_VAR_NOCMDOPT | PLUGIN_VAR_MEMALLOC)
so fall back to just PLUGIN_VAR_READONLY | PLUGIN_VAR_MEMALLOC
whem MySQL started.
Building AWS C++ SDK as external project needs CMake 2.8.12
Centos7 out of the box has 2.8.11, thus the build fails.
Fixed check for required CMake version.
Clean-up nolock.h: it doesn't serve any purpose anymore. Appropriate code moved
to x86-gcc.h and my_atomic.h.
If gcc sync bultins were detected, we want to make use of them independently of
__GNUC__ definition. E.g. XLC simulates those, but doesn't define __GNUC__.
HS/Spider: According to AIX manual alloca() returns char*, which cannot be
casted to any type with static_cast. Use explicit cast instead.
MDL: Removed namemangling pragma, which didn't let MariaDB build with XLC.
WSREP: _int64 seem to be conflicting name with XLC, replaced with _integer64.
CONNECT: RTLD_NOLOAD is GNU extention. Removed rather meaningless check if
library is loaded. Multiple dlopen()'s of the same library are permitted,
and it never gets closed anyway. Except for error, which was a bug: it may
close library, which can still be referenced by other subsystems.
InnoDB: __ppc_get_timebase() is GNU extention. Only use it when __GLIBC__ is
defined.
Based on contribution by flynn1973.
AWS C++ SDK always defaults region to us-east-1 for clientConfiguration
(ignoring config file or env.variable)
This patch introduces a plugin variable 'region' to make it usable
for master keys created in regions other than 'us-east-1'.
On OS X with clang 7.3.0 running any test as embedded crashes. As
libfeedback was not compiled with embedded, we had 2 notions of THD,
with different number of members. When calling inline functions defined in
sql_class.h, the functions were defined for 1 THD but called on an
object that is different. This resulted in erroneous data being returned
and shortly after, a crash within the alloc_query() function.
Recompile the feedback plugin for the embedded server specifically to
not have such symbol conflicts.
- Library path's are different now
- New dependency on Linux libuuid
- Add calls for SDK Initialization/shutdown
- Also add request_timeout parameter, default SDK HTTPs timeout appears
to be too short in my tests
metadata_lock_info plugin called MDL_context::find_ticket() to obtain lock
duration, which in turn iterates foreign thread private lists. These lists
can be updated by owner thread without protection.
Fixed by iterating threads (instead of MDL locks and tickets) and obtaining
data through APC.
Also fixed mdl_iterate_lock() to initialize iterator under prlock protection.
and
MDEV-10250 InnoDB: Error: File (unknown): 'close' returned OS error 209. Cannot continue operation"
after a failed connect() feedback plugin was continuing with the
file descriptor, trying to send the data (which failed) and
closing it at the end. Even though this fd might've been reused for
something else already.
