Commit graph

1364 commits

Author SHA1 Message Date
Julius Goryavsky
2e14f2c889 MDEV-28279: Hashicorp: Cannot migrate hexadecimal keys from file key management
This commit fixes a bug in the algorithm for converting hexadecimal
strings to binary key values, which leads to incompatibility with other
plugins and reduces the effective information capacity of the keys.
The new key conversion algorithm is incompatible with tables which
alrady encrypted using a old plugin (plugin version less than or
equalt to the 1.05).
2022-05-09 07:55:41 +02:00
Julius Goryavsky
94841ba656 Hashicorp plugin: typo fixed 2022-05-09 07:55:41 +02:00
Julius Goryavsky
8ae5408cab MDEV-28442: Hashicorp: refactoring to wrap static variables into a class
This commit contains changes to refactor the the Hashicorp plugin code
which hides all variables previously declared as "static" and which are
not user-visible parameters into a special class that contains all the
plugin's dynamic data. This was done primarily to significantly simplify
the code of the initialization and deinitialization functions, which
previously contained a large number of gotos and complex branching
conditions to control memory deallocation.
2022-05-09 07:55:41 +02:00
Julius Goryavsky
e571174e80 MDEV-28291: Hashicorp: Cache variables claim to be dynamic but changes are ignored
This commit fixes an issue with no visible update in caching
option values after changing them dynamically while the server
is running. This issue was related to forgotten copy operations
of new values into dynamic variables. At the same time, internal
variables (responsible for caching) were always updated correctly.
The commit includes a test that checks that the update is now
reflected in the values of dynamic variables.
2022-05-09 07:55:41 +02:00
Julius Goryavsky
0902cfaec8 MDEV-28330: Hashicorp: Key caching doesn't appear to be working
Added test files for mtr that test failed configuration with
--loose-hashicorp-key-management-cache-version-timeout=180000
correctly set.
2022-05-09 07:55:41 +02:00
Julius Goryavsky
3d1f765066 MDEV-28276: Hashicorp: checking that kv storage is created with version 2+
For the plugin to work properly, we need support for key versioning,
and for this, the kv storage in Hashicorp Vault must be created with
version 2 or higher. This commit adds such a check performed during
plugin initialization.

Note: checking for kv storage version during plugin initialization
can be disabled via --hashicorp-key-management-check-kv-version=off
command-line option or via the corresponding option in the server
configuration files.
2022-05-09 07:55:41 +02:00
Julius Goryavsky
1c22a9d8ae MDEV-28277: Checking for mandatory "/v1/" prefix in the URL
According to the Hashicorp Vault API specifications,
the URL to access the keys must include the "/v1/" prefix
at the beginning of the path. This commit adds this parameter
check, as well as a check for the presence of at least one
letter in the hostname inside the URL and in the secret
store name (after "/v1/").
2022-05-09 07:55:41 +02:00
Julius Goryavsky
35989d9cc1 MDEV-28281: Hashicorp: Key ID is not indicated in the log record
This commit adds an indication of the ID of the not found key
(and, when appropriate, also an indication of the version number
of the key) in the log file, making it easier to find errors.
2022-05-09 07:55:41 +02:00
Julius Goryavsky
0c5d8b8730 MDEV-28275: Hashicorp: ASAN heap-use-after-free in get_version()
Passing a string as a parameter by value has been replaced by
passing by reference to avoid using memory after it has been freed.
2022-05-09 07:55:41 +02:00
Julius Goryavsky
1146b713b2 MDEV-19281: Plugin implementation for the Hashicorp Vault KMS
- Authentication is done using the Hashicorp Vault's token
  authentication method;
- If additional client authentication is required, then the
  path to the CA authentication bundle file may be passed
  as a plugin parameter;
- The creation of the keys and their management is carried
  out using the Hashicorp Vault KMS and their tools;
- Key values stored as hexadecimal strings;
- Key values caching is supported.
- Implemented a time-invalidated cache for key values and
  for key version numbers received from the Hashicorp Valult
  server;
- The plugin uses libcurl (https) as an interface to
  the HashiCorp Vault server;
- JSON parsing is performed through the JSON service
  (through the include/mysql/service_json.h);
- HashiCorp Vault 1.2.4 was used for development and testing.
2022-05-09 07:55:41 +02:00
Marko Mäkelä
6cb6ba8b7b Merge 10.8 into 10.9 2022-04-06 13:33:33 +03:00
Marko Mäkelä
b2baeba415 Merge 10.7 into 10.8 2022-04-06 13:28:25 +03:00
Marko Mäkelä
2d8e38bc94 Merge 10.6 into 10.7 2022-04-06 13:00:09 +03:00
Marko Mäkelä
9d94c60f2b Merge 10.5 into 10.6 2022-04-06 12:08:30 +03:00
Marko Mäkelä
cacb61b6be Merge 10.4 into 10.5 2022-04-06 10:06:39 +03:00
Marko Mäkelä
d6d66c6e90 Merge 10.3 into 10.4 2022-04-06 08:59:09 +03:00
Daniel Black
7ae46ced37 Merge branch 10.8 into 10.9
Closes #2084
2022-04-06 14:33:27 +10:00
Daniel Black
7b06bc9a94 Merge branch 10.7 into 10.8
Closes #2083
2022-04-06 14:23:20 +10:00
Alexander Barkov
0ffaf19c53 Adding a "const" qualifier to arguments of create_func(), create_native() etc
The "const" qualifier was obviously forgotten.

This change will also simpily fixing of MDEV-27744.
2022-04-04 09:50:32 +04:00
Sergei Golubchik
d48774e0e0 MDEV-27354 Change maturity of plugins 2022-04-02 17:19:43 +02:00
Rucha Deodhar
3eb1e11d8a MDEV-23479: Add a THD* argument to Item_func_or_sum::fix_length_and_dec()
Fix: Added THD *thd argument in Item_func_or_sum::fix_length_and_dec() and in
fix_length_and_dec() for all derived classes of Item_func_or_sum.
2022-03-30 17:00:17 +05:30
Marko Mäkelä
5c69e93630 Merge 10.7 into 10.8 2022-03-30 09:34:07 +03:00
Marko Mäkelä
a4d753758f Merge 10.6 into 10.7 2022-03-30 08:52:05 +03:00
Marko Mäkelä
b242c3141f Merge 10.5 into 10.6 2022-03-29 16:16:21 +03:00
Marko Mäkelä
d62b0368ca Merge 10.4 into 10.5 2022-03-29 12:59:18 +03:00
Marko Mäkelä
ae6e214fd8 Merge 10.3 into 10.4 2022-03-29 11:13:18 +03:00
Marko Mäkelä
020e7d89eb Merge 10.2 into 10.3 2022-03-29 09:53:15 +03:00
hongdongjian
7af133cc11 MDEV-28177: server_audit; Update the offset of dbName on the aarch64 platform.
On the aarch64 platform, MySQL 5.7.33 cannot install this version of the audit
plugin, but X86_64 can run well。
2022-03-28 13:42:16 +11:00
Marko Mäkelä
9f5a3e5689 Merge 10.7 into 10.8 2022-03-15 18:18:07 +02:00
Marko Mäkelä
de4ec44b4f Fix clang -Wtypedef-redefinition
Include my_global.h before mysql.h so that the latter will not
redefine my_socket and my_ulonglong.

Fixup for commit 77c184df7c
2022-03-15 12:57:39 +02:00
Marko Mäkelä
e8a2a70cf8 Merge 10.7 into 10.8 2022-03-08 10:03:45 +02:00
Marko Mäkelä
af87186c1d Merge 10.6 into 10.7 2022-03-08 09:51:31 +02:00
Vlad Lesin
202316a38f Merge 10.5 into 10.6 2022-03-07 18:42:47 +03:00
Rucha Deodhar
e8e755ea6c MDEV-26230: mysql_upgrade fails to load type_mysql_json due to insufficient
maturity level

Fix: Bumped maturity of the mysql_json plugin to gamma.
2022-03-03 12:37:07 +05:30
Oleksandr Byelkin
4fb2cb1a30 Merge branch '10.7' into 10.8 2022-02-04 14:50:25 +01:00
Oleksandr Byelkin
9ed8deb656 Merge branch '10.6' into 10.7 2022-02-04 14:11:46 +01:00
Oleksandr Byelkin
f5c5f8e41e Merge branch '10.5' into 10.6 2022-02-03 17:01:31 +01:00
Oleksandr Byelkin
cf63eecef4 Merge branch '10.4' into 10.5 2022-02-01 20:33:04 +01:00
Oleksandr Byelkin
a576a1cea5 Merge branch '10.3' into 10.4 2022-01-30 09:46:52 +01:00
Oleksandr Byelkin
41a163ac5c Merge branch '10.2' into 10.3 2022-01-29 15:41:05 +01:00
Alexander Barkov
059a8fd87e MDEV-27668 Assertion `item->type_handler()->is_traditional_scalar_type() || item->type_handler() == type_handler()' failed in Field_inet6::can_optimize_keypart_ref 2022-01-28 22:32:56 +04:00
Alexander Barkov
430d60d1fc MDEV-24487 Error after update to 10.5.8 on CentOS-8: DBD::mysql::st execute failed: Unknown MySQL error
The problem happened because the the new client capability flag
CLIENT_EXTENDED_METADATA was not put into the cache entry key.
So results cached by a new client were sent to the old client (and vica versa)
with a mis-matching metadata, which made the client abort the connection on
an unexpected result set metadata packet format.

The problem was caused by the patch for:
  MDEV-17832 Protocol: extensions for Pluggable types and JSON, GEOMETRY
which forgot to adjust the query cache code.

Fix:

- Adding a new member Query_cache_query_flags::client_extended_metadata,
  so only clients with equal CLIENT_EXTENDED_METADATA flag values can
  reuse results.

- Adding a new column CLIENT_EXTENDED_METADATA into
  INFORMATION_SCHEMA.QUERY_CACHE_INFO (privided by the qc_info plugin).
2022-01-27 15:54:20 +04:00
Alexander Barkov
62e320c86d MDEV-18918 SQL mode EMPTY_STRING_IS_NULL breaks RBR upon CREATE TABLE .. SELECT
The 10.5 version of the patch.

Removing DEFAULT from INFORMATION_SCHEMA columns.
DEFAULT in read-only tables is rather meaningless.
Upgrade should go smoothly.

Also fixes:
 MDEV-20254 Problems with EMPTY_STRING_IS_NULL and I_S tables
2022-01-25 10:31:55 +04:00
Alexander Barkov
da37bfd8d6 MDEV-18918 SQL mode EMPTY_STRING_IS_NULL breaks RBR upon CREATE TABLE .. SELECT
Removing DEFAULT from INFORMATION_SCHEMA columns.
DEFAULT in read-only tables is rather meaningless.
Upgrade should go smoothly.

Also fixes:
 MDEV-20254 Problems with EMPTY_STRING_IS_NULL and I_S tables
2022-01-25 10:31:03 +04:00
Alexander Barkov
e4b302e436 MDEV-27018 IF and COALESCE lose "json" property
Hybrid functions (IF, COALESCE, etc) did not preserve the JSON property
from their arguments. The same problem was repeatable for single row subselects.

The problem happened because the method Item::is_json_type() was inconsistently
implemented across the Item hierarchy. For example, Item_hybrid_func
and Item_singlerow_subselect did not override is_json_type().

Solution:

- Removing Item::is_json_type()

- Implementing specific JSON type handlers:
  Type_handler_string_json
  Type_handler_varchar_json
  Type_handler_tiny_blob_json
  Type_handler_blob_json
  Type_handler_medium_blob_json
  Type_handler_long_blob_json

- Reusing the existing data type infrastructure to pass JSON
  type handlers across all item types, including classes Item_hybrid_func
  and Item_singlerow_subselect. Note, these two classes themselves do not
  need any changes!

- Extending the data type infrastructure so data types can inherit
  their properties (e.g. aggregation rules) from their base data types.
  E.g. VARCHAR/JSON acts as VARCHAR, LONGTEXT/JSON acts as LONGTEXT
  when mixed to a non-JSON data type. This is done by:
    - adding virtual method Type_handler::type_handler_base()
    - adding a helper class Type_handler_pair
    - refactoring Type_handler_hybrid_field_type methods
      aggregate_for_result(), aggregate_for_min_max(),
      aggregate_for_num_op() to use Type_handler_pair.

This change also fixes:

  MDEV-27361 Hybrid functions with JSON arguments do not send format metadata

Also, adding mtr tests for JSON replication. It was not covered yet.
And the current patch changes the replication code slightly.
2022-01-21 19:28:48 +04:00
Monty
9d4c0a6cab Fixed compiler error in auth_pam plugin
Code copied from 10.6
2022-01-20 15:14:59 +02:00
Marko Mäkelä
1abc476f0b Merge 10.5 into 10.6 2022-01-18 12:59:50 +02:00
Sergei Golubchik
745aa8bee7 MDEV-26230 mysql_upgrade fails to load type_mysql_json due to insufficient maturity level
bump maturity to beta
2022-01-17 18:16:27 +01:00
Marko Mäkelä
daf4fa5238 Merge 10.7 into 10.8 2022-01-04 10:30:45 +02:00
Alexander Barkov
756568f26c MDEV-22256 Assertion `length == pack_length()' failed in Field_timestamp_with_dec::sort_string
Adding 10.5 specific tests only. No code change.

The patches for MDEV-21580 and MDEV-22715 fixed this problem.
2021-12-29 22:58:52 +04:00