Commit graph

430 commits

Author SHA1 Message Date
Marko Mäkelä
15139964d5 Merge 10.5 into 10.6 2021-09-11 17:55:27 +03:00
Vicențiu Ciorbaru
8fe927e6de Expand performance_schema tables definitions with column comments
Cover all columns that did not have comments. Adjust docs based off of
MariaDB implementation.
2021-09-10 17:16:50 +03:00
Haidong Ji
cc71dc0b61 MDEV-25325 built-in documentation for performance_schema tables
Improve documentation of performance_schema tables by appending COLUMN
comments to tables. Additionally improve test coverage and update corresponding
tests.

This is part of the patch covering newer columns and tables in 10.5.
2021-09-10 17:16:40 +03:00
Vicențiu Ciorbaru
7c33ecb665 Merge remote-tracking branch 'upstream/10.4' into 10.5 2021-09-10 17:16:18 +03:00
Vicențiu Ciorbaru
de7e027d5e Merge remote-tracking branch 'upstream/10.3' into 10.4 2021-09-09 09:23:35 +03:00
Vicențiu Ciorbaru
b85b8348e7 Merge branch '10.2' into 10.3 2021-09-07 16:32:35 +03:00
Haidong Ji
528abc749e MDEV-25325 built-in documentation for performance_schema tables
Improve documentation of performance_schema tables by appending COLUMN
comments to tables. Additionally improve test coverage and update corresponding
tests.
2021-09-07 08:45:19 +03:00
Marko Mäkelä
e94172c2a0 Merge 10.5 into 10.6 2021-08-31 11:00:41 +03:00
Marko Mäkelä
e62120cec7 Merge 10.4 into 10.5 2021-08-31 10:04:56 +03:00
Marko Mäkelä
0464761126 Merge 10.3 into 10.4 2021-08-31 09:22:21 +03:00
Marko Mäkelä
e835cc851e Merge 10.2 into 10.3 2021-08-31 08:36:59 +03:00
Marko Mäkelä
fda704c82c Fix GCC 11 -Wmaybe-uninitialized for PLUGIN_PERFSCHEMA
init_mutex_v1_t: Stop lying that the mutex parameter is const.
GCC 11.2.0 assumes that it is and could complain about any mysql_mutex_t
being uninitialized even after mysql_mutex_init() as long as
PLUGIN_PERFSCHEMA is enabled.

init_rwlock_v1_t, init_cond_v1_t: Remove untruthful const qualifiers.

Note: init_socket_v1_t is expecting that the socket fd has already
been created before PSI_SOCKET_CALL(init_socket), and therefore that
parameter really is being treated as a pointer to const.
2021-08-30 11:52:59 +03:00
Marko Mäkelä
f3fcf5f45c Merge 10.5 to 10.6 2021-08-19 12:25:00 +03:00
Marko Mäkelä
4a25957274 Merge 10.4 into 10.5 2021-08-18 18:22:35 +03:00
Marko Mäkelä
f84e28c119 Merge 10.3 into 10.4 2021-08-18 16:51:52 +03:00
Marko Mäkelä
cd65845a0e Merge 10.2 into 10.3
MDEV-18734 FIXME: vcol.partition triggers ASAN heap-use-after-free
2021-08-18 12:26:58 +03:00
Sergei Golubchik
175c9fe1d5 cleanup: specifying plugin dependencies in CMakeLists.txt
1. rename option DEPENDENCIES in MYSQL_ADD_PLUGIN() to DEPENDS
   to be consistent with other cmake commands and macros

2. use this DEPENDS option in plugins

3. add dependencies to the plugin embedded target too

4. plugins don't need to add GenError dependency explicitly,
   all plugins depend on it automatically
2021-08-03 10:10:00 +02:00
Vladislav Vaintroub
e7f4daf88c merge 10.5 to 10.6 2021-07-16 22:12:09 +02:00
Oleksandr Byelkin
a7d880f0b0 MDEV-21916: COM_STMT_BULK_EXECUTE with RETURNING insert wrong values
The problem is that array binding uses net buffer to read parameters for each
execution while each execiting with RETURNING write in the same buffer.

Solution is to allocate new net buffer to avoid changing buffer we are reading
from.
2021-07-15 16:28:13 +02:00
Marko Mäkelä
a722ee88f3 Merge 10.5 into 10.6 2021-06-01 11:39:38 +03:00
Daniel Black
0d44792a83 perfschema: native type for my_thread_os_id_t
Though these will all get case to unsigned long
long where it is populated into the perfschema's BIGINT
type.

Use uintptr_t for NetBSD per Nia Alarie's original .
2021-06-01 13:51:39 +10:00
Daniel Black
90adf2aa59 perfschema: use glibc gettid if available 2021-06-01 13:51:39 +10:00
nia
68eac8a3ad my_thread: Use unsigned long long for storing pthread IDs
This is a fix for operating systems that have pthread_t defined
as a pointer and use the default pthread_self() mechanism for
identifying threads. More specifically, this is a build fix
for NetBSD.

Any changes I submit are freely available under the new BSD
license.

Signed-off-by: Nia Alarie <nia@NetBSD.org>
2021-06-01 13:51:39 +10:00
Sujatha
fe9450676f MDEV-25502: rpl.rpl_perfschema_applier_status_by_worker failed in bb with: Test assertion failed
Problem:
=======
Test fails with 3 different symptoms
connection slave;
Assertion text: 'Last_Seen_Transaction should show .'
Assertion condition: '"0-1-1" = ""'
Assertion condition, interpolated: '"0-1-1" = ""'
Assertion result: '0'

connection slave;
Assertion text: 'Value returned by SSS and PS table for Last_Error_Number
                 should be same.'
Assertion condition: '"1146" = "0"'
Assertion condition, interpolated: '"1146" = "0"'
Assertion result: '0'

connection slave;
Assertion text: 'Value returned by PS table for worker_idle_time should be
                >= 1'
Assertion condition: '"0" >= "1"'
Assertion condition, interpolated: '"0" >= "1"'
Assertion result: '0'

Fix1:
====
Performance schema table's Last_Seen_Transaction is compared with 'SELECT
gtid_slave_pos'. Since DDLs are not transactional changes to user table and
gtid_slave_pos table are not guaranteed to be synchronous. To fix the
issue Gtid_IO_Pos value from SHOW SLAVE STATUS command will be used to
verify the correctness of Performance schema specific
Last_Seen_Transaction.

Fix2:
====
On error worker thread information is stored as part of backup pool. Access
to this backup pool should be protected by 'LOCK_rpl_thread_pool' mutex so
that simultaneous START SLAVE cannot destroy the backup pool, while it is
being queried by performance schema.

Fix3:
====
When a worker is waiting for events if performance schema table is queried,
at present it just returns the difference between current_time and
start_time.  This is incorrect. It should be worker_idle_time +
(current_time - start_time).

For example a worker thread was idle for 10 seconds and then it got events
to process. Upon completion it goes to idle state, now if the pfs table is
queried it should return current_idle time  + worker_idle_time.
2021-05-13 10:34:32 +05:30
Marko Mäkelä
916b237b3f Merge 10.5 into 10.6 2021-05-07 15:00:27 +03:00
Nikita Malyavin
3f55c56951 Merge branch bb-10.4-release into bb-10.5-release 2021-05-05 23:57:11 +03:00
Nikita Malyavin
509e4990af Merge branch bb-10.3-release into bb-10.4-release 2021-05-05 23:03:01 +03:00
Nikita Malyavin
a8a925dd22 Merge branch bb-10.2-release into bb-10.3-release 2021-05-04 14:49:31 +03:00
Dmitry Shulga
e788738e18 MDEV-25543: Building failure on MacOS in case MariadDB server is compiled with XCode 12.5
Attempt to build MariaDB server on MacOS could result in
compilation errors like the following one:

  In file included from server-10.2/storage/perfschema/cursor_by_account.cc:28:
  In file included from server-10.2/include/my_global.h:287:
  In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/math.h:309:
  In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/type_traits:418:
    server-10.2/version:1:1: error: expected unqualified-id

    MYSQL_VERSION_MAJOR=10
    ^
  server-10.2/build.dir/include/my_config.h:529:29: note: expanded from macro 'MYSQL_VERSION_MAJOR'

This kind of compiler errors occur by the reson that compiler's system headers
contain the directive '#include <version>' and a compiler is invoked
with -I${CMAKE_SOURCE_DIR}.

The MariaDB source code root directory contains the file VERSION that is handled
by the compiler during processing the directive #include <version>
since file names on MacOS are case insensetive, so version and VERSION is treated as
the same file name.

To fix the issue the source code root directory should be removed from a list
of directories used by the compiler for include search path.
2021-04-28 23:02:06 +07:00
Marko Mäkelä
54c460ace6 Merge 10.5 into 10.6 2021-04-22 08:43:03 +03:00
Marko Mäkelä
9abc6fd73f Cleanup: constexpr PFS_table_context::m_word_size 2021-04-21 12:44:23 +03:00
Marko Mäkelä
b59d07624c WITH_UBSAN: shift is too large for 32-bit int
As suggested by Vladislav Vaintroub, we must shift a constant of
unsigned long, not int.
2021-04-21 12:34:54 +03:00
Sujatha
410832313e MDEV-16437: merge 5.7 P_S replication instrumentation and tables
Merge 'replication_applier_status' table.

This table captures SQL_THREAD status.
2021-04-16 09:11:48 +05:30
Sujatha
767648cc2b MDEV-16437: merge 5.7 P_S replication instrumentation and tables
Merge 'replication_applier_configuration' table.

This table captures SQL_THREAD configuration parameters.
2021-04-16 09:05:39 +05:30
Sujatha
70642871bc MDEV-16437: merge 5.7 P_S replication instrumentation and tables
Merge 'replication_applier_status_by_coordinator' table.

This table captures SQL_THREAD status in case of both single threaded and
multi threaded slave configuration. When multi_source replication is enabled
this table will display each source specific SQL_THREAD status.

Added new columns for:
 - LAST_SEEN_TRANSACTION
 - LAST_TRANS_RETRY_COUNT
2021-04-16 09:02:00 +05:30
Sujatha
2674365c8e MDEV-16437: merge 5.7 P_S replication instrumentation and tables
Merge 'replication_connection_configuration' table.

Replaced following column:
  - AUTO_POSITION with USING_GTID
Added new columns for:
  - IGNORE_SERVER_IDS
  - DO_DOMAIN_IDS
  - IGNORE_SERVER_IDS
Removed following columns as they are not part of mariadb replication
connection configuration:
  - NETWORK_INTERFACE
  - TLS_VERSION

@sql/mysqld.cc
  Changed "master-retry-count" default value to 100000.
2021-04-16 08:54:19 +05:30
Sujatha
f9bd7f2012 MDEV-20220: Merge 5.7 P_S replication table 'replication_applier_status_by_worker
Step 3:
======

Preserve worker pool information on either STOP SLAVE/Error.  In case STOP
SLAVE is executed worker threads will be gone, hence worker threads will be
unavailable. Querying the table at this stage will give empty rows. To
address this case when worker threads are about to stop, due to an error or
forced stop, create a backup pool and preserve the data which is relevant to
populate performance schema table. Clear the backup pool upon slave start.
2021-04-08 17:19:51 +05:30
Sujatha
036ee61246 MDEV-20220: Merge 5.7 P_S replication table 'replication_applier_status_by_worker
Step2:
=====
Add two extra columns mentioned below.
---------------------------------------------------------------------------
|Column Name:           |        Description:                             |
|-------------------------------------------------------------------------|
|                       |                                                 |
|WORKER_IDLE_TIME       | Total idle time in seconds that the worker      |
|                       | thread has spent waiting for work from          |
|                       | co-ordinator thread                             |
|                       |                                                 |
|LAST_TRANS_RETRY_COUNT | Total number of retries attempted by last       |
|                       | transaction                                     |
---------------------------------------------------------------------------
2021-04-08 17:19:51 +05:30
Sujatha
94f1d0f84d MDEV-20220: Merge 5.7 P_S replication table 'replication_applier_status_by_worker
Step1:
=====
Backport 'replication_applier_status_by_worker' from upstream.

Iterate through rpl_parallel_thread_pool and display slave worker thread
specific information as part of 'replication_applier_status_by_worker'
table.

---------------------------------------------------------------------------
|Column Name:           |        Description:                             |
|-------------------------------------------------------------------------|
|                       |                                                 |
|CHANNEL_NAME           | Name of replication channel through which the   |
|                       | transaction is received.                        |
|                       |                                                 |
|THREAD_ID              | Thread_Id as displayed in 'performance_schema.  |
|                       | threads' table for thread with name             |
|                       | 'thread/sql/rpl_parallel_thread'                |
|                       |                                                 |
|                       | THREAD_ID will be NULL when worker threads are  |
|                       | stopped due to an error/force stop              |
|                       |                                                 |
|SERVICE_STATE          | Thread is running or not                        |
|                       |                                                 |
|LAST_SEEN_TRANSACTION  | Last GTID executed by worker                    |
|                       |                                                 |
|LAST_ERROR_NUMBER      | Last Error that occured on a particular worker  |
|                       |                                                 |
|LAST_ERROR_MESSAGE     | Last error specific message                     |
|                       |                                                 |
|LAST_ERROR_TIMESTAMP   | Time stamp of last error                        |
|                       |                                                 |
---------------------------------------------------------------------------

CHANNEL_NAME will be empty when the worker has not processed any
transaction. Channel_name points to valid source channel_name when it is
processing a transaction/event group.
2021-04-08 17:19:51 +05:30
Daniel Black
86d60fc9e7 Merge remote-tracking branch 'origin/10.4' into 10.5 2021-02-26 13:23:13 +11:00
Daniel Black
2628fa2dba MDEV-20857: perf schema conflict name filename_hash
filename_hash is a function from libiberty.a from the system
but also an expored name in the perf schema static library.

We'll use a different name.
2021-02-24 19:45:58 +11:00
Sergei Golubchik
25d9d2e37f Merge branch 'bb-10.4-release' into bb-10.5-release 2021-02-15 16:43:15 +01:00
Sergei Golubchik
00a313ecf3 Merge branch 'bb-10.3-release' into bb-10.4-release
Note, the fix for "MDEV-23328 Server hang due to Galera lock conflict resolution"
was null-merged. 10.4 version of the fix is coming up separately
2021-02-12 17:44:22 +01:00
Sergei Golubchik
60ea09eae6 Merge branch '10.2' into 10.3 2021-02-01 13:49:33 +01:00
Nikita Malyavin
21809f9a45 MDEV-17556 Assertion `bitmap_is_set_all(&table->s->all_set)' failed
The assertion failed in handler::ha_reset upon SELECT under
READ UNCOMMITTED from table with index on virtual column.

This was the debug-only failure, though the problem is mush wider:
* MY_BITMAP is a structure containing my_bitmap_map, the latter is a raw
 bitmap.
* read_set, write_set and vcol_set of TABLE are the pointers to MY_BITMAP
* The rest of MY_BITMAPs are stored in TABLE and TABLE_SHARE
* The pointers to the stored MY_BITMAPs, like orig_read_set etc, and
 sometimes all_set and tmp_set, are assigned to the pointers.
* Sometimes tmp_use_all_columns is used to substitute the raw bitmap
 directly with all_set.bitmap
* Sometimes even bitmaps are directly modified, like in
TABLE::update_virtual_field(): bitmap_clear_all(&tmp_set) is called.

The last three bullets in the list, when used together (which is mostly
always) make the program flow cumbersome and impossible to follow,
notwithstanding the errors they cause, like this MDEV-17556, where tmp_set
pointer was assigned to read_set, write_set and vcol_set, then its bitmap
was substituted with all_set.bitmap by dbug_tmp_use_all_columns() call,
and then bitmap_clear_all(&tmp_set) was applied to all this.

To untangle this knot, the rule should be applied:
* Never substitute bitmaps! This patch is about this.
 orig_*, all_set bitmaps are never substituted already.

This patch changes the following function prototypes:
* tmp_use_all_columns, dbug_tmp_use_all_columns
 to accept MY_BITMAP** and to return MY_BITMAP * instead of my_bitmap_map*
* tmp_restore_column_map, dbug_tmp_restore_column_maps to accept
 MY_BITMAP* instead of my_bitmap_map*

These functions now will substitute read_set/write_set/vcol_set directly,
and won't touch underlying bitmaps.
2021-01-27 00:50:55 +10:00
Sergei Golubchik
a216672dab MDEV-16341 Wrong length for USER columns in performance_schema tables
use USERNAME_CHAR_LENGTH and HOSTNAME_LENGTH for perfschema
USER and HOST columns
2021-01-11 21:54:48 +01:00
Nikita Malyavin
e25623e78a MDEV-17556 Assertion `bitmap_is_set_all(&table->s->all_set)' failed
The assertion failed in handler::ha_reset upon SELECT under
READ UNCOMMITTED from table with index on virtual column.

This was the debug-only failure, though the problem is mush wider:
* MY_BITMAP is a structure containing my_bitmap_map, the latter is a raw
 bitmap.
* read_set, write_set and vcol_set of TABLE are the pointers to MY_BITMAP
* The rest of MY_BITMAPs are stored in TABLE and TABLE_SHARE
* The pointers to the stored MY_BITMAPs, like orig_read_set etc, and
 sometimes all_set and tmp_set, are assigned to the pointers.
* Sometimes tmp_use_all_columns is used to substitute the raw bitmap
 directly with all_set.bitmap
* Sometimes even bitmaps are directly modified, like in
TABLE::update_virtual_field(): bitmap_clear_all(&tmp_set) is called.

The last three bullets in the list, when used together (which is mostly
always) make the program flow cumbersome and impossible to follow,
notwithstanding the errors they cause, like this MDEV-17556, where tmp_set
pointer was assigned to read_set, write_set and vcol_set, then its bitmap
was substituted with all_set.bitmap by dbug_tmp_use_all_columns() call,
and then bitmap_clear_all(&tmp_set) was applied to all this.

To untangle this knot, the rule should be applied:
* Never substitute bitmaps! This patch is about this.
 orig_*, all_set bitmaps are never substituted already.

This patch changes the following function prototypes:
* tmp_use_all_columns, dbug_tmp_use_all_columns
 to accept MY_BITMAP** and to return MY_BITMAP * instead of my_bitmap_map*
* tmp_restore_column_map, dbug_tmp_restore_column_maps to accept
 MY_BITMAP* instead of my_bitmap_map*

These functions now will substitute read_set/write_set/vcol_set directly,
and won't touch underlying bitmaps.
2021-01-08 16:04:29 +10:00
Etienne Guesnet
2f5d372444 Add build on AIX 2020-12-16 08:07:04 +11:00
Marko Mäkelä
133b4b46fe Merge 10.4 into 10.5 2020-11-03 16:24:47 +02:00
Marko Mäkelä
533a13af06 Merge 10.3 into 10.4 2020-11-03 14:49:17 +02:00