Commit graph

7350 commits

Author SHA1 Message Date
Marko Mäkelä
d8303c3ee7 Merge 10.3 into 10.4 2019-04-08 08:22:34 +03:00
Marko Mäkelä
cc492bfd4f Merge 10.2 into 10.3 2019-04-07 11:49:50 +03:00
Igor Babaev
b4a7bde76c MDEV-19112 WITH clause does not work with information_schema as default database
With INFORMATION_SCHEMA set as the default database the check that a table
referred in the processed query is defined in INORMATION_SCHEMA must
be postponed until all CTE names can be identified.
2019-04-05 15:00:42 -07:00
Marko Mäkelä
b896f60a73 Fix -Wformat and -Wnonnull-compare for WSREP 2019-04-03 09:20:21 +03:00
Marko Mäkelä
5c3ff5cb93 Merge 10.3 into 10.4 2019-04-02 11:04:54 +03:00
Marko Mäkelä
0bc4260226 Merge 10.3 into 10.4 2019-03-26 17:43:59 +02:00
Marko Mäkelä
ffc69dbd05 Merge 10.2 into 10.3 2019-03-26 15:03:37 +02:00
Marko Mäkelä
ac4934535d Merge 10.1 into 10.2 2019-03-26 14:59:32 +02:00
Marko Mäkelä
f2c1c9590c Fix cmake -DENABLED_PROFILING=OFF 2019-03-26 14:57:53 +02:00
Marko Mäkelä
226ca250ed Merge 10.1 into 10.2 2019-03-26 14:17:19 +02:00
Marko Mäkelä
42fd537244 Merge 10.0 into 10.1 2019-03-26 13:51:40 +02:00
Marko Mäkelä
8b480df63e Merge 10.3 into 10.4 2019-03-25 17:18:15 +02:00
Eugene Kosov
67b601c503 remove unused argument 2019-03-22 22:57:16 +04:00
Eugene Kosov
a523444eda remove dead code 2019-03-22 22:30:36 +04:00
Alexander Barkov
f704361cd6 Backporting slow log simulation logic details from 10.2 to 10.1
Simulate slow statements only for COM_QUERY and COM_STMT_EXECUTE commands,
to exclude mysqld_stmt_prepare() and mysqld_stmt_close() entries from the log,
as they are not relevant for log_slow_debug.test. This simplifies the test.
2019-03-22 15:56:45 +04:00
Chris Calender
9d2d80aace Update sql_parse.cc
Corrected one more "refering" typo.
2019-03-22 00:40:32 +04:00
Marko Mäkelä
514b305dfb Merge 10.3 into 10.4
The MDEV-17262 commit 26432e49d3
was skipped. In Galera 4, the implementation would seem to require
changes to the streaming replication.

In the tests archive.rnd_pos main.profiling, disable_ps_protocol
for SHOW STATUS and SHOW PROFILE commands until MDEV-18974
has been fixed.
2019-03-20 10:41:32 +02:00
Sergei Golubchik
b64fde8f38 Merge branch '10.2' into 10.3 2019-03-17 13:06:41 +01:00
Sergei Golubchik
1f020299f8 post-merge: --ps-protocol fixes
Includes:

MDEV-17302 Add support for ALTER USER command in prepared statement

and

MDEV-17673 main.cte_recursive fails in bb-10.4-ps branch in --ps

  Set correct SELECT_LEX linkage for recursive CTEs.
  Do not delegate this job to TABLE_LIST::set_as_with_table,
  because it is only run on prepare, while With_element::move_anchors_ahead
  is run both on prepare and execute (fix by Igor)
2019-03-15 21:00:56 +01:00
Sergei Golubchik
0508d327ae Merge branch '10.1' into 10.2 2019-03-15 21:00:41 +01:00
Sergei Golubchik
3d2d060b62 fix gcc 8 compiler warnings
There were two newly enabled warnings:
1. cast for a function pointers. Affected sql_analyse.h, mi_write.c
   and ma_write.cc, mf_iocache-t.cc, mysqlbinlog.cc, encryption.cc, etc

2. memcpy/memset of nontrivial structures. Fixed as:
* the warning disabled for InnoDB
* TABLE, TABLE_SHARE, and TABLE_LIST got a new method reset() which
  does the bzero(), which is safe for these classes, but any other
  bzero() will still cause a warning
* Table_scope_and_contents_source_st uses `TABLE_LIST *` (trivial)
  instead of `SQL_I_List<TABLE_LIST>` (not trivial) so it's safe to
  bzero now.
* added casts in debug_sync.cc and sql_select.cc (for JOIN)
* move assignment method for MDL_request instead of memcpy()
* PARTIAL_INDEX_INTERSECT_INFO::init() instead of bzero()
* remove constructor from READ_RECORD() to make it trivial
* replace some memcpy() with c++ copy assignments
2019-03-14 16:33:17 +01:00
Alexander Barkov
d7a38eaf1a MDEV-18907 Slow log: RENAME TABLE is not "admin", while ALTER TABLE..RENAME is 2019-03-13 15:38:33 +04:00
Sergey Vojtovich
3568427d11 MDEV-18450 Slaves wait shutdown
The patches features an optional shutdown behavior to hold on until
after all connected slaves have been sent the last binlogged event.
The connected slave is one whose START SLAVE has been acknowledged and
that was not stopped since that though it could be technically
reconnecting in background.

The solution therefore disallows killing the dump thread until is has
found EOF of the latest binlog file.  It is up to the shutdown
requester (DBA) to set up a sufficiently large shutdown timeout value
for shudown to wait patiently until lagging behind slaves have been
synchronized. On the other hand if a specific slave needs exclusion
from synchronization the DBA would have to stop it manually which
would terminate its dump thread.

`mysqladmin shutdown' is extended with a `--wait_for_all_slaves' option
which translates to `SHUTDOW WAIT FOR ALL SLAVES' sql query
to enable the feature on the client side.

The patch also performs a small refactoring of the server shutdown
around close_connections() to introduce kill thread phases which
are two as of current.
2019-03-12 17:34:48 +02:00
Alexander Barkov
90ce95de4b Tests for MDEV-18892 Regression in slow log and admin statements
The patch for MDEV-15945 fixed MDEV-18892. Adding tests only.
2019-03-12 18:46:37 +04:00
Sergei Golubchik
a62e9a83c0 MDEV-15945 --ps-protocol does not test some queries
Make mysqltest to use --ps-protocol more

use prepared statements for everything that server supports
with the exception of CALL (for now).

Fix discovered test failures and bugs.

tests:
* PROCESSLIST shows Execute state, not Query
* SHOW STATUS increments status variables more than in text protocol
* multi-statements should be avoided (see tests with a wrong delimiter)
* performance_schema events have different names in --ps-protocol
* --enable_prepare_warnings

mysqltest.cc:
* make sure run_query_stmt() doesn't crash if there's
  no active connection (in wait_until_connected_again.inc)
* prepare all statements that server supports

protocol.h
* Protocol_discard::send_result_set_metadata() should not send
  anything to the client.

sql_acl.cc:
* extract the functionality of getting the user for SHOW GRANTS
  from check_show_access(), so that mysql_test_show_grants() could
  generate the correct column names in the prepare step

sql_class.cc:
* result->prepare() can fail, don't ignore its return value
* use correct number of decimals for EXPLAIN columns

sql_parse.cc:
* discard profiling for SHOW PROFILE. In text protocol it's done in
  prepare_schema_table(), but in --ps it is called on prepare only,
  so nothing was discarding profiling during execute.
* move the permission checking code for SHOW CREATE VIEW to
  mysqld_show_create_get_fields(), so that it would be called during
  prepare step too.
* only set sel_result when it was created here and needs to be
  destroyed in the same block. Avoid destroying lex->result.
* use the correct number of tables in check_show_access(). Saying
  "as many as possible" doesn't work when first_not_own_table isn't
  set yet.

sql_prepare.cc:
* use correct user name for SHOW GRANTS columns
* don't ignore verbose flag for SHOW SLAVE STATUS
* support preparing REVOKE ALL and ROLLBACK TO SAVEPOINT
* don't ignore errors from thd->prepare_explain_fields()
* use select_send result for sending ANALYZE and EXPLAIN, but don't
  overwrite lex->result, because it might be needed to issue execute-time
  errors (select_dumpvar - too many rows)

sql_show.cc:
* check grants for SHOW CREATE VIEW here, not in mysql_execute_command

sql_view.cc:
* use the correct function to check privileges. Old code was doing
  check_access() for thd->security_ctx, which is invoker's sctx,
  not definer's sctx. Hide various view related errors from the invoker.

sql_yacc.yy:
* initialize lex->select_lex for LOAD, otherwise it'll contain garbage
  data that happen to fail tests with views in --ps (but not otherwise).
2019-03-12 13:10:49 +01:00
Sergei Golubchik
22f1cf9292 cleanup: misc 2019-03-12 13:10:49 +01:00
Sergei Golubchik
dda2e940fb pass the slow logging information in thd->query_plan_flags
This solves the following issues:

* unlike lex->m_sql_cmd and lex->sql_command, thd->query_plan_flags
  is not reset in Prepared_statement::execute, it survives
  till the log_slow_statement(), so slow logging behaves correctly in --ps

* using thd->query_plan_flags for both slow_log_filter and
  log_slow_admin_statements means the definition of "admin" statements
  for the slow log is the same no matter how it is filtered out.
2019-03-12 13:10:49 +01:00
Monty
ec66320b0c Replace setting QPLAN_ADMIN directly with prepare_logs_for_admin_command() 2019-03-08 10:17:56 +02:00
Marko Mäkelä
aa4b2c1509 Merge 10.3 into 10.4 2019-03-07 08:02:33 +02:00
Alexander Barkov
f0cd707503 After-merge fix for MDEV-18333 Slow_queries count doesn't increase when slow_query_log is turned off 2019-03-06 23:44:58 +04:00
Marko Mäkelä
77103e9832 Merge 10.2 into 10.3 2019-03-06 16:20:13 +02:00
Marko Mäkelä
c155946c90 Merge 10.1 into 10.2 2019-03-06 15:15:59 +02:00
Alexander Barkov
26f0d72a3f A cleanup for MDEV-18333 Slow_queries count doesn't increase when slow_query_log is turned off
thd->lex->m_sql_cmd was not cleared between queries.
log_slow_query() could crash (when running mtr --ps) because of this.
2019-03-06 17:08:03 +04:00
Sergey Vojtovich
2b711d231a Adieu slave_list
slave_list was used to provide data for SHOW SLAVE HOSTS and
Slaves_connected status variable.

Introduced binlog_dump_thread_count which is exposed via Slaves_connected
(replaces slave_list.records).

Store Slave_info on THD and access it by iterating server_threads
(replaces slave_list).

Added:
THD::slave_info
binlog_dump_thread_count
show_slave_hosts_callback()

Removed:
slave_list
SLAVE_LIST_CHUNK
SLAVE_ERRMSG_SIZE
slave_list_key()
slave_info_free()
init_slave_list()
end_slave_list()
all_slave_list_mutexes
init_all_slave_list_mutexes()
key_LOCK_slave_list
LOCK_slave_list

Moved:
SLAVE_INFO -> Slave_info
register_slave() -> THD::register_slave()
unregister_slave() -> THD::unregister_slave()

Also removed redundant end_slave() from close_connections(): it is called
again soon afterwards by clean_up().

Pre-requisite for clean MDEV-18450 solution.
2019-03-06 17:06:09 +04:00
Marko Mäkelä
2a791c53ad Merge 10.3 into 10.4 2019-03-06 09:00:52 +02:00
Marko Mäkelä
446b3ebdfc Merge 10.2 into 10.3
FIXME: Properly resolve conflicts between MDEV-18883
and MDEV-7742/MDEV-8305, and record the correct result for
main.log_slow
2019-03-05 12:56:05 +02:00
Marko Mäkelä
9835f7b80f Merge 10.1 into 10.2 2019-03-04 16:46:58 +02:00
Alexander Barkov
19df45a705 MDEV-18333 Slow_queries count doesn't increase when slow_query_log is turned off 2019-03-04 13:49:15 +04:00
Robert Bindar
90ad4dbd17 MDEV-7597 Expiration of user passwords
This patch adds support for expiring user passwords.
The following statements are extended:
  CREATE USER user@localhost PASSWORD EXPIRE [option]
  ALTER USER user@localhost PASSWORD EXPIRE [option]
If no option is specified, the password is expired with immediate
effect. If option is DEFAULT, global policy applies according to
the default_password_lifetime system var (if 0, password never
expires, if N, password expires every N days). If option is NEVER,
the password never expires and if option is INTERVAL N DAY, the
password expires every N days.
The feature also supports the disconnect_on_expired_password system
var and the --connect-expired-password client option.

Closes #1166
2019-02-21 15:04:03 +01:00
Nikita Malyavin
073c93b194 MDEV-17082 Application-time periods: CREATE
* add syntax `CREATE TABLE ... PERIOD FOR <apptime>`
* add table period entity
2019-02-21 14:48:04 +01:00
Teemu Ollakka
3be9982f6a MDEV-18571 Fix wsrep PS protocol crash
Calls to wsrep_after_statement() were missing on PS protocol
codepath. Added calls after mysqld_stmt_execute() and
mysqld_stmt_bulk_execute().
2019-02-18 08:26:20 +02:00
Varun Gupta
be8709eb7b MDEV-6111 Optimizer Trace
This task involves the implementation for the optimizer trace.

This feature produces a trace for any SELECT/UPDATE/DELETE/,
which contains information about decisions taken by the optimizer during
the optimization phase (choice of table access method, various costs,
transformations, etc). This feature would help to tell why some decisions were
taken by the optimizer and why some were rejected.

Trace is session-local, controlled by the @@optimizer_trace variable.
To enable optimizer trace we need to write:
   set @@optimizer_trace variable= 'enabled=on';

To display the trace one can run:
   SELECT trace FROM INFORMATION_SCHEMA.OPTIMIZER_TRACE;

This task also involves:
    MDEV-18489: Limit the memory used by the optimizer trace
    introduces a switch optimizer_trace_max_mem_size which limits
    the memory used by the optimizer trace. This was implemented by
    Sergei Petrunia.
2019-02-13 11:52:36 +05:30
Sergey Vojtovich
4d5f85a3ec Bootstrap cleanups
Remove unused key_thread_bootstrap.
No more global bootstrap_error: it is return value of bootstrap() instead.
2019-02-12 23:07:51 +04:00
Marko Mäkelä
e80bcd7f64 Merge 10.3 into 10.4 2019-02-05 12:48:02 +02:00
Sergei Golubchik
5b15cc613e MDEV-11340 Allow multiple alternative authentication methods for the same user
introduce the syntax

... IDENTIFIED { WITH | VIA }
      plugin [ { USING | AS } auth ]
 [ OR plugin [ { USING | AS } auth ]
 [ OR ... ]]

Server will try auth plugins in the specified order until the first
success. No protocol changes, server uses the existing "switch plugin"
packet.

The auth chain is stored in json as

  "auth_or":[{"plugin":"xxx","authentication_string":"yyy"},
             {},
             {"plugin":"foo","authentication_string":"bar"},
            ...],
  "plugin":"aaa", "authentication_string":"bbb"

Note:
* "auth_or" implies that there might be "auth_and" someday;
* one entry in the array is an empty object, meaning to take plugin/auth
  from the main json object. This preserves compatibility with
  the existing mysql.global_priv table and with the mysql.user view.
  This entry is preferrably a mysql_native_password plugin for a
  non-empty mysql.user.password column.

SET PASSWORD is supported and changes the password for the *first*
plugin in the chain that has a notion of a "password"
2019-02-04 16:06:57 +01:00
Vladislav Vaintroub
261ce5286f MDEV-18281 COM_RESET_CONNECTION changes the connection encoding
Store original charset during client authentication, and restore it for
COM_RESET_CONNECTION
2019-02-02 17:32:15 +01:00
Vladislav Vaintroub
e214aa1cd3 MDEV-18281 COM_RESET_CONNECTION changes the connection encoding
Store original charset during client authentication, and restore it for
COM_RESET_CONNECTION
2019-02-02 17:29:33 +01:00
Sergey Vojtovich
4b3656a44d Avoid taking LOCK_thread_count for thread_count protection
Replaced wait on COND_thread_count with busy waiting with 1 millisecond
sleep.

Aim is to reduce usage of LOCK_thread_count and COND_thread_count.
2019-01-29 11:56:35 +04:00
Sergey Vojtovich
3503fbbebf Move THD list handling to THD_list
Implemented and integrated THD_list as a replacement for the global
thread list. It uses own mutex instead of LOCK_thread_count for THD
list protection.

Removed unused first_global_thread() and next_global_thread().

delayed_insert_threads is now protected by LOCK_delayed_insert. Although
this patch doesn't fix very wrong synchronization of this variable.

After this patch there are only 2 legitimate uses of LOCK_thread_count
left, both in mysqld.cc: thread_count and ready_to_exit.

Aim is to reduce usage of LOCK_thread_count and COND_thread_count.
Part of MDEV-15135.
2019-01-28 17:39:07 +04:00
Sergey Vojtovich
c88fd54d17 Execute bootstrap in main thread
Bootstrap in a separate thread was introduced in 746f0b3b7 to workaround
OS/2 small stack size. OS/2 support was discontinued in 2006 and modern
operating systems have default stack size a few times larger than
default thread_stack and it is tunable.

Aim is to reduce usage of LOCK_thread_count and COND_thread_count.
Part of MDEV-15135.
2019-01-28 17:39:06 +04:00