Commit graph

8708 commits

Author SHA1 Message Date
Marko Mäkelä
1e9c2b2305 Merge 10.1 into 10.2 2019-03-27 12:26:11 +02:00
Marko Mäkelä
a6585d5ce9 Merge 10.0 into 10.1 2019-03-27 11:56:08 +02:00
Marko Mäkelä
1933cf98e8 Merge 5.5 into 10.0 2019-03-26 14:13:46 +02:00
chriscalender
f00e25b4c4 Fix for MDEV-15538, '-N' Produce html output wrong 2019-03-22 01:01:59 +04:00
Marko Mäkelä
514b305dfb Merge 10.3 into 10.4
The MDEV-17262 commit 26432e49d3
was skipped. In Galera 4, the implementation would seem to require
changes to the streaming replication.

In the tests archive.rnd_pos main.profiling, disable_ps_protocol
for SHOW STATUS and SHOW PROFILE commands until MDEV-18974
has been fixed.
2019-03-20 10:41:32 +02:00
Sergei Golubchik
b64fde8f38 Merge branch '10.2' into 10.3 2019-03-17 13:06:41 +01:00
Sergei Golubchik
0508d327ae Merge branch '10.1' into 10.2 2019-03-15 21:00:41 +01:00
Sergei Golubchik
3d2d060b62 fix gcc 8 compiler warnings
There were two newly enabled warnings:
1. cast for a function pointers. Affected sql_analyse.h, mi_write.c
   and ma_write.cc, mf_iocache-t.cc, mysqlbinlog.cc, encryption.cc, etc

2. memcpy/memset of nontrivial structures. Fixed as:
* the warning disabled for InnoDB
* TABLE, TABLE_SHARE, and TABLE_LIST got a new method reset() which
  does the bzero(), which is safe for these classes, but any other
  bzero() will still cause a warning
* Table_scope_and_contents_source_st uses `TABLE_LIST *` (trivial)
  instead of `SQL_I_List<TABLE_LIST>` (not trivial) so it's safe to
  bzero now.
* added casts in debug_sync.cc and sql_select.cc (for JOIN)
* move assignment method for MDL_request instead of memcpy()
* PARTIAL_INDEX_INTERSECT_INFO::init() instead of bzero()
* remove constructor from READ_RECORD() to make it trivial
* replace some memcpy() with c++ copy assignments
2019-03-14 16:33:17 +01:00
Sergey Vojtovich
3568427d11 MDEV-18450 Slaves wait shutdown
The patches features an optional shutdown behavior to hold on until
after all connected slaves have been sent the last binlogged event.
The connected slave is one whose START SLAVE has been acknowledged and
that was not stopped since that though it could be technically
reconnecting in background.

The solution therefore disallows killing the dump thread until is has
found EOF of the latest binlog file.  It is up to the shutdown
requester (DBA) to set up a sufficiently large shutdown timeout value
for shudown to wait patiently until lagging behind slaves have been
synchronized. On the other hand if a specific slave needs exclusion
from synchronization the DBA would have to stop it manually which
would terminate its dump thread.

`mysqladmin shutdown' is extended with a `--wait_for_all_slaves' option
which translates to `SHUTDOW WAIT FOR ALL SLAVES' sql query
to enable the feature on the client side.

The patch also performs a small refactoring of the server shutdown
around close_connections() to introduce kill thread phases which
are two as of current.
2019-03-12 17:34:48 +02:00
Marko Mäkelä
e450527938 Merge 10.3 into 10.4 2019-03-12 16:14:31 +02:00
Sergei Golubchik
a62e9a83c0 MDEV-15945 --ps-protocol does not test some queries
Make mysqltest to use --ps-protocol more

use prepared statements for everything that server supports
with the exception of CALL (for now).

Fix discovered test failures and bugs.

tests:
* PROCESSLIST shows Execute state, not Query
* SHOW STATUS increments status variables more than in text protocol
* multi-statements should be avoided (see tests with a wrong delimiter)
* performance_schema events have different names in --ps-protocol
* --enable_prepare_warnings

mysqltest.cc:
* make sure run_query_stmt() doesn't crash if there's
  no active connection (in wait_until_connected_again.inc)
* prepare all statements that server supports

protocol.h
* Protocol_discard::send_result_set_metadata() should not send
  anything to the client.

sql_acl.cc:
* extract the functionality of getting the user for SHOW GRANTS
  from check_show_access(), so that mysql_test_show_grants() could
  generate the correct column names in the prepare step

sql_class.cc:
* result->prepare() can fail, don't ignore its return value
* use correct number of decimals for EXPLAIN columns

sql_parse.cc:
* discard profiling for SHOW PROFILE. In text protocol it's done in
  prepare_schema_table(), but in --ps it is called on prepare only,
  so nothing was discarding profiling during execute.
* move the permission checking code for SHOW CREATE VIEW to
  mysqld_show_create_get_fields(), so that it would be called during
  prepare step too.
* only set sel_result when it was created here and needs to be
  destroyed in the same block. Avoid destroying lex->result.
* use the correct number of tables in check_show_access(). Saying
  "as many as possible" doesn't work when first_not_own_table isn't
  set yet.

sql_prepare.cc:
* use correct user name for SHOW GRANTS columns
* don't ignore verbose flag for SHOW SLAVE STATUS
* support preparing REVOKE ALL and ROLLBACK TO SAVEPOINT
* don't ignore errors from thd->prepare_explain_fields()
* use select_send result for sending ANALYZE and EXPLAIN, but don't
  overwrite lex->result, because it might be needed to issue execute-time
  errors (select_dumpvar - too many rows)

sql_show.cc:
* check grants for SHOW CREATE VIEW here, not in mysql_execute_command

sql_view.cc:
* use the correct function to check privileges. Old code was doing
  check_access() for thd->security_ctx, which is invoker's sctx,
  not definer's sctx. Hide various view related errors from the invoker.

sql_yacc.yy:
* initialize lex->select_lex for LOAD, otherwise it'll contain garbage
  data that happen to fail tests with views in --ps (but not otherwise).
2019-03-12 13:10:49 +01:00
Sergei Golubchik
22f1cf9292 cleanup: misc 2019-03-12 13:10:49 +01:00
Monty
f010c90807 Fixed memory leak in mysqltest
read_command_buf was not freed at exit, which could cause a warning from
valgrind
2019-03-11 20:03:27 +02:00
Marko Mäkelä
f06868e56b Unbreak the Windows build after d3e3bb77c2 2019-03-08 12:13:53 +02:00
Marko Mäkelä
c67b306e4f Merge 10.3 into 10.4 2019-03-08 11:19:48 +02:00
Monty
d3e3bb77c2 Print value of 'protocol' option in --help 2019-03-08 10:05:30 +02:00
Marko Mäkelä
2d0dd62cf7 Merge 10.2 into 10.3 2019-03-08 00:26:55 +02:00
Sergei Golubchik
2f742b571f MDEV-18376 Memory leak in main.mysqladmin
don't initialize mysql structure before it actually becomes needed.

This fixes main.mysqladmin failures
in ASAN_OPTIONS="abort_on_error=1" runs
2019-03-06 15:28:27 +01:00
Robert Bindar
90ad4dbd17 MDEV-7597 Expiration of user passwords
This patch adds support for expiring user passwords.
The following statements are extended:
  CREATE USER user@localhost PASSWORD EXPIRE [option]
  ALTER USER user@localhost PASSWORD EXPIRE [option]
If no option is specified, the password is expired with immediate
effect. If option is DEFAULT, global policy applies according to
the default_password_lifetime system var (if 0, password never
expires, if N, password expires every N days). If option is NEVER,
the password never expires and if option is INTERVAL N DAY, the
password expires every N days.
The feature also supports the disconnect_on_expired_password system
var and the --connect-expired-password client option.

Closes 
2019-02-21 15:04:03 +01:00
Oleksandr Byelkin
93ac7ae70f Merge branch '10.3' into 10.4 2019-02-21 14:40:52 +01:00
Marko Mäkelä
fc124778ea Merge 10.2 into 10.3 2019-02-19 17:41:13 +02:00
Vladislav Vaintroub
3a42926c88 MDEV-18204 Fix rocksdb incremental backup
Fix incremental prepare to copy #rocksdb subdirectory from the
incremental dir.
2019-02-18 18:59:05 +01:00
Marko Mäkelä
e80bcd7f64 Merge 10.3 into 10.4 2019-02-05 12:48:02 +02:00
Marko Mäkelä
a249e57b68 Merge 10.1 into 10.2
Temporarily disable a test for
commit 2175bfce3e
because fixing it in 10.2 requires updating libmariadb.
2019-02-03 17:22:05 +02:00
Vladislav Vaintroub
261ce5286f MDEV-18281 COM_RESET_CONNECTION changes the connection encoding
Store original charset during client authentication, and restore it for
COM_RESET_CONNECTION
2019-02-02 17:32:15 +01:00
Vladislav Vaintroub
e214aa1cd3 MDEV-18281 COM_RESET_CONNECTION changes the connection encoding
Store original charset during client authentication, and restore it for
COM_RESET_CONNECTION
2019-02-02 17:29:33 +01:00
Marko Mäkelä
213ece2f2e Merge 10.1 into 10.1
This is joint work with Oleksandr Byelkin.
2019-02-02 13:00:15 +02:00
Marko Mäkelä
081fd8bfa2 Merge 10.1 into 10.2 2019-02-02 11:40:02 +02:00
Oleksandr Byelkin
c4f97d3cfa Merge branch '5.5' into 10.0 2019-01-28 20:52:47 +01:00
Oleksandr Byelkin
a3df9bcadc Merge branch '5.5' into 10.0 2019-01-28 10:36:12 +01:00
Sergei Golubchik
2175bfce3e Crude "auto-load-data-local-infile" mode
Disable LOAD DATA LOCAL INFILE suport by default and
auto-enable it for the duration of one query, if the query
string starts with the word "load". In all other cases the application
should enable LOAD DATA LOCAL INFILE support explicitly.
2019-01-27 18:54:12 +01:00
Marko Mäkelä
78829a5780 Merge 10.3 into 10.4 2019-01-24 22:42:35 +02:00
Andrei Elkin
5d48ea7d07 MDEV-10963 Fragmented BINLOG query
The problem was originally stated in
  http://bugs.mysql.com/bug.php?id=82212
The size of an base64-encoded Rows_log_event exceeds its
vanilla byte representation in 4/3 times.
When a binlogged event size is about 1GB mysqlbinlog generates
a BINLOG query that can't be send out due to its size.

It is fixed with fragmenting the BINLOG argument C-string into
(approximate) halves when the base64 encoded event is over 1GB size.
The mysqlbinlog in such case puts out

    SET @binlog_fragment_0='base64-encoded-fragment_0';
    SET @binlog_fragment_1='base64-encoded-fragment_1';
    BINLOG @binlog_fragment_0, @binlog_fragment_1;

to represent a big BINLOG.
For prompt memory release BINLOG handler is made to reset the BINLOG argument
user variables in the middle of processing, as if @binlog_fragment_{0,1} = NULL
is assigned.

Notice the 2 fragments are enough, though the client and server still may
need to tweak their @@max_allowed_packet to satisfy to the fragment
size (which they would have to do anyway with greater number of
fragments, should that be desired).

On the lower level the following changes are made:

Log_event::print_base64()
  remains to call encoder and store the encoded data into a cache but
  now *without* doing any formatting. The latter is left for time
  when the cache is copied to an output file (e.g mysqlbinlog output).
  No formatting behavior is also reflected by the change in the meaning
  of the last argument which specifies whether to cache the encoded data.

Rows_log_event::print_helper()
  is made to invoke a specialized fragmented cache-to-file copying function
  which is

copy_cache_to_file_wrapped()
  that takes care of fragmenting also optionally wraps encoded
  strings (fragments) into SQL stanzas.

my_b_copy_to_file()
  is refactored to into my_b_copy_all_to_file(). The former function
  is generalized
  to accepts more a limit argument to constraint the copying and does
  not reinitialize anymore the cache into reading mode.
  The limit does not do any effect on the fully read cache.
2019-01-24 20:44:50 +02:00
Marko Mäkelä
947b6b849d Merge 10.2 into 10.3 2019-01-24 16:14:12 +02:00
Marko Mäkelä
25161e6219 Merge 10.1 into 10.2 2019-01-24 14:43:29 +02:00
Marko Mäkelä
65350042a4 Merge 10.0 into 10.1 2019-01-24 13:24:13 +02:00
Sergei Golubchik
edeba0c873 MDEV-17868 mysqltest fails to link with system PCRE libraries
pcre needs symbols from pcreposix
(but this is only an issue when linking with system static libraries)
2019-01-24 10:34:05 +01:00
Eugene Kosov
ad220b96fb MDEV-16658 Memory leak in mysqltest on connect failure
Close connection handler on connection failure. This fixes 14 failing tests in
main suite under clang+ASAN build.

ASAN report for main.connect looks like this:
=================================================================
==25495==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 146280 byte(s) in 115 object(s) allocated from:
     0x4fba47 in calloc /fun/cpp_projects/llvm_toolchain/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:138
     0x5a7a02 in mysql_init /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:977:26
     0x570a7a in do_connect(st_command*) /work/mariadb/client/mysqltest.cc:6096:26
     0x584c39 in main /work/mariadb/client/mysqltest.cc:9321:9
     0x7fd15514db96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310

Indirect leak of 7065600 byte(s) in 115 object(s) allocated from:
     0x4fb80f in __interceptor_malloc /fun/cpp_projects/llvm_toolchain/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:129
     0x637a83 in my_context_init /work/mariadb/libmariadb/libmariadb/ma_context.c:367:23
     0x59fd16 in mysql_optionsv /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:2738:9
     0x5bc1d4 in mysql_options /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:3242:10
     0x570b94 in do_connect(st_command*) /work/mariadb/client/mysqltest.cc:6103:7
     0x584c39 in main /work/mariadb/client/mysqltest.cc:9321:9
     0x7fd15514db96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310

Indirect leak of 940240 byte(s) in 115 object(s) allocated from:
     0x4fb80f in __interceptor_malloc /fun/cpp_projects/llvm_toolchain/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:129
     0x64386e in ma_init_dynamic_array /work/mariadb/libmariadb/libmariadb/ma_array.c:49:31
     0x649ead in _hash_init /work/mariadb/libmariadb/libmariadb/ma_hash.c:52:7
     0x5a3080 in mysql_optionsv /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:2938:13
     0x5bc20c in mysql_options4 /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:3248:10
     0x56f63b in connect_n_handle_errors(st_command*, st_mysql*, char const*, char const*, char const*, char const*, int, char const*) /work/mariadb/client/mysqltest.cc:5874:3
     0x57146b in do_connect(st_command*) /work/mariadb/client/mysqltest.cc:6193:7
     0x584c39 in main /work/mariadb/client/mysqltest.cc:9321:9
     0x7fd15514db96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
...

Closes 
2019-01-23 17:00:33 +01:00
Marko Mäkelä
6dbc50a376 Merge 10.3 into 10.4 2018-12-13 22:05:34 +02:00
Marko Mäkelä
f6e16bdc62 Merge 10.2 into 10.3 2018-12-13 21:58:35 +02:00
Marko Mäkelä
2e5aea4bab Merge 10.1 into 10.2 2018-12-13 15:47:38 +02:00
Marko Mäkelä
621041b676 Merge 10.0 into 10.1
Also, apply the MDEV-17957 changes to encrypted page checksums,
and remove error message output from the checksum function,
because these messages would be useless noise when mariabackup
is retrying reads of corrupted-looking pages, and not that
useful during normal server operation either.

The error messages in fil_space_verify_crypt_checksum()
should be refactored separately.
2018-12-13 13:37:21 +02:00
Sergei Golubchik
b58f28725b Merge branch '5.5' into 10.0 2018-12-12 20:19:06 +01:00
Sergei Golubchik
4abb8216a0 MDEV-17658 change the structure of mysql.user table
Implement User_table_json.
Fix scripts to use mysql.global_priv.
Fix tests.
2018-12-12 00:31:44 +01:00
Monty
c82855d882 Changed some MySQL names in messages to MariaDB
- Remove obsolete documentation files
- Removed old obsolete information from some documentation files
2018-12-09 20:49:05 +02:00
Monty
ae58cd6b87 Simple cleanups (no logic changes) 2018-12-09 20:49:05 +02:00
Sergei Golubchik
32b7d456d5 mysqltest: use a dynamically growing command buffer 2018-11-28 20:18:49 +01:00
Alexander Barkov
b9a9055793 MDEV-17712 Remove C_TIME_FUZZY_DATES, C_TIME_DATETIME_ONLY, C_TIME_TIME_ONLY 2018-11-14 16:00:38 +04:00
Marko Mäkelä
c761b43451 Merge 10.3 into 10.4 2018-11-08 10:19:55 +02:00
Marko Mäkelä
862af4d255 Merge 10.2 into 10.3 2018-11-07 13:11:04 +02:00