Commit graph

3374 commits

Author SHA1 Message Date
Marko Mäkelä
6d390bab4a Merge 10.2 into bb-10.2-ext 2018-01-30 20:18:25 +02:00
Vladislav Vaintroub
7cdf759c86 MDEV-14485 Server hangs on startup in THD::init
Solve 3 way deadlock between plugin_initialiaze(), THD::init() and
mysql_sys_var_char().

The deadlock exists because of the lock order inversion between
LOCK_global_system_variables mutex and LOCK_system_variables_hash
read-write lock-

In this case, it is enough to change LOCK_system_variables_hash to prefer
reads to fix the deadlock, i.e change it to mysql_prlock_t
2018-01-29 07:33:52 +00:00
Monty
5e0b13d173 Fixed wrong arguments to printf and related functions
Other things, mainly to get
create_mysqld_error_find_printf_error tool to work:

- Added protection to not include mysqld_error.h twice
- Include "unireg.h" instead of "mysqld_error.h" in server
- Added protection if ER_XX messages are already defined
- Removed wrong calls to my_error(ER_OUTOFMEMORY) as
  my_malloc() and my_alloc will do this automatically
- Added missing %s to ER_DUP_QUERY_NAME
- Removed old and wrong calls to my_strerror() when using
  MY_ERROR_ON_RENAME (wrong merge)
- Fixed deadlock error message from Galera. Before the extra
  information given to ER_LOCK_DEADLOCK was missing because
  ER_LOCK_DEADLOCK doesn't provide any extra information.

I kept #ifdef mysqld_error_find_printf_error_used in sql_acl.h
to make it easy to do this kind of check again in the future
2018-01-04 16:24:09 +02:00
Monty
d2f557fa3d Fixed crash in show processlist with blocked connection
If connection was not blocked, the 'db' was not properly reset
This bug didn't affect information_schema
2017-12-12 15:42:22 +02:00
Marko Mäkelä
e312a407b8 Merge 10.2 into bb-10.2-ext 2017-12-11 15:06:11 +02:00
Sergei Petrunia
ddc1d6904a MDEV-14123: .rocksdb folder may break workflow which re-create data directory
Part2: make MyRocks add its directory into @@ignore_db_dirs when starting.

This is necessary because apparently not everybody are using plugin's my.cnf
So load ha_rocksdb.{so,dll} manually and then hit MDEV-12451, MDEV-14461
etc.
2017-12-11 12:31:58 +03:00
Monty
c2118a08b1 Move all kill mutex protection to LOCK_thd_kill
LOCK_thd_data was used to protect both THD data and
ensure that the THD is not deleted while it was in use

This patch moves the THD delete protection to LOCK_thd_kill,
which already protects the THD for kill.

The benefits are:
- More well defined what LOCK_thd_data protects
- LOCK_thd_data usage is now much simpler and easier to verify
- Less chance of deadlocks in SHOW PROCESS LIST as there is less
  chance of interactions between mutexes
- Remove not needed LOCK_thread_count from
  thd_get_error_context_description()
- Fewer mutex taken for thd->awake()

Other things:
- Don't take mysys->var mutex in show processlist to check if thread
  is kill marked
- thd->awake() now automatically takes the LOCK_thd_kill mutex
  (Simplifies code)
- Apc uses LOCK_thd_kill instead of LOCK_thd_data
2017-12-08 13:46:23 +02:00
Monty
7891a713da Don't wait too long in SHOW PROCESSLIST
This will allow show processlist to continue, without blocking
all new connections, if some threads gets stuck while holding
LOCK_thd_data or mysys_var->mutex

Connections that has mutex 'stuck' are marked as 'Busy' in 'Command'

Todo:
Make F_BACKOFF to do 'pause' instead of just (1)
2017-12-08 13:44:46 +02:00
Alexander Barkov
5b697c5a23 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2017-11-29 12:06:48 +04:00
Sergei Golubchik
3422ceb10c remove dead code 2017-11-22 12:52:16 +01:00
Monty
9ec19b9b41 Reducing memory when using information schema
The background is that one user had a lot of views and using some complex
queries on information schema temporary memory of more than 2G was used.

- Added new element 'total_alloc' to MEM_ROOT for easier debugging.
- Added MAX_MEMORY_USED to information_schema.processlist.
- Added new status variable "Memory_used_initial" that shows how much MariaDB
  uses at startup. This gives the base value for "Memory_used".
- Reuse memory continuously for information schema queries instead of
  only freeing memory at query end.

Other things
- Removed some not needed set_notnull() calls for not null columns.
2017-11-02 20:37:25 +02:00
Alexander Barkov
003cb2f424 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2017-10-30 16:42:46 +04:00
Sergei Golubchik
e0a1c745ec Merge branch '10.1' into 10.2 2017-10-24 14:53:18 +02:00
Sergei Golubchik
9d2e2d7533 Merge branch '10.0' into 10.1 2017-10-22 13:03:41 +02:00
Alexander Barkov
edfdf0d0a3 A cleanup for MDEV-13415 Wrap the code in sp.cc into a class Sp_handler
Fixing mysql_execute_command() and store_schema_proc() not to use
stored object type codes TYPE_ENUM_PROCEDURE and TYPE_ENUM_FUNCTION.
Using pointers to Sp_handler instead, to make the code symmetric across
existing (function,procedure) and future (e.g package) SP object types.
2017-10-20 18:44:28 +04:00
Sergei Golubchik
da4503e956 Merge branch '5.5' into 10.0 2017-10-18 15:14:39 +02:00
Alexander Barkov
30e7d6709f Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2017-10-18 14:11:55 +04:00
Sergei Golubchik
d76f5774fe MDEV-13459 Warnings, when compiling with gcc-7.x
mostly caused by -Wimplicit-fallthrough
2017-10-17 07:37:39 +02:00
Sergey Vojtovich
8be76a6a90 MDEV-10892 - rpl.rpl_semi_sync_uninstall_plugin fails with Assertion `0' failure
in buildbot

Removed plugin_array_version: it is being checked without mutex protection and
thus is prone to data race and race conditions. In effect plugins are not
protected from concurrent destruction.

Removed state_mask inversion: doesn't seem to make any sense.

When collecting local plugins list, only add plugins that match state_mask.

Use plugin ref counting to protect against concurrent plugin destruction.
2017-10-13 16:50:58 +04:00
Vesa Pentti
93aadda513 MDEV-13149 -- show function status now works with PAD_CHAR_TO_FULL_LENGTH 2017-10-10 16:16:12 +03:00
Vladislav Vaintroub
dc93ce8dea Windows : Fix truncation warnings in sql/ 2017-10-10 06:19:50 +00:00
Sergei Golubchik
9b11956e86 MDEV-13049 Querying INFORMATION_SCHEMA becomes slow in MariaDB 10.1
Optimizations:
* avoid faster ext_table_discovery_simple() *only* when there are
  engines with discover_table_names() method *and* they look at
  file names. P_S implements discover_table_names(), but it's not
  a reason to use slower extension_based_table_discovery().
* don't pre-sort table names if ORDER BY or GROUP BY was specified
* starting from 10.3 only sort table names for SHOW commands
2017-10-09 17:58:28 +02:00
Sergei Golubchik
559dec33cc cleanup: is_show_command(thd) 2017-10-09 17:58:28 +02:00
Vladislav Vaintroub
da05d0276a merge 10.1->10.2
Some innobase/xtrabackup changes around  from 10.1 are null merged
, in partucular using os_set_file_size to extend tablespaces in server
or mariabackup.
They require non-trivial amount of additional  work in 10.2, due to
innobase differences between 10.1 and 10.2
2017-10-07 17:43:26 +00:00
Alexander Barkov
1cfaafafee MDEV-13242 Wrong results for queries with row constructors and information_schema 2017-10-05 13:41:16 +04:00
Alexander Barkov
8ae8cd6348 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2017-10-02 22:35:13 +04:00
Vladislav Vaintroub
157d130a87 Fix some conversion warnings. 2017-09-30 09:26:58 +00:00
Sergei Golubchik
e6e23c0d49 MDEV-9739 Assertion `m_status == DA_ERROR || m_status == DA_OK' failed in Diagnostics_area::message() ; connect.xml* tests fail in buildbot 2017-08-01 09:52:57 +02:00
Alexander Barkov
c9218ff439 MDEV-13415 Wrap the code in sp.cc into a class Sp_handler 2017-07-31 23:00:02 +04:00
Alexander Barkov
4937474f86 MDEV-13414 Fix the SP code to avoid excessive use of strlen 2017-07-31 17:34:59 +04:00
Alexander Barkov
8b2c7c9444 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2017-07-07 12:43:10 +04:00
Sergei Golubchik
f6633bf058 Merge branch '10.1' into 10.2 2017-07-05 19:08:55 +02:00
Sergei Golubchik
291411c96c MDEV-13132 Information Schema does not show whether column default is expression or literal
Fix INFORMATION_SCHEMA.COLUMNS.COLUMN_DEFAULT to be standard-compliant,
but keep SHOW COLUMNS backward-compatibly unchanged.
2017-07-05 17:15:59 +02:00
Sergei Golubchik
785e2248bd MDEV-13089 identifier quoting in partitioning
don't print partitioning expression as it was entered by the user,
use Item::print() according to the sql_mode and sql_quote_show_create
2017-07-05 17:15:59 +02:00
Sergei Golubchik
504eff0ca1 cleanup: generate_partition_syntax()
Don't write to a temporary file, use String.
Remove strange one-liner "helpers", use String methods.
Don't use current_thd, don't allocate memory for 1-byte strings, etc.
2017-07-05 17:15:58 +02:00
Alexander Barkov
5c0df0e4a8 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2017-07-04 15:31:25 +04:00
Igor Babaev
e608023947 Fixed the bug mdev-13107 and some similar unreported bugs.
The problems were in the code of sql_show.cc. There the tables
could be opened in such a way that mysql_derived_init() never
worked for CTE tables. As a result they were not marked as
derived and mysql_handle_derived() were not called for derived
tables used in their specifications.
2017-06-28 11:39:33 -07:00
Vicențiu Ciorbaru
2e335a471c Merge remote-tracking branch '10.0' into 10.1 2017-06-21 16:19:43 +03:00
Vicențiu Ciorbaru
34da3be8a8 MDEV-10463: Granted as a whole to roles, databases are not show in SHOW DATABASES
The problem lies in not checking role privileges as well during SHOW
DATABASES command. This problem is also apparent for SHOW CREATE
DATABASE command.

Other SHOW COMMANDS make use of check_access, which in turn makes use of
acl_get for both priv_user and priv_role parts, which allows them to
function correctly.
2017-06-15 19:20:35 +03:00
Alexander Barkov
765347384a Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2017-06-15 15:27:11 +04:00
Monty
36ae8846ca Fixed sequences based on comments from Peter Gulutzan and Andrii Nikitin
- Changed names of SEQUENCE table columns to be more close to ANSI
- Fixed error message for SHOW SEQUENCE non_existing_sequence
- Allow syntax CACHE +1
- Fixed ALTER TABLE for TEMPORARY sequences.
2017-06-03 16:08:23 +03:00
Marko Mäkelä
b61700c221 Merge 10.0 into 10.1 2017-05-23 08:59:03 +03:00
Marko Mäkelä
70505dd45b Merge 10.1 into 10.2 2017-05-22 09:46:51 +03:00
Sergei Golubchik
7c03edf2fe MDEV-6262 analyze the coverity report on mariadb
uploaded 10.0, analyzed everything with the Impact=High
(and a couple of Medium)
2017-05-19 20:26:56 +02:00
Marko Mäkelä
71cd205956 Silence bogus GCC 7 warnings -Wimplicit-fallthrough
Do not silence uncertain cases, or fix any bugs.

The only functional change should be that ha_federated::extra()
is not calling DBUG_PRINT to report an unhandled case for
HA_EXTRA_PREPARE_FOR_DROP.
2017-05-17 08:27:04 +03:00
Marko Mäkelä
7972da8aa1 Silence bogus GCC 7 warnings -Wimplicit-fallthrough
Do not silence uncertain cases, or fix any bugs.

The only functional change should be that ha_federated::extra()
is not calling DBUG_PRINT to report an unhandled case for
HA_EXTRA_PREPARE_FOR_DROP.
2017-05-17 08:07:02 +03:00
Alexander Barkov
791374354c MDEV-9217 Split Item::tmp_table_field_from_field_type() into virtual methods in Type_handler
- Adding Type_handler::make_table_field() and moving pieces of the code
  from Item::tmp_table_field_from_field_type() to virtual implementations
  for various type handlers.

- Adding a new Type_all_attributes, to access to Item's extended
  attributes, such as decimal_precision() and geometry_type().

- Adding a new class Record_addr, to pass record related information
  to Type_handler methods (ptr, null_ptr and null_bit) as a single structure.
  Note, later it will possibly be extended for BIT-alike field purposes,
  by adding new members (bit_ptr_arg, bit_ofs_arg).

- Moving the code from Field_new_decimal::create_from_item()
  to Type_handler_newdecimal::make_table_field().

- Removing Field_new_decimal() and Field_geom() helper constructor
  variants that were used for temporary field creation.

- Adding Item_field::type_handler(), Field::type_handler() and
  Field_blob::type_handler() to return correct type handlers for
  blob variants, according to Field_blob::packlength.

- Adding Type_handler_blob_common, as a common parent for
  Type_handler_tiny_blob, Type_handler_blob, Type_handler_medium_blob
  and Type_handler_long_blob.

- Implementing Type_handler_blob_common::Item_hybrid_func_fix_attributes().

  It's needed for cases when TEXT variants of different character sets are mixed
  in LEAST, GREATEST, CASE and its abreviations (IF, IFNULL, COALESCE), e.g.:
      CREATE TABLE t1 (
        a TINYTEXT CHARACTER SET latin1,
        b TINYTEXT CHARACTER SET utf8
      );
      CREATE TABLE t2 AS SELECT COALESCE(a,b) FROM t1;
  Type handler aggregation returns TINYTEXT as a common data type
  for the two columns. But as conversion from latin1 to utf8
  happens for "a", the maximum possible length of "a" grows from 255 to 255*3.
  Type_handler_blob_common::Item_hybrid_func_fix_attributes() makes sure
  to update the blob type handler according to max_length.

- Adding Type_handler::blob_type_handler(uint max_octet_length).

- Adding a few m_type_aggregator_for_result.add() pairs, because
  now Item_xxx::type_handler() can return pointers to type_handler_tiny_blob,
  type_handler_blob, type_handler_medium_blob, type_handler_long_blob.
  Before the patch only type_handler_blob was possible result of type_handler().

- Making type_handler_tiny_blob, type_handler_blob, type_handler_medium_blob,
  type_handler_long_blob public.

- Removing the condition in Item_sum_avg::create_tmp_field()
  checking Item_sum_avg::result_type() against DECIMAL_RESULT.
  Now both REAL_RESULT and DECIMAL_RESULT are symmetrically handled
  by tmp_table_field_from_field_type().

- Removing Item_geometry_func::create_field_for_create_select(),
  as the inherited version perfectly works.

- Fixing Item_func_as_wkb::field_type() to return MYSQL_TYPE_LONG_BLOB
  rather than MYSQL_TYPE_BLOB. It's needed to make sure that
  tmp_table_field_from_field_type() creates a LONGBLOB field for AsWKB().

- Fixing Item_func_as_wkt::fix_length_and_dec() to set max_length to
  UINT32_MAX rather than MAX_BLOB_WIDTH, to make sure that
  tmp_table_field_from_field_type() creates a LONGTEXT field for AsWKT().

- Removing Item_func_set_user_var::create_field_for_create_select(),
  as the inherited version works fine.

- Adding Item_func_get_user_var::create_field_for_create_select() to
  make sure that "CREATE TABLE t1 AS SELECT @string_user variable"
  always creates a field of LONGTEXT/LONGBLOB type.

- Item_func_ifnull::create_field_for_create_select()
  behavior has changed. Before the patch it passed set_blob_packflag=false,
  which meant to create LONGBLOB for all blob variants.
  Now it takes into account max_length, which gives better column
  data types for:
    CREATE TABLE t2 AS SELECT IFNULL(blob_column1, blob_column2) FROM t1;

- Fixing Item_func_nullif::fix_length_and_dec() to use
  set_handler(args[2]->type_handler()) instead of
  set_handler_by_field_type(args[2]->field_type()).
  This is needed to distinguish between BLOB variants.

- Implementing Item_blob::type_handler(), to make sure to create
  proper BLOB field variant, according to max_length, for queries like:
    CREATE TABLE t1 AS
      SELECT some_blob_field FROM INFORMATION_SCHEMA.SOME_TABLE;

- Fixing Item_field::real_type_handler() to make sure that
  the code aggregating fields for UNION gets a proper BLOB
  variant type handler from fields.

- Adding a special code into Item_type_holder::make_field_by_type(),
  to make sure that after aggregating field types it also properly
  takes into account max_length when mixing TEXT variants of different
  character sets and chooses a proper TEXT variant:
      CREATE TABLE t1 (
        a TINYTEXT CHARACTER SET latin1,
        b TINYTEXT CHARACTER SET utf8
      );
      CREATE TABLE t2 AS SELECT a FROM t1 UNION SELECT b FROM t1;

- Adding tests, for better coverage of IFNULL, NULLIF, UNION.

- The fact that tmp_table_field_from_field_type() now takes
  into account BLOB variants (instead of always creating LONGBLOB),
  tests results for WEIGHT_STRING() and NULLIF() and UNION
  have become more precise.
2017-04-24 12:09:25 +04:00
Monty
5a759d31f7 Changing field::field_name and Item::name to LEX_CSTRING
Benefits of this patch:
- Removed a lot of calls to strlen(), especially for field_string
- Strings generated by parser are now const strings, less chance of
  accidently changing a string
- Removed a lot of calls with LEX_STRING as parameter (changed to pointer)
- More uniform code
- Item::name_length was not kept up to date. Now fixed
- Several bugs found and fixed (Access to null pointers,
  access of freed memory, wrong arguments to printf like functions)
- Removed a lot of casts from (const char*) to (char*)

Changes:
- This caused some ABI changes
  - lex_string_set now uses LEX_CSTRING
  - Some fucntions are now taking const char* instead of char*
- Create_field::change and after changed to LEX_CSTRING
- handler::connect_string, comment and engine_name() changed to LEX_CSTRING
- Checked printf() related calls to find bugs. Found and fixed several
  errors in old code.
- A lot of changes from LEX_STRING to LEX_CSTRING, especially related to
  parsing and events.
- Some changes from LEX_STRING and LEX_STRING & to LEX_CSTRING*
- Some changes for char* to const char*
- Added printf argument checking for my_snprintf()
- Introduced null_clex_str, star_clex_string, temp_lex_str to simplify
  code
- Added item_empty_name and item_used_name to be able to distingush between
  items that was given an empty name and items that was not given a name
  This is used in sql_yacc.yy to know when to give an item a name.
- select table_name."*' is not anymore same as table_name.*
- removed not used function Item::rename()
- Added comparision of item->name_length before some calls to
  my_strcasecmp() to speed up comparison
- Moved Item_sp_variable::make_field() from item.h to item.cc
- Some minimal code changes to avoid copying to const char *
- Fixed wrong error message in wsrep_mysql_parse()
- Fixed wrong code in find_field_in_natural_join() where real_item() was
  set when it shouldn't
- ER_ERROR_ON_RENAME was used with extra arguments.
- Removed some (wrong) ER_OUTOFMEMORY, as alloc_root will already
  give the error.

TODO:
- Check possible unsafe casts in plugin/auth_examples/qa_auth_interface.c
- Change code to not modify LEX_CSTRING for database name
  (as part of lower_case_table_names)
2017-04-23 22:35:46 +03:00
Monty
17a87d6063 MDEV-10139 Support for SEQUENCE objects
Working features:
CREATE OR REPLACE [TEMPORARY] SEQUENCE [IF NOT EXISTS] name
    [ INCREMENT [ BY | = ] increment ]
    [ MINVALUE [=] minvalue | NO MINVALUE ]
    [ MAXVALUE [=] maxvalue | NO MAXVALUE ]
    [ START [ WITH | = ] start ] [ CACHE [=] cache ] [ [ NO ] CYCLE ]
    ENGINE=xxx COMMENT=".."
SELECT NEXT VALUE FOR sequence_name;
SELECT NEXTVAL(sequence_name);
SELECT PREVIOUS VALUE FOR sequence_name;
SELECT LASTVAL(sequence_name);

SHOW CREATE SEQUENCE sequence_name;
SHOW CREATE TABLE sequence_name;
CREATE TABLE sequence-structure ... SEQUENCE=1
ALTER TABLE sequence RENAME TO sequence2;
RENAME TABLE sequence TO sequence2;
DROP [TEMPORARY] SEQUENCE  [IF EXISTS] sequence_names

Missing features
- SETVAL(value,sequence_name), to be used with replication.
- Check replication, including checking that sequence tables are marked
  not transactional.
- Check that a commit happens for NEXT VALUE that changes table data (may
  already work)
- ALTER SEQUENCE. ANSI SQL version of setval.
- Share identical sequence entries to not add things twice to table list.
- testing insert/delete/update/truncate/load data
- Run and fix Alibaba sequence tests (part of mysql-test/suite/sql_sequence)
- Write documentation for NEXT VALUE / PREVIOUS_VALUE
- NEXTVAL in DEFAULT
  - Ensure that NEXTVAL in DEFAULT uses database from base table
- Two NEXTVAL for same row should give same answer.
- Oracle syntax sequence_table.nextval, without any FOR or FROM.
- Sequence tables are treated as 'not read constant tables' by SELECT; Would
  be better if we would have a separate list for sequence tables so that
  select doesn't know about them, except if refereed to with FROM.

Other things done:
- Improved output for safemalloc backtrack
- frm_type_enum changed to Table_type
- Removed lex->is_view and replaced with lex->table_type. This allows
  use to more easy check if item is view, sequence or table.
- Added table flag HA_CAN_TABLES_WITHOUT_ROLLBACK, needed for handlers
  that want's to support sequences
- Added handler calls:
 - engine_name(), to simplify getting engine name for partition and sequences
 - update_first_row(), to be able to do efficient sequence implementations.
 - Made binlog_log_row() global to be able to call it from ha_sequence.cc
- Added handler variable: row_already_logged, to be able to flag that the
  changed row is already logging to replication log.
- Added CF_DB_CHANGE and CF_SCHEMA_CHANGE flags to simplify
  deny_updates_if_read_only_option()
- Added sp_add_cfetch() to avoid new conflicts in sql_yacc.yy
- Moved code for add_table_options() out from sql_show.cc::show_create_table()
- Added String::append_longlong() and used it in sql_show.cc to simplify code.
- Added extra option to dd_frm_type() and ha_table_exists to indicate if
  the table is a sequence. Needed by DROP SQUENCE to not drop a table.
2017-04-07 18:09:56 +04:00
Alexander Barkov
46d076d67a MDEV-10577 sql_mode=ORACLE: %TYPE in variable declarations 2017-04-05 15:02:54 +04:00