Commit graph

991 commits

Author SHA1 Message Date
Sergei Golubchik
f51ecfee93 MDEV-15146 SQLError[4122]: View is not system versioned
don't expand AS OF in views, and, in particular, don't auto-add
AS OF NOW().
2018-02-12 23:43:48 +01:00
Vladislav Vaintroub
6c279ad6a7 MDEV-15091 : Windows, 64bit: reenable and fix warning C4267 (conversion from 'size_t' to 'type', possible loss of data)
Handle string length as size_t, consistently (almost always:))
Change function prototypes to accept size_t, where in the past
ulong or uint were used. change local/member variables to size_t
when appropriate.

This fix excludes rocksdb, spider,spider, sphinx and connect for now.
2018-02-06 12:55:58 +00:00
Alexander Barkov
217fc122c8 Merge remote-tracking branch 'origin/bb-10.2-ext' into 10.3 2018-02-04 18:40:06 +04:00
Monty
d69642dedd Added name to MEM_ROOT for esier debugging
This will make it easier to how memory allocation is done when debugging
with either DBUG or gdb.

Will especially help when debugging stored procedures

Main change is a name argument as second argument to init_alloc_root()
init_sql_alloc()

Other things:
- Added DBUG_ENTER/EXIT to some Virtual_tmp_table functions
2018-02-02 11:08:36 +02:00
Monty
18e22cb69f Removed not used functions and variables 2018-01-30 21:33:56 +02:00
Monty
a7e352b54d Changed database, tablename and alias to be LEX_CSTRING
This was done in, among other things:
- thd->db and thd->db_length
- TABLE_LIST tablename, db, alias and schema_name
- Audit plugin database name
- lex->db
- All db and table names in Alter_table_ctx
- st_select_lex db

Other things:
- Changed a lot of functions to take const LEX_CSTRING* as argument
  for db, table_name and alias. See init_one_table() as an example.
- Changed some function arguments from LEX_CSTRING to const LEX_CSTRING
- Changed some lists from LEX_STRING to LEX_CSTRING
- threads_mysql.result changed because process list_db wasn't always
  correctly updated
- New append_identifier() function that takes LEX_CSTRING* as arguments
- Added new element tmp_buff to Alter_table_ctx to separate temp name
  handling from temporary space
- Ensure we store the length after my_casedn_str() of table/db names
- Removed not used version of rename_table_in_stat_tables()
- Changed Natural_join_column::table_name and db_name() to never return
  NULL (used for print)
- thd->get_db() now returns db as a printable string (thd->db.str or "")
2018-01-30 21:33:55 +02:00
Alexey Botchkov
b4a2baffa8 MDEV-11084 Select statement with partition selection against MyISAM table opens all partitions.
Now we don't open partitions if it was explicitly cpecified.
        ha_partition::m_opened_partition bitmap added to track
        partitions that were actually opened.
2018-01-29 11:01:14 +04:00
Sergei Golubchik
edb6375910 compilation warning on windows 2018-01-17 00:44:11 +01:00
Sergei Golubchik
826f615fc9 MDEV-14788 System versioning cannot be based on local timestamps, as it is now
followup to be81b00c84
Fix updates and deletes too
2018-01-13 02:01:35 +01:00
Sergei Golubchik
a544f920e3 remove "Transaction-based system versioning is EXPERIMENTAL" warning
and the system_versioning_transaction_registry variable.
The user enables transaction registry by specifying BIGINT for
row_start/row_end columns.

check mysql.transaction_registry structure on the first open,
not on startup. Avoid warnings unless transaction_registry
is actually used.
2018-01-13 02:01:34 +01:00
Aleksey Midenkov
c59c1a0736 System Versioning 1.0 pre8
Merge branch '10.3' into trunk
2018-01-10 12:36:55 +03:00
Sergei Golubchik
e6a7457653 SQL: derived, hiding, error messages
Many related changes.

Note that AS OF condition must always be pushed down to physical tables,
it cannot be applied to a derived or a view. Thus:

* no versioning for internal temporary tables, they can never store
  historical data.
* remove special versioning code from mysql_derived_prepare and
  remove ER_VERS_DERIVED_PROHIBITED - derived can have no historical
  data and cannot be prohibited for system versioning related reasons.
* do not expand select list for derived/views with sys vers fields,
  derived/views can never have historical data.
* remove special invisiblity rules for sys vers fields, they are no
  longer needed after the previous change
* remove system_versioning_hide, it lost the meaning after the
  previous change.
* remove ER_VERS_SYSTEM_TIME_CLASH, it's no "clash", the inner
  AS OF clause always wins.
* non-versioned fields in a historical query

reword the warning text, downgrade to note, don't
replace values with NULLs
2018-01-09 14:12:41 +03:00
Aleksey Midenkov
912769b7dc SQL: lower priority of warning in vers_part_rotate() for ALTER [fixes ] 2018-01-06 11:56:51 +03:00
Marko Mäkelä
145ae15a33 Merge bb-10.2-ext into 10.3 2018-01-04 09:22:59 +02:00
Marko Mäkelä
f7fd6ace18 Merge 10.2 into bb-10.2-ext 2018-01-03 15:48:47 +02:00
Marko Mäkelä
d361401bc2 Merge 10.1 into 10.2, with some MDEV-14799 fixups
trx_undo_page_report_modify(): For SPATIAL INDEX, keep logging
updated off-page columns twice, so that
the minimum bounding rectangle (MBR) will be logged.
Avoiding the redundant logging would require larger changes
to the undo log format.

row_build_index_entry_low(): Handle SPATIAL_UNKNOWN more robustly,
by refusing to purge the record from the spatial index.
We can get this code when processing old undo log from 10.2.10 or
10.2.11 (the releases affected by MDEV-14799, which was a regression
from MDEV-14051).
2018-01-03 11:56:24 +02:00
Marko Mäkelä
016caa3d20 Merge 10.0 into 10.1 2018-01-02 21:57:22 +02:00
Marko Mäkelä
51e4650ed0 Merge 5.5 into 10.0 2018-01-02 21:52:46 +02:00
Aleksey Midenkov
8efca72f4a MDEV-14792 INSERT without column list into table with explicit versioning columns produces bad data 2018-01-01 23:37:02 +03:00
Igor Babaev
4f0299f8b3 This is a full cost-based implementation of the optimization that employs
splitting technique for equi-joins of materialized derived tables/views/CTEs.
(see mdev-13369 and mdev-13389).
2017-12-30 12:29:09 -08:00
Oleksandr Byelkin
462808f3b6 MDEV-10657: incorrect result returned with binary protocol (prepared statements)
If translation table present when we materialize the derived table then
change it to point to the materialized table.

Added debug info to see really what happens with what derived.
2017-12-27 16:01:37 +01:00
Monty
9cc7789e90 MDEV 13679 Enabled sequences to be used in DEFAULT
Other changes done to get this to work:
- Added 'internal_tables' to TABLE object to list which sequence tables
  is needed to use the table.
- Mark any expression using DEFAULT() with LEX->default_used.
  This is needed when deciding if we should open internal sequence
  tables when a table is opened (we don't need to open sequence tables
  if the main table is only used with SELECT).
- Create_and_open_temporary_table() can now also open all internal
  sequence tables.
- Added option MYSQL_LOCK_USE_MALLOC to mysql_lock_tables()
  to force memory allocation to be used with malloc instead of
  memroot.
- Added flag to MYSQL_LOCK to remember if allocation was done with
  malloc or memroot (makes code simpler and safer).
- init_one_table_for_prelocking() now takes argument for what lock to
  use instead of it's a routine or something else.
- Renamed prelocking placeholders to make them more understandable as
  they are now used in more code.
- Changed test in check_lock_and_start_stmt() if found table has correct
  locks. The old test didn't work for tables that has lock
  TL_WRITE_ALLOW_WRITE, which is what sequence tables are using.
- Added VCOL_NOT_VIRTUAL option to ensure that sequence functions can't
  be used with virtual columns
- More sequence tests
2017-12-22 14:56:58 +02:00
Aleksey Midenkov
e6c5eb5c58 MDEV-14730 Assertion `m_lock_type == 2' failed in handler::ha_close
SQL: class Table_locker
2017-12-21 18:24:58 +03:00
Sergei Golubchik
368ad5ec36 compiler warning
sql/table.h:1531:17: warning: enumeral and non-enumeral type in conditional expression [-Wextra]
2017-12-21 15:22:38 +03:00
Aleksey Midenkov
36888cc531 MDEV-14686 Server crashes in Item_field::used_tables on 2nd call of SP [fixes ] 2017-12-21 10:25:32 +03:00
Aleksey Midenkov
ee68d019d1 SQL: removed VERS_HIDDEN_FLAG [closes ] 2017-12-19 16:12:56 +03:00
Aleksey Midenkov
b0d9dc43ca SQL: system_versioning_transaction_registry default OFF 2017-12-18 22:17:52 +03:00
Aleksey Midenkov
b55a149194
Timestamp-based versioning for InnoDB [closes ]
* Removed integer_fields check
* Reworked Vers_parse_info::check_sys_fields()
* Misc renames
* versioned as vers_sys_type_t

* Removed versioned_by_sql(), versioned_by_engine()

versioned() works as before;
versioned(VERS_TIMESTAMP) is versioned_by_sql();
versioned(VERS_TRX_ID) is versioned_by_engine().

* create_tmp_table() fix
* Foreign constraints for timestamp-based
* Range auto-specifier fix
* SQL: 1-row partition rotation fix [fixes ]
* Fix 'drop system versioning, algorithm=inplace'
2017-12-18 19:03:51 +03:00
Aleksey Midenkov
4624e565f3 System Versioning 1.0 pre6
Merge remote-tracking branch 'mariadb/bb-10.3-temporal-serg' into trunk
2017-12-15 18:12:18 +03:00
Aleksey Midenkov
73606a3977 System Versioning 1.0 pre5 [closes ]
Merge branch '10.3' into trunk

Both field_visibility and VERS_HIDDEN_FLAG exist independently.

TODO:
VERS_HIDDEN_FLAG should be replaced with SYSTEM_INVISIBLE (or COMPLETELY_INVISIBLE?).
2017-12-15 15:18:59 +03:00
Sachin Setiya
84726906c9 MDEV-10177 Invisible Columns and Invisible Index
Feature Definition:-

This feature adds invisible column functionality to server.
There is 4 level of "invisibility":

1. Not invisible (NOT_INVISIBLE) — Normal columns created by the user

2. A little bit invisible (USER_DEFINED_INVISIBLE) — columns that the
    user has marked invisible. They aren't shown in SELECT * and they
    don't require values in INSERT table VALUE (...). Otherwise
    they behave as normal columns.

3. More invisible (SYSTEM_INVISIBLE) — Can be queried explicitly,
    otherwise invisible from everything. Think ROWID sytem column.
    Because they're invisible from ALTER TABLE and from CREATE TABLE
    they cannot be created or dropped, they're created by the system.
    User cant not create a column name which is same as of
    SYSTEM_INVISIBLE.

4. Very invisible (COMPLETELY_INVISIBLE) — as above, but cannot be
    queried either. They can only show up in EXPLAIN EXTENDED (might
    be possible for a very invisible indexed virtual column) but
    otherwise they don't exist for the user.If user creates a columns
    which has same name as of COMPLETELY_INVISIBLE then
    COMPLETELY_INVISIBLE column is renamed again. So it is completely
    invisible from user.

Invisible Index(HA_INVISIBLE_KEY):-
   Creation of invisible columns require a new type of index which
   will be only visible to system. User cant see/alter/create/delete
   this index. If user creates a index which is same name as of
   invisible index then it will be renamed.

Syntax Details:-

  Only USER_DEFINED_INVISIBLE column can be created by user. This
  can be created by adding INVISIBLE suffix after column definition.

  Create table t1( a int invisible, b int);

Rules:-
  There are some rules/restrictions related to use of invisible columns
  1. All the columns in table cant be invisible.
     Create table t1(a int invisible); \\error
     Create table t1(a int invisible, b int invisble); \\error
  2. If you want invisible column to be NOT NULL then you have to supply
     Default value for the column.
     Create table t1(a int, b int not null); \\error
  3. If you create a view/create table with select * then this wont copy
     invisible fields. So newly created view/table wont have any invisible
     columns.
     Create table t2 as select * from t1;//t2 wont have t1 invisible column
     Create view v1 as select * from t1;//v1 wont have t1 invisible column
  4. Invisibility wont be forwarded to next table in any case of create
     table/view as select */(a,b,c) from table.
     Create table t2 as select a,b,c from t1; // t2 will have t1 invisible
                           // column(b), but this wont be invisible in t2
     Create view v1 as select a,b,c from t1; // v1 will have t1 invisible
                           // column(b), but this wont be invisible in v1

Implementation Details:-
  Parsing:- INVISIBLE_SYM is added into vcol_attribute(so its like unique
      suffix), It is also added into keyword_sp_not_data_type so that table
      can have column with name invisible.
  Implementation detail is given by each modified function/created function.
   (Some function are left as they were self explanatory)
   (m= Modified, n= Newly Created)

  mysql_prepare_create_table(m):- Extra checks for invisible columns are
  added. Also some DEBUG_EXECUTE_IF are also added for test cases.

  mysql_prepare_alter_table(m):- Now this will drop all the
  COMPLETELY_INVISIBLE column and HA_INVISIBLE_KEY index. Further
  Modifications are made to stop drop/change/delete of SYSTEM_INVISIBLE
  column.

  build_frm_image(m):- Now this allows incorporating field_visibility
  status into frm image. To remain compatible with old frms
  field_visibility info will be only written when any of the field is
  not NOT_INVISIBLE.

  extra2_write_additional_field_properties(n):- This will write field
  visibility info into buffer. We first write EXTRA2_FIELD_FLAGS into
  buffer/frm , then each next char will have field_visibility for each
  field.

  init_from_binary_frm_image(m):- Now if we get EXTRA2_FIELD_FLAGS,
  then we will read the next n(n= number of fields) chars and set the
  field_visibility. We also increment
  thd->status_var.feature_invisible_columns. One important thing to
  note if we find out that key contains a field whose visibility is
  > USER_DEFINED_INVISIBLE then , we declare this key as invisible
  key.

  sql_show.cc is changed accordingly to make show table, show keys
  correct.

  mysql_insert(m):- If we get to know that we are doing insert in
  this way insert into t1 values(1,1); without explicitly specifying
  columns, then we check for if we have invisible fields if yes then
  we reset the whole record, Why ? Because first we want hidden columns
  to get default/null value. Second thing auto_increment has property
  no default and no null which voilates invisible key rule 2, And
  because of this it was giving error. Reseting table->record[0]
  eliminates this issue. More info put breakpoint on handler::write_row
  and see auto_increment value.

  fill_record(m):- we continue loop if we find invisible column because
  this is already reseted/will get its value if it is default.

Test cases:- Since we can not directly add > USER_DEFINED_INVISIBLE
  column then I have debug_dbug to create it in mysql_prepare_create_table.

  Patch Credit:- Serg Golubchik
2017-12-15 02:41:52 +05:30
Sergei Golubchik
e77080c7d5 if a table is partitioned by system_time, its partitions are not versioned
they store history and the history does not have history
2017-12-13 21:44:30 +01:00
Aleksey Midenkov
8e8363bb75 SQL: VIEW system fields propagation removed [fixes ] 2017-12-13 18:23:32 +03:00
Aleksey Midenkov
79dd77e6ae System Versioning 1.0 pre3
Merge branch '10.3' into trunk
2017-12-11 15:43:41 +03:00
Aleksey Midenkov
37adb4c066 SQL: vers_update_user_fields() mem_root fix [ bug 12]
Related to  bug 3.
2017-12-08 16:26:18 +03:00
Sergei Golubchik
ea1ccfa500 SQL: regression fix: make NOW a valid identifier again []
* again, as in 10.2, NOW is a keyword only if followed by parentheses
* use AS OF CURRENT_TIMESTAMP or AS OF NOW()
* AS OF CURRENT_TIMESTAMP and AS OF NOW() mean AS OF NOW(6),
  not AS OF NOW(0), (same behavior as in a DEFAULT clause)
2017-12-08 16:24:56 +03:00
Aleksey Midenkov
56adced376 SQL,IB: REPLACE semantics [ bug 8] 2017-12-04 13:19:37 +03:00
Kentoku SHIBA
e53ef202bd Adding direct update/delete to the server and to the partition engine.
Add support for direct update and direct delete requests for spider.
A direct update/delete request handles all qualified rows in a single
operation rather than one row at a time.

Contains Spiral patches:
006_mariadb-10.2.0.direct_update_rows.diff      MDEV-7704
008_mariadb-10.2.0.partition_direct_update.diff MDEV-7706
010_mariadb-10.2.0.direct_update_rows2.diff     MDEV-7708
011_mariadb-10.2.0.aggregate.diff               MDEV-7709
027_mariadb-10.2.0.force_bulk_update.diff       MDEV-7724
061_mariadb-10.2.0.mariadb-10.1.8.diff          MDEV-12870

- The differences compared to the original patches:
  - Most of the parameters of the new functions are unnecessary.  The
    unnecessary parameters have been removed.
  - Changed bit positions for new handler flags upon consideration of
    handler flags not needed by other Spiral patches and handler flags
    merged from MySQL.
  - Added info_push() (Was originally part of bulk access patch)
  - Didn't include code related to handler socket
  - Added HA_CAN_DIRECT_UPDATE_AND_DELETE

Original author: Kentoku SHIBA
First reviewer:  Jacob Mathew
Second reviewer: Michael Widenius
2017-12-03 13:58:36 +02:00
Aleksey Midenkov
2305666a05 SQL: insert delayed fix [ bug 3]
Affected tests (forced mode): rpl.rpl_timezone
2017-11-29 11:58:18 +03:00
Aleksey Midenkov
8d548f8e86 SQL: fill_record() field-value inconsistency fix [ bug 2]
Affected tests (forced mode): binlog_encryption.encrypted_slave
2017-11-29 10:48:14 +03:00
Aleksey Midenkov
ababd6a935 SQL: destroy Vers_min_max_stats [] 2017-11-27 18:28:56 +03:00
Marko Mäkelä
0b89a42ffc Remove the flag vers_update_trt
THD::vers_update_trt, trx_t::vers_update_trt, trx_savept_t::vers_update_trt:
Remove. Instead, determine from trx_t::mod_tables whether versioned
columns were affected by the transaction.

handlerton::prepare_commit_versioned: Replaces vers_get_trt_data.
Return the transaction start ID and also the commit ID, in case
the transaction modified any system-versioned columns (0 if not).

TR_table::store_data(): Remove (merge with update() below).

TR_table::update(): Add the parameters start_id, end_id.

ha_commit_trans(): Remove a condition on SQLCOM_ALTER_TABLE.
If we need something special for ALTER TABLE...ALGORITHM=INPLACE,
that can be done inside InnoDB by modifying trx_t::mod_tables.

innodb_prepare_commit_versioned(): Renamed from innodb_get_trt_data().
Check trx_t::mod_tables to see if any changes to versioned columns
are present.

trx_mod_table_time_t: A pair of logical timestamps, replacing the
undo_no_t in trx_mod_tables_t. Keep track of not only the first
modification to a persistent table in each transaction, but also
the first modification of a versioned column in a table.

dtype_t, dict_col_t: Add the accessor is_any_versioned(), to check
if the type refers to a system-versioned user or system column.

upd_t::affects_versioned(): Check if an update affects a versioned
column.

trx_undo_report_row_operation(): If a versioned column is affected
by the update, invoke trx_mod_table_time_t::set_versioned().

trx_rollback_to_savepoint_low(): If all changes to versioned columns
were rolled back, invoke trx_mod_table_time_t::rollback_versioned(),
so that trx_mod_table_time_t::is_versioned() will no longer hold.
2017-11-27 15:07:33 +03:00
Eugene Kosov
0cdc1164dc SQL, IB: various refactoring [] 2017-11-27 15:07:32 +03:00
Aleksey Midenkov
6e0b2c7fe0 System Versioning 1.0pre2
Merge branch '10.3' into trunk
2017-11-23 19:41:44 +03:00
Aleksey Midenkov
cbe93291e4 SQL,IB: add auto_inc copy mode fix []
Related to .

Extra update of sys_trx_start removed [].
2017-11-23 18:31:55 +03:00
Aleksey Midenkov
5994b6873c Daemon: TRT check doesn't abort [fixes ]
Added schema check logging messages.
2017-11-22 22:20:14 +03:00
Aleksey Midenkov
a4439fef96 SQL: don't update TRT when altered [, ]
Tests affected: main.mysqlcheck
2017-11-17 14:28:45 +03:00
Michael Widenius
5b3da95bf3 Simplify fn_rext
- Change interface according to real usage
- Fix comment
- Rename to fn_frm_ext
2017-11-17 07:30:05 +02:00
Aleksey Midenkov
72274c10f0 SQL: TRT failure fixes [closes ]
Unclosed stmt transaction
TRT failure in inplace

Related to 
2017-11-15 00:22:12 +03:00
Eugene Kosov
08099bc05a Daemon: check TRT schema [closes ] 2017-11-15 00:22:11 +03:00