Commit graph

428 commits

Author SHA1 Message Date
Monty
36cdd5c3cd Optimize usage of c_ptr(), c_ptr_quick() and String::alloc()
The problem was that when one used String::alloc() to allocate a string,
the String ensures that there is space for an extra NULL byte in the
buffer and if not, reallocates the string. This is a problem with the
String::set_int() that calls alloc(21), which forces extra
malloc/free calls to happen.

- We do not anymore re-allocate String if alloc() is called with the
  Allocated_length. This reduces number of malloc() allocations,
  especially one big re-allocation in Protocol::send_result_Set_metadata()
  for almost every query that produced a result to the connnected client.
- Avoid extra mallocs when using LONGLONG_BUFFER_SIZE
  This can now be done as alloc() doesn't increase buffers if new length is
  not bigger than old one.
- c_ptr() is redesigned to be safer (but a bit longer) than before.
- Remove wrong usage of c_ptr_quick()
  c_ptr_quick() was used in many cases to get the pointer to the used
  buffer, even when it didn't need to be \0 terminated. In this case
  ptr() is a better substitute.
  Another problem with c_ptr_quick() is that it did not guarantee that
  the string would be \0 terminated.
- item_val_str(), an API function not used currently by the server,
  now always returns a null terminated string (before it didn't always
  do that).
- Ensure that all String allocations uses STRING_PSI_MEMORY_KEY. The old
  mixed usage of performance keys caused assert's when String buffers
  where shrunk.
- Binary_string::shrink() is simplifed
- Fixed bug in String(const char *str, size_t len, CHARSET_INFO *cs) that
  used Binary_string((char *) str, len) instead of Binary_string(str,len).
- Changed argument to String() creations and String.set() functions to use
  'const char*' instead of 'char*'. This ensures that Alloced_length is
  not set, which gives safety against someone trying to change the
  original string. This also would allow us to use !Alloced_length in
  c_ptr() if needed.
- Changed string_ptr_cmp() to use memcmp() instead of c_ptr() to avoid
  a possible malloc during string comparision.
2021-05-19 22:27:27 +02:00
Sergei Golubchik
25d9d2e37f Merge branch 'bb-10.4-release' into bb-10.5-release 2021-02-15 16:43:15 +01:00
Nikita Malyavin
21809f9a45 MDEV-17556 Assertion `bitmap_is_set_all(&table->s->all_set)' failed
The assertion failed in handler::ha_reset upon SELECT under
READ UNCOMMITTED from table with index on virtual column.

This was the debug-only failure, though the problem is mush wider:
* MY_BITMAP is a structure containing my_bitmap_map, the latter is a raw
 bitmap.
* read_set, write_set and vcol_set of TABLE are the pointers to MY_BITMAP
* The rest of MY_BITMAPs are stored in TABLE and TABLE_SHARE
* The pointers to the stored MY_BITMAPs, like orig_read_set etc, and
 sometimes all_set and tmp_set, are assigned to the pointers.
* Sometimes tmp_use_all_columns is used to substitute the raw bitmap
 directly with all_set.bitmap
* Sometimes even bitmaps are directly modified, like in
TABLE::update_virtual_field(): bitmap_clear_all(&tmp_set) is called.

The last three bullets in the list, when used together (which is mostly
always) make the program flow cumbersome and impossible to follow,
notwithstanding the errors they cause, like this MDEV-17556, where tmp_set
pointer was assigned to read_set, write_set and vcol_set, then its bitmap
was substituted with all_set.bitmap by dbug_tmp_use_all_columns() call,
and then bitmap_clear_all(&tmp_set) was applied to all this.

To untangle this knot, the rule should be applied:
* Never substitute bitmaps! This patch is about this.
 orig_*, all_set bitmaps are never substituted already.

This patch changes the following function prototypes:
* tmp_use_all_columns, dbug_tmp_use_all_columns
 to accept MY_BITMAP** and to return MY_BITMAP * instead of my_bitmap_map*
* tmp_restore_column_map, dbug_tmp_restore_column_maps to accept
 MY_BITMAP* instead of my_bitmap_map*

These functions now will substitute read_set/write_set/vcol_set directly,
and won't touch underlying bitmaps.
2021-01-27 00:50:55 +10:00
Marko Mäkelä
50a11f396a Merge 10.4 into 10.5 2020-08-01 14:42:51 +03:00
Nikita Malyavin
34f2be3b29 MDEV-16023 Unfortunate error message WARN_VERS_PART_FULL (partition <name> is full) when rotation time for the last interval passed
* remove one case of WARN_VERS_PART_FULL
2020-07-29 14:32:07 +10:00
Marko Mäkelä
6877ef9a7c Merge 10.4 into 10.5 2020-06-05 20:36:43 +03:00
Aleksey Midenkov
05693cf214 MDEV-22112 Assertion `tab_part_info->part_type == RANGE_PARTITION || tab_part_info->part_type == LIST_PARTITION' failed in prep_alter_part_table
Incorrect syntax for SYSTEM_TIME partition. work_part_info is detected
as HASH partition. We cannot add partition of different type neither
we cannot reorganize SYSTEM_TIME into/from different type
partitioning.

The sidefix for version until 10.5 corrects the message:
"For LIST partitions each partition must be defined"
2020-06-04 12:12:49 +03:00
Aleksey Midenkov
73aa78ea9d MDEV-22155 ALTER add default history partitions name clash on non-default partitions
If any of default names clashes with existing names find next large
enough name gap for the requested number of partitions.
2020-04-27 16:36:03 +03:00
Aleksey Midenkov
139117528a MDEV-22153 ALTER add default history partitions makes table inaccessible
ADD default history partitions generates wrong partition name,
f.ex. p2 instead of p1. Gap in sequence of partition names leads to
ha_partition::open_read_partitions() fail on inexistent name.

Manual fixing such broken table requires:

1. create empty table by any name (t_empty) with correct number
of partitions;
2. stop the server;
3. rename data files (.myd, .myi or .ibd) of broken table to t_empty
fixing the partition sequence (#p2 to #p1, #p3 to #p2);
4. start the server;
5. drop the broken table;
6. rename t_empty to correct table name.
2020-04-06 06:26:46 +03:00
Alexander Barkov
a1e330de5a MDEV-21743 Split up SUPER privilege to smaller privileges 2020-03-10 23:49:47 +04:00
Sergei Golubchik
c1c5222cae cleanup: PSI key is *always* the first argument 2020-03-10 19:24:23 +01:00
Sergei Golubchik
7c58e97bf6 perfschema memory related instrumentation changes 2020-03-10 19:24:22 +01:00
Sergei Golubchik
98adcffe46 Revert "MDEV-17554 Auto-create new partition for system versioned tables with history partitioned by INTERVAL/LIMIT"
This reverts commit 9894751a2a.
This reverts commit f707c83fff.
2020-02-27 10:30:21 +01:00
Aleksey Midenkov
9894751a2a Compilation fix 2020-02-25 20:58:03 +03:00
Aleksey Midenkov
f707c83fff MDEV-17554 Auto-create new partition for system versioned tables with history partitioned by INTERVAL/LIMIT
When there are E empty partitions left, auto-create N new empty
partitions for SYSTEM_TIME partitioning rotated by INTERVAL/LIMIT and
marked by AUTO_INCREMENT keyword. Syntax change: AUTO_INCREMENT
keyword (or shorter AUTO may be used instead) after LIMIT/INTERVAL
clause.

CREATE OR REPLACE TABLE t (x INT) WITH SYSTEM VERSIONING
PARTITION BY SYSTEM_TIME LIMIT 100000 AUTO_INCREMENT;

CREATE OR REPLACE TABLE t (x INT) WITH SYSTEM VERSIONING
PARTITION BY SYSTEM_TIME INTERVAL 1 WEEK AUTO_INCREMENT;

The current revision implements hard-coded values of 1 for E and N. As
well as auto-creation threshold MinInterval = 1 hour, MinLimit = 1000.

The name for newly added partition will be first chosen as "pX", where
X is partition number and "p" is hard-coded name prefix. If this name
is already occupied, the X will be incremented until the resulting
name will be free to use.

ALTER TABLE ADD PARTITION is now always fast. If there some history
partition overflow occurs manual ALTER TABLE REBUILD PARTITION is
needed.
2020-02-25 15:43:23 +03:00
Aleksey Midenkov
9ed8d364cd MDEV-17554 history partitioning cleanups
* Fixed missed warning on condition boundary
* REORGANIZE cases
* vers_utils.h removed
* test cases cleanup
2019-12-03 11:53:25 +03:00
Aleksey Midenkov
777b399618 MDEV-19903 Setup default partitions for system versioning
Implement syntax like:

create table t1 (x int) with system versioning partition by system_time;

which will create 1 history partition and 1 current partition.
Also it is possible to specify the number of history partitions:

create table t1 (x int) with system versioning partition by system_time partitions 5;

which will create 4 history partitions (and 1 current partition).

Tests:
partition.test cases are duplicated where it is appropriate for default partitions.
partition_rotation.test cases are replaced by default partitions where possible.
2019-11-22 18:34:28 +03:00
Aleksey Midenkov
db56543993 Compilation fix with partitioning off (MDEV-17553) 2019-11-07 20:25:20 +03:00
Aleksey Midenkov
1e73d7d6c6 MDEV-17553 Enable setting start datetime for interval partitioned history of system versioned tables
* Explicit STARTS syntax
* SHOW CREATE
* Default STARTS rounding depending on INTERVAL type
* Warn when STARTS timestamp is later than query time
* Fix uninitialized Lex->create_last_non_select_table under
  mysql_unpack_partition()

Default STARTS rounding depending on INTERVAL type

If STARTS clause is omitted, default one is assigned with value
derived from query timestamp. The rounding is done on STARTS value
depending on INTERVAL type:

SECOND: no rounding is done;
MINUTE: timestamp seconds is set to 0;
HOUR: timestamp seconds and minutes are set to 0;
DAY, WEEK, MONTH and YEAR: timestamp seconds, minutes and hours are
set to 0 (the date of rotation is kept as current date).
2019-11-07 19:24:06 +03:00
Marko Mäkelä
780d2bb8a7 Merge 10.4 into 10.5 2019-09-06 14:25:20 +03:00
Aleksey Midenkov
597b070f82 Compilation fix
Caused by:
MDEV-18501 Partition pruning doesn't work for historical queries (cleanup)
2019-09-01 18:00:04 +03:00
Aleksey Midenkov
c3f35ea55a MDEV-18501 Partition pruning doesn't work for historical queries (refactoring)
SYSTEM_TYPE partitioning: COLUMN properties removed. Partitioning is
now pure RANGE based on UNIX_TIMESTAMP(row_end).

DECIMAL type is now allowed as RANGE partitioning, we can partition by
UNIX_TIMESTAMP() (but not for DATETIME which depends on local timezone
of course).
2019-09-01 14:04:25 +03:00
Aleksey Midenkov
a3e49c0d36 MDEV-18501 Partition pruning doesn't work for historical queries (cleanup)
Cleanup removes useless allocation.
2019-09-01 14:04:24 +03:00
Alexander Barkov
afe6eb499d Revert "MDEV-20342 Turn Field::flags from a member to a method"
This reverts commit e86010f909.

Reverting on Monty's request, as this change makes merging
things from 10.5 to 10.2 much harder.
2019-08-14 20:27:00 +04:00
Alexander Barkov
e86010f909 MDEV-20342 Turn Field::flags from a member to a method 2019-08-14 13:33:01 +04:00
Alexander Barkov
d1d6fe9abf Using more of Sql_mode_save. Adding a similar class for THD::abort_on_warnings. 2019-05-28 10:26:08 +04:00
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Vicențiu Ciorbaru
cb248f8806 Merge branch '5.5' into 10.1 2019-05-11 22:19:05 +03:00
Vicențiu Ciorbaru
5543b75550 Update FSF Address
* Update wrong zip-code
2019-05-11 21:29:06 +03:00
Sergei Golubchik
b64fde8f38 Merge branch '10.2' into 10.3 2019-03-17 13:06:41 +01:00
Sergei Golubchik
0508d327ae Merge branch '10.1' into 10.2 2019-03-15 21:00:41 +01:00
Sergei Golubchik
3d2d060b62 fix gcc 8 compiler warnings
There were two newly enabled warnings:
1. cast for a function pointers. Affected sql_analyse.h, mi_write.c
   and ma_write.cc, mf_iocache-t.cc, mysqlbinlog.cc, encryption.cc, etc

2. memcpy/memset of nontrivial structures. Fixed as:
* the warning disabled for InnoDB
* TABLE, TABLE_SHARE, and TABLE_LIST got a new method reset() which
  does the bzero(), which is safe for these classes, but any other
  bzero() will still cause a warning
* Table_scope_and_contents_source_st uses `TABLE_LIST *` (trivial)
  instead of `SQL_I_List<TABLE_LIST>` (not trivial) so it's safe to
  bzero now.
* added casts in debug_sync.cc and sql_select.cc (for JOIN)
* move assignment method for MDL_request instead of memcpy()
* PARTIAL_INDEX_INTERSECT_INFO::init() instead of bzero()
* remove constructor from READ_RECORD() to make it trivial
* replace some memcpy() with c++ copy assignments
2019-03-14 16:33:17 +01:00
Andrei Elkin
69e552279c merge fixes: sql_table.cc and partition_info.cc (caused by ef4ccb6ce2) done 2019-02-07 17:12:49 +02:00
Sergei Golubchik
6bb11efa4a Merge branch '10.2' into 10.3 2019-01-03 13:09:41 +01:00
Sergei Golubchik
67240858b2 Merge branch '10.1' into 10.2 2018-12-30 18:30:29 +01:00
Sergei Golubchik
aeefd26ecb Merge branch '10.0' into 10.1 2018-12-29 23:44:45 +01:00
Varun Gupta
7e606a2d5c MDEV-17589: Stack-buffer-overflow with indexed varchar (utf8) field
Create a new constant MAX_DATA_LENGTH_FOR_KEY.
Replace the value of MAX_KEY_LENGTH to also include the LENGTH and NULL BYTES
of a field.
2018-12-19 10:38:46 +05:30
Marko Mäkelä
8e613458e1 Fix cmake -DWITH_PARTITION_STORAGE_ENGINE:BOOL=OFF
This is a backport of a part of
commit 18455ec3f1
from 10.1.
2018-12-13 12:37:40 +02:00
Marko Mäkelä
839cf16bb2 Merge 10.2 into 10.3 2018-12-12 13:46:06 +02:00
Marko Mäkelä
db1210f939 Merge 10.1 into 10.2 2018-12-12 12:13:43 +02:00
Marko Mäkelä
f77f8f6d1a Merge 10.0 into 10.1 2018-12-12 10:48:53 +02:00
Varun Gupta
4886d14827 MDEV-17032: Estimates are higher for partitions of a table with @@use_stat_tables= PREFERABLY
The problem here is EITS statistics does not calculate statistics for the partitions of the table.
So a temporary solution would be to not read EITS statistics for partitioned tables.

Also disabling reading of EITS for columns that participate in the partition list of a table.
2018-12-07 19:59:45 +05:30
Jacob Mathew
95e9c7054f MDEV-16101: ADD PARTITION on table partitioned by list does not work with more than 32 list values.
This problem occured because the reorganization of the list of values when the
number of elements exceeds 32 was not handled correctly.  I have fixed the
problem by fixing the way that the list values are reorganized when the number
of list values exceeds 32.

Author:
  Jacob Mathew.

Reviewer:
  Alexey Botchkov.

Merged From:
  Commit 8e01598 on branch bb-10.3-MDEV-16101
2018-05-14 16:30:25 -07:00
Jacob Mathew
86f9932e80 MDEV-16101: ADD PARTITION on table partitioned by list does not work with more than 32 list values.
This problem occured because the reorganization of the list of values when the
number of elements exceeds 32 was not handled correctly.  I have fixed the
problem by fixing the way that the list values are reorganized when the number
of list values exceeds 32.

Author:
  Jacob Mathew.

Reviewer:
  Alexey Botchkov.

Merged From:
  Branch bb-10.3-MDEV-16101
2018-05-14 15:12:40 -07:00
Jacob Mathew
8e01598620 MDEV-16101: ADD PARTITION on table partitioned by list does not work with more than 32 list values.
This problem occured because the reorganization of the list of values when the
number of elements exceeds 32 was not handled correctly.  I have fixed the
problem by fixing the way that the list values are reorganized when the number
of list values exceeds 32.

Author:
  Jacob Mathew.

Reviewer:
  Alexey Botchkov.
2018-05-14 13:25:41 -07:00
Sergei Golubchik
a4272bf154 versioning: use @@timestamp
Don't use hidden system time in versioning,
but keep the system time logic in THD
to workaround low-res system clock and
replication not versioned to versioned.

This reverts MDEV-14788 (System versioning cannot
be based on local timestamps, as it is now).
Versioning is based on local timestamps again,
but timestamps are protected by MDEV-15923
(option to control who can set session @@timestamp).
2018-05-12 10:16:46 +02:00
Michael Widenius
062a3176e7 Remove mem_alloc_error()
As thd->alloc() and new automatically calls my_error(ER_OUTOFMEORY)
there is no reason to call mem_alloc_error()

Other things:
- Fixed bug in mysql_unpack_partition() where lex.part_info was
  changed even if it would be a null pointer
2018-05-07 00:07:33 +03:00
Monty
30ebc3ee9e Add likely/unlikely to speed up execution
Added to:
- if (error)
- Lex
- sql_yacc.yy and sql_yacc_ora.yy
- In header files to alloc() calls
- Added thd argument to thd_net_is_killed()
2018-05-07 00:07:32 +03:00
Sergei Golubchik
35678c9572 dead code - related to vtmd
(will be added back when it'll be used)
2018-04-10 13:12:36 +02:00