On 64-bit Windows: querying MERGE table with keys may cause
server crash.The problem is generic and may affect any statement
accessing MERGE table cardinality values.
When MERGE engine was copying cardinality statistics, it was
using incorrect size of element in cardinality statistics array
(sizeof(ptr)==8 instead of sizeof(ulong)==4), causing access
of memory beyond of the allocated bounds.
mysqld is optimized for the default
case (up to 64-indices); for a greater
number of indices it goes through a
different code path. As that code-path
is a compile-time option and can not
easily be covered in standard tests,
bitrot occurred. key-fields need an
explicit initialization in the non-
optimized case; this setup was
presumably not added when a new key-
vector was added.
Changeset adds the necessary
initialisations.
No test case added due to dependence
on compile-time option.
Took the Xfree implementation (based on the same rewrite as the NDB one)
and added it instead of the current implementation.
Added a macro to make the calls to MD5 more streamlined.
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
(server crash)
Altering a table with fulltext index[es] which use
pluggable fulltext parser may cause server crash
in debug builds.
The problem was that ALTER TABLE code wrongly assigned
fulltext parser name.
Also fixed that altering a table with fulltext index[es]
leave stale fulltext parser locks, which prevent
fulltext parsers from being uninstalled after
ALTER TABLE.
column
When the storage engine uses secondary keys clustered with the primary key MySQL was
adding the primary key parts to each secondary key.
In doing so it was not checking whether the index was on full columns and this
resulted in the secondary keys being added to the list of covering keys even if
they have partial columns.
Fixed by not adding a primary key part to the list of columns that can be used
for index read of the secondary keys when the primary key part is a partial key part.
``FLUSH TABLES WITH READ LOCK''
Concurrent execution of 1) multitable update with a
NATURAL/USING join and 2) a such query as "FLUSH TABLES
WITH READ LOCK" or "ALTER TABLE" of updating table led
to a server crash.
The mysql_multi_update_prepare() function call is optimized
to lock updating tables only, so it postpones locking to
the last, and if locking fails, it does cleanup of modified
syntax structures and repeats a query analysis. However,
that cleanup procedure was incomplete for NATURAL/USING join
syntax data: 1) some Field_item items pointed into freed
table structures, and 2) the TABLE_LIST::join_columns fields
was not reset.
Major change:
short-living Field *Natural_join_column::table_field has
been replaced with long-living Item*.
This patch also fixes bugs 36963 and 35600.
- In many places a view was confused with an anonymous derived
table, i.e. access checking was skipped. Fixed by introducing a
predicate to tell the difference between named and anonymous
derived tables.
- When inserting fields for "SELECT * ", there was no
distinction between base tables and views, where one should be
made. View privileges are checked elsewhere.
Tables in the table definition cache are keeping a cache buffer for blob
fields which can consume a lot of memory.
This patch introduces a maximum size threshold for these buffers.
When openning a table with unsupported (disabled or not compiled)
storage engine, confusing error message is returned.
Return better error message when we're attempting to open a table
that uses unsupported engine.
cause ROLLBACK of statement", part 1. Review fixes.
Do not send OK/EOF packets to the client until we reached the end of
the current statement.
This is a consolidation, to keep the functionality that is shared by all
SQL statements in one place in the server.
Currently this functionality includes:
- close_thread_tables()
- log_slow_statement().
After this patch and the subsequent patch for Bug#12713, it shall also include:
- ha_autocommit_or_rollback()
- net_end_statement()
- query_cache_end_of_result().
In future it may also include:
- mysql_reset_thd_for_next_command().
corrupts a MERGE table
Bug 26867 - LOCK TABLES + REPAIR + merge table result in
memory/cpu hogging
Bug 26377 - Deadlock with MERGE and FLUSH TABLE
Bug 25038 - Waiting TRUNCATE
Bug 25700 - merge base tables get corrupted by
optimize/analyze/repair table
Bug 30275 - Merge tables: flush tables or unlock tables
causes server to crash
Bug 19627 - temporary merge table locking
Bug 27660 - Falcon: merge table possible
Bug 30273 - merge tables: Can't lock file (errno: 155)
The problems were:
Bug 26379 - Combination of FLUSH TABLE and REPAIR TABLE
corrupts a MERGE table
1. A thread trying to lock a MERGE table performs busy waiting while
REPAIR TABLE or a similar table administration task is ongoing on
one or more of its MyISAM tables.
2. A thread trying to lock a MERGE table performs busy waiting until all
threads that did REPAIR TABLE or similar table administration tasks
on one or more of its MyISAM tables in LOCK TABLES segments do UNLOCK
TABLES. The difference against problem #1 is that the busy waiting
takes place *after* the administration task. It is terminated by
UNLOCK TABLES only.
3. Two FLUSH TABLES within a LOCK TABLES segment can invalidate the
lock. This does *not* require a MERGE table. The first FLUSH TABLES
can be replaced by any statement that requires other threads to
reopen the table. In 5.0 and 5.1 a single FLUSH TABLES can provoke
the problem.
Bug 26867 - LOCK TABLES + REPAIR + merge table result in
memory/cpu hogging
Trying DML on a MERGE table, which has a child locked and
repaired by another thread, made an infinite loop in the server.
Bug 26377 - Deadlock with MERGE and FLUSH TABLE
Locking a MERGE table and its children in parent-child order
and flushing the child deadlocked the server.
Bug 25038 - Waiting TRUNCATE
Truncating a MERGE child, while the MERGE table was in use,
let the truncate fail instead of waiting for the table to
become free.
Bug 25700 - merge base tables get corrupted by
optimize/analyze/repair table
Repairing a child of an open MERGE table corrupted the child.
It was necessary to FLUSH the child first.
Bug 30275 - Merge tables: flush tables or unlock tables
causes server to crash
Flushing and optimizing locked MERGE children crashed the server.
Bug 19627 - temporary merge table locking
Use of a temporary MERGE table with non-temporary children
could corrupt the children.
Temporary tables are never locked. So we do now prohibit
non-temporary chidlren of a temporary MERGE table.
Bug 27660 - Falcon: merge table possible
It was possible to create a MERGE table with non-MyISAM children.
Bug 30273 - merge tables: Can't lock file (errno: 155)
This was a Windows-only bug. Table administration statements
sometimes failed with "Can't lock file (errno: 155)".
These bugs are fixed by a new implementation of MERGE table open.
When opening a MERGE table in open_tables() we do now add the
child tables to the list of tables to be opened by open_tables()
(the "query_list"). The children are not opened in the handler at
this stage.
After opening the parent, open_tables() opens each child from the
now extended query_list. When the last child is opened, we remove
the children from the query_list again and attach the children to
the parent. This behaves similar to the old open. However it does
not open the MyISAM tables directly, but grabs them from the already
open children.
When closing a MERGE table in close_thread_table() we detach the
children only. Closing of the children is done implicitly because
they are in thd->open_tables.
For more detail see the comment at the top of ha_myisammrg.cc.
Changed from open_ltable() to open_and_lock_tables() in all places
that can be relevant for MERGE tables. The latter can handle tables
added to the list on the fly. When open_ltable() was used in a loop
over a list of tables, the list must be temporarily terminated
after every table for open_and_lock_tables().
table_list->required_type is set to FRMTYPE_TABLE to avoid open of
special tables. Handling of derived tables is suppressed.
These details are handled by the new function
open_n_lock_single_table(), which has nearly the same signature as
open_ltable() and can replace it in most cases.
In reopen_tables() some of the tables open by a thread can be
closed and reopened. When a MERGE child is affected, the parent
must be closed and reopened too. Closing of the parent is forced
before the first child is closed. Reopen happens in the order of
thd->open_tables. MERGE parents do not attach their children
automatically at open. This is done after all tables are reopened.
So all children are open when attaching them.
Special lock handling like mysql_lock_abort() or mysql_lock_remove()
needs to be suppressed for MERGE children or forwarded to the parent.
This depends on the situation. In loops over all open tables one
suppresses child lock handling. When a single table is touched,
forwarding is done.
Behavioral changes:
===================
This patch changes the behavior of temporary MERGE tables.
Temporary MERGE must have temporary children.
The old behavior was wrong. A temporary table is not locked. Hence
even non-temporary children were not locked. See
Bug 19627 - temporary merge table locking.
You cannot change the union list of a non-temporary MERGE table
when LOCK TABLES is in effect. The following does *not* work:
CREATE TABLE m1 ... ENGINE=MRG_MYISAM ...;
LOCK TABLES t1 WRITE, t2 WRITE, m1 WRITE;
ALTER TABLE m1 ... UNION=(t1,t2) ...;
However, you can do this with a temporary MERGE table.
You cannot create a MERGE table with CREATE ... SELECT, neither
as a temporary MERGE table, nor as a non-temporary MERGE table.
CREATE TABLE m1 ... ENGINE=MRG_MYISAM ... SELECT ...;
Gives error message: table is not BASE TABLE.
partitioned table
Trying INSERT DELAYED on a partitioned table, that has not been
used right before, crashes the server. When a table is used for
select or update, it is kept open for some time. This period I
mean with "right before".
Information about partitioning of a table is stored in form of
a string in the .frm file. Parsing of this string requires a
correctly set up lexical analyzer (lex). The partitioning code
uses a new temporary instance of a lex. But it does still refer
to the previously active lex. The delayd insert thread does not
initialize its lex though...
Added initialization for thd->lex before open table in the delayed
thread and at all other places where it is necessary to call
lex_start() if all tables would be partitioned and need to parse
the .frm file.
causes the Server to crash.
Accessing partitioned table with an apostrophe in partition options
like DATA DIRECTORY, INDEX DIRECTORY or COMMENT causes server crash.
Partition options were saved in .frm file without escaping.
When accessing such table it is not possible to properly restore
partition information.
Crashed because there was no check for partition info parser failure.
Fixed by escaping quoted text in the partition info when writing it to
the frm-file and added a check that it was able to parse the partition
info before using it
NOTE: If the comment is written by an earlier version of the server,
the corrupted frm-file is not fixed, but left corrupted, you have to
manually drop the table and recreate it.
- Reserver namespace and place in frm for TABLE_CHECKSUM and PAGE_CHECKSUM create options
- Added syncing of directory when creating .frm files
- Portability fixes
- Added missing cast that could cause bugs
- Code cleanups
- Made some bit functions inline
- Moved things out of myisam.h to my_handler.h to make them more accessable
- Renamed some myisam variables and defines to make them more globaly usable (as they are used outside of MyISAM)
- Fixed bugs in error conditions
- Use compiler time asserts instead of run time
- Fixed indentation
HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP as the old name was wrong
(Added a define for old value to ensure we don't break any old code)
Added HA_EXTRA_PREPARE_FOR_RENAME as a signal for rename (before we used a DROP signal which is wrong)
- Initialize error messages early to get better errors when mysqld or an engine fails to start
- Fix windows bug that query_performance_frequency was not initialized if registry code failed
- thread_stack -> my_thread_stack_size
Fixed the usage of spatial data (and Point in specific) with
non-spatial indexes.
Several problems :
- The length of the Point class was not updated to include the
spatial reference system identifier. Fixed by increasing with 4
bytes.
- The storage length of the spatial columns was not accounting for
the length that is prepended to it. Fixed by treating the
spatial data columns as blobs (and thus increasing the storage
length)
- When creating the key image for comparison in index read wrong
key image was created (the one needed for and r-tree search,
not the one for b-tree/other search). Fixed by treating the
spatial data columns as blobs (and creating the correct kind of
image based on the index type).
Faster thr_alarm()
Added 'Opened_files' status variable to track calls to my_open()
Don't give warnings when running mysql_install_db
Added option --source-install to mysql_install_db
I had to do the following renames() as used polymorphism didn't work with Forte compiler on 64 bit systems
index_read() -> index_read_map()
index_read_idx() -> index_read_idx_map()
index_read_last() -> index_read_last_map()
When a table uses two (or more) fulltext keys with different fulltext parsers,
all fulltext keys (excluding those that use built-in parser) were using first
parser that was mentioned in .frm.
With this fix fulltext keys use proper (assigned during create table) fulltext
parser.
This code was contributed by Yoshinori Matsunobu ymatsunobu@mysql.com
on 2007-03-12 The copyright was assigned and transferred under the
terms of the MySQL Contributor License Agreement.
Moved duplicated code to inline function store_timestamp()
Save thd->time_zone_used when logging to table as CSV internally cases it to be changed
Added MYSQL_LOCK_IGNORE_FLUSH to log tables to avoid deadlock in case of flush tables.
Mark log tables with TIMESTAMP_NO_AUTO_SET to avoid automatic timestamping
Set TABLE->no_replicate on open
