Apart strict-aliasing warnings, fix the remaining warnings
generated by GCC 4.4.4 -Wall and -Wextra flags.
One major source of warnings was the in-house function my_bcmp
which (unconventionally) took pointers to unsigned characters
as the byte sequences to be compared. Since my_bcmp and bcmp
are deprecated functions whose only difference with memcmp is
the return value, every use of the function is replaced with
memcmp as the special return value wasn't actually being used
by any caller.
There were also various other warnings, mostly due to type
mismatches, missing return values, missing prototypes, dead
code (unreachable) and ignored return values.
strict aliasing violations.
One somewhat major source of strict-aliasing violations and
related warnings is the SQL_LIST structure. For example,
consider its member function `link_in_list` which takes
a pointer to pointer of type T (any type) as a pointer to
pointer to unsigned char. Dereferencing this pointer, which
is done to reset the next field, violates strict-aliasing
rules and might cause problems for surrounding code that
uses the next field of the object being added to the list.
The solution is to use templates to parametrize the SQL_LIST
structure in order to deference the pointers with compatible
types. As a side bonus, it becomes possible to remove quite
a few casts related to acessing data members of SQL_LIST.
SunStudio
SunStudio compilers of late warn about methods that might hide
methods in base classes due to the use of overloading combined
with overriding. SunStudio also warns about variables defined
in local socpe or method arguments that have the same name as
a member attribute of the class.
This patch renames methods that might hide base class methods,
to make it easier both for humans and compilers to see what is
actually called. It also renames variables in local scope.
work in 5.1.40)
MERGE engine fails to open child table from a different
database if child table/database name contains characters
that are subject for table name to filename encoding
(WL1324).
Another problem is that MERGE engine didn't properly open
child table from the same database if child table name
contains characters like '/', '#'.
The problem was that table name to file name encoding was
applied inconsistently:
* On CREATE: encode table name + database name if child
table is in different database; do not encode table
name if child table is in the same database;
* No decoding on open.
With this fix child table/database names are always
encoded on CREATE and decoded on open. Compatibility
with older tables preserved.
Along with this patch comes fix for SHOW CREATE TABLE,
which used to show child table/database path instead
of child table/database names.
A REPLACE in the MERGE engine is actually a REPLACE
into one (FIRST or LAST) of the underlying MyISAM
tables. So in effect the server works on the meta
data of the MERGE table, while the real insert happens
in the MyISAM table.
The MERGE table has no index, while MyISAM has a
unique index. When a REPLACE into a MERGE table (
and the REPLACE conflicts with a duplicate in a
child table) is done, we try to access the duplicate
key information for the MERGE table. This information
actually does not exist, hence this results in a crash.
The problem can be resolved by modifying the MERGE
engine to provide us the duplicate key information
directly, instead of just returning the MyISAM index
number as the error key. Then the SQL layer (or "the
server") does not try to access the key_info of the
MERGE table, which does not exist.
The current patch modifies the MERGE engine to provide
the position for a record where a unique key violation
occurs.
and base tables
myrg_attach_children() could reuse a buffer that was allocated
previously based on a definition of a child table. The problem
was that the child's definition might have been changed, so
reusing the buffer could lead to crashes or valgrind errors
under some circumstances.
Fixed by changing myrg_attach_children() so that the
rec_per_key_part buffer is reused only when the child table
have not changed, and reallocated otherwise (the old buffer is
deallocated if necessary).
The conformance checker was not taking into
account, and, making concessions for acceptable
incompatibilites in tables created by
versions earlier than 4.1.
The current patch relaxes the conformance
checker to ignore differences in key_alg
and language for tables created by versions
earlier than 4.1.
This problem comes while inserting a duplicate row in merge table
without key but the child table has a primary key.
While forming the error message handler tries to locate the key field
which is creating this problem but as there is no key on the merge
table there is a segmentation fault.
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
A Query in the MyISAM merge table was crashing
if the index merge algorithm was being used
Index Merge optimization requires the reading of
multiple indexes at the same time. Reading multiple
indexes at once with current SE API means that we
need to have handler instance for each to-be-read
index. This is done by creating clones of the handlers
instances. The clone internally does a open of the handler.
The open for a MERGE engine is handled in the following
phases
1) open parent table
2) generate list of underlying
table
3) attach underlying tables
But the current implementation does only the first
phase (i.e.) open parent table.
The current patch fixes this at the MERGE engine level,
by handling the clone operation within the MERGE engine
rather than in the storage engine API. It opens and
attaches the MyISAM tables on the MyISAM storage engine
interface directly within the MERGE engine. The new MyISAM
table instances, as well as the MERGE clone itself, are not
visible in the table cache. This is not a problem because
all locking is handled by the original MERGE table from which
this is cloned of.
Every subsequent query to a merge table with indexes was lowering
down cardinality.
The problem was that key statistics was not cleared when merge
children were detached. Causing next attach children perform
incremental key statistics calculation.
Fixed by clearing key statistics when attaching first child.
for ENGINE=MRG_MYISAM (should be optimized out).
Before WL#3281 MERGE engine had the HA_NOT_EXACT_COUNT flag
unset, and it worked with COUNT optimization as desired.
After the removal of the HA_NOT_EXACT_COUNT flag neither
HA_STATS_RECORDS_IS_EXACT (opposite to former HA_NOT_EXACT_COUNT
flag) nor modern HA_HAS_RECORDS flag were not added to MERGE
table flag mask.
1. The HA_HAS_RECORDS table flag has been set.
2. The ha_myisammrg::records method has been overridden to
calculate total number of records in underlying tables.
The patch for Bug 26379 (Combination of FLUSH TABLE and
REPAIR TABLE corrupts a MERGE table) fixed this bug too.
However it revealed a new bug that crashed the server.
Flushing a merge table at the moment when it is between open
and attach of children crashed the server.
The flushing thread wants to abort locks on the flushed table.
It calls ha_myisammrg::lock_count() and ha_myisammrg::store_lock()
on the TABLE object of the other thread.
Changed ha_myisammrg::lock_count() and ha_myisammrg::store_lock()
to accept non-attached children. ha_myisammrg::lock_count() returns
the number of MyISAM tables in the MERGE table so that the memory
allocation done by get_lock_data() is done correctly, even if the
children become attached before ha_myisammrg::store_lock() is
called. ha_myisammrg::store_lock() will not return any lock if the
children are not attached.
This is however a change in the handler interface. lock_count()
can now return a higher number than store_lock() stores locks.
This is more safe than the reverse implementation would be.
get_lock_data() in the SQL layer is adjusted accordingly. It sets
MYSQL_LOCK::lock_count based on the number of locks returned by
the handler::store_lock() calls, not based on the numbers returned
by the handler::lock_count() calls. The latter are only used for
allocation of memory now.
No test case. The test suite cannot reliably run FLUSH between
lock_count() and store_lock() of another thread. The bug report
contains a program that can repeat the problem with some
probability.
corrupts a MERGE table
Bug 26867 - LOCK TABLES + REPAIR + merge table result in
memory/cpu hogging
Bug 26377 - Deadlock with MERGE and FLUSH TABLE
Bug 25038 - Waiting TRUNCATE
Bug 25700 - merge base tables get corrupted by
optimize/analyze/repair table
Bug 30275 - Merge tables: flush tables or unlock tables
causes server to crash
Bug 19627 - temporary merge table locking
Bug 27660 - Falcon: merge table possible
Bug 30273 - merge tables: Can't lock file (errno: 155)
The problems were:
Bug 26379 - Combination of FLUSH TABLE and REPAIR TABLE
corrupts a MERGE table
1. A thread trying to lock a MERGE table performs busy waiting while
REPAIR TABLE or a similar table administration task is ongoing on
one or more of its MyISAM tables.
2. A thread trying to lock a MERGE table performs busy waiting until all
threads that did REPAIR TABLE or similar table administration tasks
on one or more of its MyISAM tables in LOCK TABLES segments do UNLOCK
TABLES. The difference against problem #1 is that the busy waiting
takes place *after* the administration task. It is terminated by
UNLOCK TABLES only.
3. Two FLUSH TABLES within a LOCK TABLES segment can invalidate the
lock. This does *not* require a MERGE table. The first FLUSH TABLES
can be replaced by any statement that requires other threads to
reopen the table. In 5.0 and 5.1 a single FLUSH TABLES can provoke
the problem.
Bug 26867 - LOCK TABLES + REPAIR + merge table result in
memory/cpu hogging
Trying DML on a MERGE table, which has a child locked and
repaired by another thread, made an infinite loop in the server.
Bug 26377 - Deadlock with MERGE and FLUSH TABLE
Locking a MERGE table and its children in parent-child order
and flushing the child deadlocked the server.
Bug 25038 - Waiting TRUNCATE
Truncating a MERGE child, while the MERGE table was in use,
let the truncate fail instead of waiting for the table to
become free.
Bug 25700 - merge base tables get corrupted by
optimize/analyze/repair table
Repairing a child of an open MERGE table corrupted the child.
It was necessary to FLUSH the child first.
Bug 30275 - Merge tables: flush tables or unlock tables
causes server to crash
Flushing and optimizing locked MERGE children crashed the server.
Bug 19627 - temporary merge table locking
Use of a temporary MERGE table with non-temporary children
could corrupt the children.
Temporary tables are never locked. So we do now prohibit
non-temporary chidlren of a temporary MERGE table.
Bug 27660 - Falcon: merge table possible
It was possible to create a MERGE table with non-MyISAM children.
Bug 30273 - merge tables: Can't lock file (errno: 155)
This was a Windows-only bug. Table administration statements
sometimes failed with "Can't lock file (errno: 155)".
These bugs are fixed by a new implementation of MERGE table open.
When opening a MERGE table in open_tables() we do now add the
child tables to the list of tables to be opened by open_tables()
(the "query_list"). The children are not opened in the handler at
this stage.
After opening the parent, open_tables() opens each child from the
now extended query_list. When the last child is opened, we remove
the children from the query_list again and attach the children to
the parent. This behaves similar to the old open. However it does
not open the MyISAM tables directly, but grabs them from the already
open children.
When closing a MERGE table in close_thread_table() we detach the
children only. Closing of the children is done implicitly because
they are in thd->open_tables.
For more detail see the comment at the top of ha_myisammrg.cc.
Changed from open_ltable() to open_and_lock_tables() in all places
that can be relevant for MERGE tables. The latter can handle tables
added to the list on the fly. When open_ltable() was used in a loop
over a list of tables, the list must be temporarily terminated
after every table for open_and_lock_tables().
table_list->required_type is set to FRMTYPE_TABLE to avoid open of
special tables. Handling of derived tables is suppressed.
These details are handled by the new function
open_n_lock_single_table(), which has nearly the same signature as
open_ltable() and can replace it in most cases.
In reopen_tables() some of the tables open by a thread can be
closed and reopened. When a MERGE child is affected, the parent
must be closed and reopened too. Closing of the parent is forced
before the first child is closed. Reopen happens in the order of
thd->open_tables. MERGE parents do not attach their children
automatically at open. This is done after all tables are reopened.
So all children are open when attaching them.
Special lock handling like mysql_lock_abort() or mysql_lock_remove()
needs to be suppressed for MERGE children or forwarded to the parent.
This depends on the situation. In loops over all open tables one
suppresses child lock handling. When a single table is touched,
forwarding is done.
Behavioral changes:
===================
This patch changes the behavior of temporary MERGE tables.
Temporary MERGE must have temporary children.
The old behavior was wrong. A temporary table is not locked. Hence
even non-temporary children were not locked. See
Bug 19627 - temporary merge table locking.
You cannot change the union list of a non-temporary MERGE table
when LOCK TABLES is in effect. The following does *not* work:
CREATE TABLE m1 ... ENGINE=MRG_MYISAM ...;
LOCK TABLES t1 WRITE, t2 WRITE, m1 WRITE;
ALTER TABLE m1 ... UNION=(t1,t2) ...;
However, you can do this with a temporary MERGE table.
You cannot create a MERGE table with CREATE ... SELECT, neither
as a temporary MERGE table, nor as a non-temporary MERGE table.
CREATE TABLE m1 ... ENGINE=MRG_MYISAM ... SELECT ...;
Gives error message: table is not BASE TABLE.
- Reserver namespace and place in frm for TABLE_CHECKSUM and PAGE_CHECKSUM create options
- Added syncing of directory when creating .frm files
- Portability fixes
- Added missing cast that could cause bugs
- Code cleanups
- Made some bit functions inline
- Moved things out of myisam.h to my_handler.h to make them more accessable
- Renamed some myisam variables and defines to make them more globaly usable (as they are used outside of MyISAM)
- Fixed bugs in error conditions
- Use compiler time asserts instead of run time
- Fixed indentation
HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP as the old name was wrong
(Added a define for old value to ensure we don't break any old code)
Added HA_EXTRA_PREPARE_FOR_RENAME as a signal for rename (before we used a DROP signal which is wrong)
- Initialize error messages early to get better errors when mysqld or an engine fails to start
- Fix windows bug that query_performance_frequency was not initialized if registry code failed
- thread_stack -> my_thread_stack_size
Faster thr_alarm()
Added 'Opened_files' status variable to track calls to my_open()
Don't give warnings when running mysql_install_db
Added option --source-install to mysql_install_db
I had to do the following renames() as used polymorphism didn't work with Forte compiler on 64 bit systems
index_read() -> index_read_map()
index_read_idx() -> index_read_idx_map()
index_read_last() -> index_read_last_map()
Additional changes for bug#29903
- Changed to do embedded build part as normal build, when
WITH_EMBEDDED_SERVER is set.
- Allow both normal and debug build with embedded.
- Build static embedded library by pointing out all source and compile
it all, i.e. not building libraries from libraries, not portable.
- Let embedded use generated files from the "sql" directory, added
dependencies to make sure built before embedded.
- Mark library "dbug" in TARGET_LINK_LIBRARIES() with "debug", so only
linked in when debug target is used.
- Removed change of target name with "mysqld${MYSQLD_EXE_SUFFIX}", as
others can't depend on it, not defined at configure time. Instead
set the output file name.
- Created work around for bug in CMake 2.4.6 and output names, to
set the "mysqld<suffix>.pdb" name to the same base name.
- Set the correct manifest "name" (patch by iggy)
They had been introduced in 5.1 and were only later backported to 5.0;
as a consequence, the files in the 5.1 tree do not depend on the 5.0 ones,
and changes in 5.0 do not propagate into the 5.1 files.
To fix this, the (previous) files in 5.1 now are deleted ("bk rm"),
and the previously deleted files depending on 5.0 are now moved to the
respective source directories ("bk mv").
The current 5.1 contents is restored in these files.
If you need the previous history of the 5.1 files ("bk revtool"),
access those in "BitKeeper/deleted".
Contrary to the original plan, I did not introduce the name
"CMakeLists.historic" - mostly in order not to clutter the source tree.
This fixes bug#29982.
Stopping mysql server could result in an entry in mysql error
file: "InnoDB: Error: MySQL is freeing a thd".
This happened because InnoDB assumes that the server will never
call external_lock(F_UNLCK) in case external_lock(READ/WRITE)
failed.
Prior to this patch we haven't had strict definition whether
external_lock(F_UNLCK) must be called in case external_lock(READ/WRITE)
fails.
This patch states that we never call external_lock(F_UNLCK) in case
external_lock(READ/WRITE) fails.
- Add build configuration parameter EMBEDDED_ONLY which will configure
the VS solution to produce only mysql embedded binary.
- Make necessary updates to successfully compile solution.
