Coding style: classes start with a capital letter.
Rename some classes related to parsing:
create_field -> Create_field
foreign_key -> Foreign_key
key_part_spec -> Key_part_spec
when logging is enabled.
Currently the partition engine doesn't allow log tables to
be partitioned. But this was not checked and the server crashed.
Fixed by adding a check in ALTER TABLE to disable partitioning the
log tables.
While working on the cause of the problem improved the way the log
thread structures are initialized before opening the log tables.
When constructing the path to the original .frm file ALTER .. RENAME
was unnecessary (and incorrectly) lowercasing the whole path when not
on a case-insensitive filesystem.
This path should not be modified because of lower_case_table_names
as it is already in the correct case according to that setting.
Fixed by removing the lowercasing.
Unfortunately testing this would require running the tests on a case
sensitive filesystem in a directory that has uppercase letters.
This cannot be guaranteed in all setups so no test case added.
SHOW CREATE TABLE fails
Underlying table names, that merge engine fails to open were not
reported.
With this fix CHECK TABLE issued against merge table reports all
underlying table names that it fails to open. Other statements
are unaffected, that is underlying table names are not included
into error message.
This fix doesn't solve SHOW CREATE TABLE issue.
While executing ALTER TABLE ... PARTITION the server uses
a temporary "shadow" table to create the updated table.
This shadow table then gets renamed as the original table.
The shadow table was not prefixed with the special prefix that
marks temporary tables so it was picked up by SHOW TABLE STATUS.
Fixed by isolating the code to create the shadow table name in a
separate function and prefixing the shadow table name with the
special prefix to exclude it from the list of user tables.
See bug 18775 and WL1324 for details.
To avoid unnecessary work the mysql_alter_table function takes the
list of table fields and applies all changes to it (drops/moves/renames/etc).
Then this function compares the new list and the old one. If the changes
require only .frm to be modified then the actual data isn't copied. To detect
changes all columns attributes but names are compared. When a column has been
moved and has replaced another column with the same attributes except name
the mysql_alter_table function wrongly decides that two fields has been just
renamed. As a result the data from the moved column and from all columns
after it is not copied.
Now the mysql_alter_table function forces table data copying by setting
the need_copy_table flag when it finds a moved column. The flag is set at
the stage when the modified fields are created.
Problem: we may create a deadlock committing changes in the mysql_alter_table() when
LOCK_open is set. Moreover, "in some variants of the ALTER TABLE commit
happens earlier, outside of LOCK_open, in other later - inside. It's no good, a storage
engine code that is called in between could expect a consistency - either there is a
transaction or there is not".
Fix: move the commit to happen earlier and outside of the LOCK_open.
The bug was repeated on MyISAM tables, so isn't InnoDB specific.
Reason of the bug is that partition-related members of TABLE_SHARE
wasn't properly updated after ALTER command. So if other thread doesn't
reread frm file, and just uses cached SHARE, it uses wrong data
Bug#4968 ""Stored procedure crash if cursor opened on altered table"
Bug#6895 "Prepared Statements: ALTER TABLE DROP COLUMN does nothing"
Bug#19182 "CREATE TABLE bar (m INT) SELECT n FROM foo; doesn't work from
stored procedure."
Bug#19733 "Repeated alter, or repeated create/drop, fails"
Bug#22060 "ALTER TABLE x AUTO_INCREMENT=y in SP crashes server"
Bug#24879 "Prepared Statements: CREATE TABLE (UTF8 KEY) produces a
growing key length" (this bug is not fixed in 5.0)
Re-execution of CREATE DATABASE, CREATE TABLE and ALTER TABLE
statements in stored routines or as prepared statements caused
incorrect results (and crashes in versions prior to 5.0.25).
In 5.1 the problem occured only for CREATE DATABASE, CREATE TABLE
SELECT and CREATE TABLE with INDEX/DATA DIRECTOY options).
The problem of bugs 4968, 19733, 19282 and 6895 was that functions
mysql_prepare_table, mysql_create_table and mysql_alter_table are not
re-execution friendly: during their operation they modify contents
of LEX (members create_info, alter_info, key_list, create_list),
thus making the LEX unusable for the next execution.
In particular, these functions removed processed columns and keys from
create_list, key_list and drop_list. Search the code in sql_table.cc
for drop_it.remove() and similar patterns to find evidence.
The fix is to supply to these functions a usable copy of each of the
above structures at every re-execution of an SQL statement.
To simplify memory management, LEX::key_list and LEX::create_list
were added to LEX::alter_info, a fresh copy of which is created for
every execution.
The problem of crashing bug 22060 stemmed from the fact that the above
metnioned functions were not only modifying HA_CREATE_INFO structure
in LEX, but also were changing it to point to areas in volatile memory
of the execution memory root.
The patch solves this problem by creating and using an on-stack
copy of HA_CREATE_INFO in mysql_execute_command.
Additionally, this patch splits the part of mysql_alter_table
that analizes and rewrites information from the parser into
a separate function - mysql_prepare_alter_table, in analogy with
mysql_prepare_table, which is renamed to mysql_prepare_create_table.
Bug #23667 "CREATE TABLE LIKE is not isolated from alteration
by other connections"
Bug #18950 "CREATE TABLE LIKE does not obtain LOCK_open"
As well as:
Bug #25578 "CREATE TABLE LIKE does not require any privileges
on source table".
The first and the second bugs resulted in various errors and wrong
binary log order when one tried to execute concurrently CREATE TABLE LIKE
statement and DDL statements on source table or DML/DDL statements on its
target table.
The problem was caused by incomplete protection/table-locking against
concurrent statements implemented in mysql_create_like_table() routine.
We solve it by simply implementing such protection in proper way.
Most of actual work for 5.1 was already done by fix for bug 20662 and
preliminary patch changing locking in ALTER TABLE.
The third bug allowed user who didn't have any privileges on table create
its copy and therefore circumvent privilege check for SHOW CREATE TABLE.
This patch solves this problem by adding privilege check, which was missing.
Finally it also removes some duplicated code from mysql_create_like_table()
and thus fixes bug #26869 "TABLE_LIST::table_name_length inconsistent with
TABLE_LIST::table_name".
Bug #23667 "CREATE TABLE LIKE is not isolated from alteration
by other connections"
Bug #18950 "CREATE TABLE LIKE does not obtain LOCK_open"
As well as:
Bug #25578 "CREATE TABLE LIKE does not require any privileges
on source table".
The first and the second bugs resulted in various errors and wrong
binary log order when one tried to execute concurrently CREATE TABLE LIKE
statement and DDL statements on source table or DML/DDL statements on its
target table.
The problem was caused by incomplete protection/table-locking against
concurrent statements implemented in mysql_create_like_table() routine.
We solve it by simply implementing such protection in proper way (see
comment for sql_table.cc for details).
The third bug allowed user who didn't have any privileges on table create
its copy and therefore circumvent privilege check for SHOW CREATE TABLE.
This patch solves this problem by adding privilege check, which was missing.
Finally it also removes some duplicated code from mysql_create_like_table().
Note that, altough tests covering concurrency-related aspects of CREATE TABLE
LIKE behaviour will only be introduced in 5.1, they were run manually for
this patch as well.
mode.
When a new DATE/DATETIME field without default value is being added by the
ALTER TABLE the '0000-00-00' value is used as the default one. But it wasn't
checked whether such value was allowed by the set sql mode. Due to this
'0000-00-00' values was allowed for DATE/DATETIME fields even in the
NO_ZERO_DATE mode.
Now the mysql_alter_table() function checks whether the '0000-00-00' value
is allowed for DATE/DATETIME fields by the set sql mode.
The new error_if_not_empty flag is used in the mysql_alter_table() function
to indicate that it should abort if the table being altered isn't empty.
The new new_datetime_field field is used in the mysql_alter_table() function
for error throwing purposes.
The new error_if_not_empty parameter is added to the copy_data_between_tables()
function to indicate the it should return error if the source table isn't empty.
and invalidation in the most general case (non-temporary table and
not simple RENAME or ENABLE/DISABLE KEYS or partitioning command).
See comment for sql/sql_table.cc for more information.
These changes are prerequisite for 5.1 version of fix for bug #23667
"CREATE TABLE LIKE is not isolated from alteration by other connections"
TABLES" and failures of alter_table.test on Windows which occured after
pushing fix for bugs #20662, #20903, #24508, #24738 (various problems
with CREATE TABLE SELECT).
ALTER TABLE statements which were handled using "fast" alter table
optimization were not properly working under LOCK TABLES if table
was transactional (for all table types under Windows).
Code implementing "fast" version of ALTER TABLE tried to open and
lock table using open_ltable() after renaming .FRM files (which
corresponds to renaming tables in normal case) in some cases
(for transactional tables or on Windows). This caused problems
under LOCK TABLES and conflicted with name-lock taken by
ALTER TABLE RENAME on target tables.
This patch solves this issue by using reopen_name_locked_table()
instead of open_ltable().
Bug #20662 "Infinite loop in CREATE TABLE IF NOT EXISTS ... SELECT
with locked tables"
Bug #20903 "Crash when using CREATE TABLE .. SELECT and triggers"
Bug #24738 "CREATE TABLE ... SELECT is not isolated properly"
Bug #24508 "Inconsistent results of CREATE TABLE ... SELECT when
temporary table exists"
Deadlock occured when one tried to execute CREATE TABLE IF NOT
EXISTS ... SELECT statement under LOCK TABLES which held
read lock on target table.
Attempt to execute the same statement for already existing
target table with triggers caused server crashes.
Also concurrent execution of CREATE TABLE ... SELECT statement
and other statements involving target table suffered from
various races (some of which might've led to deadlocks).
Finally, attempt to execute CREATE TABLE ... SELECT in case
when a temporary table with same name was already present
led to the insertion of data into this temporary table and
creation of empty non-temporary table.
All above problems stemmed from the old implementation of CREATE
TABLE ... SELECT in which we created, opened and locked target
table without any special protection in a separate step and not
with the rest of tables used by this statement.
This underminded deadlock-avoidance approach used in server
and created window for races. It also excluded target table
from prelocking causing problems with trigger execution.
The patch solves these problems by implementing new approach to
handling of CREATE TABLE ... SELECT for base tables.
We try to open and lock table to be created at the same time as
the rest of tables used by this statement. If such table does not
exist at this moment we create and place in the table cache special
placeholder for it which prevents its creation or any other usage
by other threads.
We still use old approach for creation of temporary tables.
Note that we have separate fix for 5.0 since there we use slightly
different less intrusive approach.
Bug #20662 "Infinite loop in CREATE TABLE IF NOT EXISTS ... SELECT
with locked tables"
Bug #20903 "Crash when using CREATE TABLE .. SELECT and triggers"
Bug #24738 "CREATE TABLE ... SELECT is not isolated properly"
Bug #24508 "Inconsistent results of CREATE TABLE ... SELECT when
temporary table exists"
Deadlock occured when one tried to execute CREATE TABLE IF NOT
EXISTS ... SELECT statement under LOCK TABLES which held
read lock on target table.
Attempt to execute the same statement for already existing
target table with triggers caused server crashes.
Also concurrent execution of CREATE TABLE ... SELECT statement
and other statements involving target table suffered from
various races (some of which might've led to deadlocks).
Finally, attempt to execute CREATE TABLE ... SELECT in case
when a temporary table with same name was already present
led to the insertion of data into this temporary table and
creation of empty non-temporary table.
All above problems stemmed from the old implementation of CREATE
TABLE ... SELECT in which we created, opened and locked target
table without any special protection in a separate step and not
with the rest of tables used by this statement.
This underminded deadlock-avoidance approach used in server
and created window for races. It also excluded target table
from prelocking causing problems with trigger execution.
The patch solves these problems by implementing new approach to
handling of CREATE TABLE ... SELECT for base tables.
We try to open and lock table to be created at the same time as
the rest of tables used by this statement. If such table does not
exist at this moment we create and place in the table cache special
placeholder for it which prevents its creation or any other usage
by other threads.
We still use old approach for creation of temporary tables.
Also note that we decided to postpone introduction of some tests
for concurrent behaviour of CREATE TABLE ... SELECT till 5.1.
The main reason for this is absence in 5.0 ability to set @@debug
variable at runtime, which can be circumvented only by using several
test files with individual .opt files. Since the latter is likely
to slowdown test-suite unnecessary we chose not to push this tests
into 5.0, but run them manually for this version and later push
their optimized version into 5.1
The following type conversions was done:
- Changed byte to uchar
- Changed gptr to uchar*
- Change my_string to char *
- Change my_size_t to size_t
- Change size_s to size_t
Removed declaration of byte, gptr, my_string, my_size_t and size_s.
Following function parameter changes was done:
- All string functions in mysys/strings was changed to use size_t
instead of uint for string lengths.
- All read()/write() functions changed to use size_t (including vio).
- All protocoll functions changed to use size_t instead of uint
- Functions that used a pointer to a string length was changed to use size_t*
- Changed malloc(), free() and related functions from using gptr to use void *
as this requires fewer casts in the code and is more in line with how the
standard functions work.
- Added extra length argument to dirname_part() to return the length of the
created string.
- Changed (at least) following functions to take uchar* as argument:
- db_dump()
- my_net_write()
- net_write_command()
- net_store_data()
- DBUG_DUMP()
- decimal2bin() & bin2decimal()
- Changed my_compress() and my_uncompress() to use size_t. Changed one
argument to my_uncompress() from a pointer to a value as we only return
one value (makes function easier to use).
- Changed type of 'pack_data' argument to packfrm() to avoid casts.
- Changed in readfrm() and writefrom(), ha_discover and handler::discover()
the type for argument 'frmdata' to uchar** to avoid casts.
- Changed most Field functions to use uchar* instead of char* (reduced a lot of
casts).
- Changed field->val_xxx(xxx, new_ptr) to take const pointers.
Other changes:
- Removed a lot of not needed casts
- Added a few new cast required by other changes
- Added some cast to my_multi_malloc() arguments for safety (as string lengths
needs to be uint, not size_t).
- Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done
explicitely as this conflict was often hided by casting the function to
hash_get_key).
- Changed some buffers to memory regions to uchar* to avoid casts.
- Changed some string lengths from uint to size_t.
- Changed field->ptr to be uchar* instead of char*. This allowed us to
get rid of a lot of casts.
- Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar
- Include zlib.h in some files as we needed declaration of crc32()
- Changed MY_FILE_ERROR to be (size_t) -1.
- Changed many variables to hold the result of my_read() / my_write() to be
size_t. This was needed to properly detect errors (which are
returned as (size_t) -1).
- Removed some very old VMS code
- Changed packfrm()/unpackfrm() to not be depending on uint size
(portability fix)
- Removed windows specific code to restore cursor position as this
causes slowdown on windows and we should not mix read() and pread()
calls anyway as this is not thread safe. Updated function comment to
reflect this. Changed function that depended on original behavior of
my_pwrite() to itself restore the cursor position (one such case).
- Added some missing checking of return value of malloc().
- Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow.
- Changed type of table_def::m_size from my_size_t to ulong to reflect that
m_size is the number of elements in the array, not a string/memory
length.
- Moved THD::max_row_length() to table.cc (as it's not depending on THD).
Inlined max_row_length_blob() into this function.
- More function comments
- Fixed some compiler warnings when compiled without partitions.
- Removed setting of LEX_STRING() arguments in declaration (portability fix).
- Some trivial indentation/variable name changes.
- Some trivial code simplifications:
- Replaced some calls to alloc_root + memcpy to use
strmake_root()/strdup_root().
- Changed some calls from memdup() to strmake() (Safety fix)
- Simpler loops in client-simple.c
- The function build_table_filename() builds up a string unconditionally
using the forward slash as a path separator. Later, when the string is
searched for FN_LIBCHAR by the set_up_table_before_create() function, a
null pointer is returned that is finally used by strlen in the
append_file_to_dir() function which causes the crash.
NO_AUTO_VALUE_ON_ZERO mode.
In the NO_AUTO_VALUE_ON_ZERO mode the table->auto_increment_field_not_null
variable is used to indicate that a non-NULL value was specified by the user
for an auto_increment column. When an INSERT .. ON DUPLICATE updates the
auto_increment field this variable is set to true and stays unchanged for the
next insert operation. This makes the next inserted row sometimes wrongly have
0 as the value of the auto_increment field.
Now the fill_record() function resets the table->auto_increment_field_not_null
variable before filling the record.
The table->auto_increment_field_not_null variable is also reset by the
open_table() function for a case if we missed some auto_increment_field_not_null
handling bug.
Now the table->auto_increment_field_not_null is reset at the end of the
mysql_load() function.
Reset the table->auto_increment_field_not_null variable after each
write_row() call in the copy_data_between_tables() function.
ARCHIVE table
ARCHIVE table was truncated by REPAIR TABLE ... USE_FRM statement.
The table handler returned its file name extensions in a wrong order.
REPAIR TABLE believed it has to use the meta file to create a new table
from it.
With the fixed order, REPAIR TABLE does now use the data file to create
a new table. So REPAIR TABLE ... USE_FRM works well with ARCHIVE engine
now.
This issue affects 5.0 only, since in 5.1 ARCHIVE engine stores meta
information and data in the same file.
thd->options' OPTION_STATUS_NO_TRANS_UPDATE bit was not restored at the end of SF() invocation, where
SF() modified non-ta table.
As the result of this artifact it was not possible to detect whether there were any side-effects when
top-level query ends.
If the top level query table was not modified and the bit is lost there would be no binlogging.
Fixed with preserving the bit inside of thd->no_trans_update struct. The struct agregates two bool flags
telling whether the current query and the current transaction modified any non-ta table.
The flags stmt, all are dropped at the end of the query and the transaction.
It was syntactically correct to define
spatial keys over parts of columns (e.g.
ALTER TABLE t1 ADD x GEOMETRY NOT NULL,
ADD SPATIAL KEY (x(32))).
This may lead to undefined results and/or
interpretation.
Fixed by not allowing partial column
specification in a SPATIAL index definition.
Different set of conditions is used to verify
the validity of index definitions over a GEOMETRY
column in ALTER TABLE and CREATE TABLE.
The difference was on how sub-keys notion validity
is checked.
Fixed by extending the CREATE TABLE condition to
support the cases allowed in ALTER TABLE.
Made the SHOW CREATE TABLE not to display spatial
indexes using the sub-key notion.
Different set of conditions is used to verify
the validity of index definitions over a GEOMETRY
column in ALTER TABLE and CREATE TABLE.
The difference was on how sub-keys notion validity
is checked.
Fixed by extending the CREATE TABLE condition to
support the cases allowed in ALTER TABLE.
Made the SHOW CREATE TABLE not to display spatial
indexes using the sub-key notion.
"Server Variables for Plugins"
Implement support for plugins to declare server variables.
Demonstrate functionality by removing InnoDB specific code from sql/*
New feature for HASH - HASH_UNIQUE flag
New feature for DYNAMIC_ARRAY - initializer accepts preallocated ptr.
Completed support for plugin reference counting.
Fixed compile-pentium64 scripts
Fixed wrong estimate of update_with_key_prefix in sql-bench
Merge bk-internal.mysql.com:/home/bk/mysql-5.1 into mysql.com:/home/my/mysql-5.1
Fixed unsafe define of uint4korr()
Fixed that --extern works with mysql-test-run.pl
Small trivial cleanups
This also fixes a bug in counting number of rows that are updated when we have many simultanous queries
Move all connection handling and command exectuion main loop from sql_parse.cc to sql_connection.cc
Split handle_one_connection() into reusable sub functions.
Split create_new_thread() into reusable sub functions.
Added thread_scheduler; Preliminary interface code for future thread_handling code.
Use 'my_thread_id' for internal thread id's
Make thr_alarm_kill() to depend on thread_id instead of thread
Make thr_abort_locks_for_thread() depend on thread_id instead of thread
In store_globals(), set my_thread_var->id to be thd->thread_id.
Use my_thread_var->id as basis for my_thread_name()
The above changes makes the connection we have between THD and threads more soft.
Added a lot of DBUG_PRINT() and DBUG_ASSERT() functions
Fixed compiler warnings
Fixed core dumps when running with --debug
Removed setting of signal masks (was never used)
Made event code call pthread_exit() (portability fix)
Fixed that event code doesn't call DBUG_xxx functions before my_thread_init() is called.
Made handling of thread_id and thd->variables.pseudo_thread_id uniform.
Removed one common 'not freed memory' warning from mysqltest
Fixed a couple of usage of not initialized warnings (unlikely cases)
Suppress compiler warnings from bdb and (for the moment) warnings from ndb
to a single statement.
---
Bug#24795: SHOW PROFILE
Profiling is only partially functional on some architectures. Where
there is no getrusage() system call, presently Null values are
returned where it would be required. Notably, Windows needs some love
applied to make it as useful.
Syntax this adds:
SHOW PROFILES
SHOW PROFILE [types] [FOR QUERY n] [OFFSET n] [LIMIT n]
where "n" is an integer
and "types" is zero or many (comma-separated) of
"CPU"
"MEMORY" (not presently supported)
"BLOCK IO"
"CONTEXT SWITCHES"
"PAGE FAULTS"
"IPC"
"SWAPS"
"SOURCE"
"ALL"
It also adds a session variable (boolean) "profiling", set to "no"
by default, and (integer) profiling_history_size, set to 15 by
default.
This patch abstracts setting THDs' "proc_info" behind a macro that
can be used as a hook into the profiling code when profiling
support is compiled in. All future code in this line should use
that mechanism for setting thd->proc_info.
---
Tests are now set to omit the statistics.
---
Adds an Information_schema table, "profiling" for access to
"show profile" data.
---
Merge zippy.cornsilk.net:/home/cmiller/work/mysql/mysql-5.0-community-3--bug24795
into zippy.cornsilk.net:/home/cmiller/work/mysql/mysql-5.0-community
---
Fix merge problems.
---
Fixed one bug in the query_source being NULL.
Updated test results.
---
Include more thorough profiling tests.
Improve support for prepared statements.
Use session-specific query IDs, starting at zero.
---
Selecting from I_S.profiling is no longer quashed in profiling, as
requested by Giuseppe.
Limit the size of captured query text.
No longer log queries that are zero length.
can crash under load
Resizing a key cache while it was in heavy use could crash the
server. There were several race conditions.
I reworked some of the algorithms to fix the race conditions.
No test case. Repeating the crashes requires heavy concurrent
load on the key cache. A test script is attached to the bug report.
More explanations to the changes are contained in a text file
attached to the bug report.
Removed a lot of compiler warnings
Removed not used variables, functions and labels
Initialize some variables that could be used unitialized (fatal bugs)
%ll -> %l
tables' lock."
Execution of ALTER TABLE ... ENABLE KEYS on a table (which can take rather
long time) prevented concurrent execution of all statements using tables.
The problem was caused by the fact that we were holding LOCK_open mutex
during whole duration of this statement and particularly during call
to handler::enable_indexes(). This behavior was introduced as part of the
fix for bug 14262 "SP: DROP PROCEDURE|VIEW (maybe more) write to binlog
too late (race cond)"
The patch simply restores old behavior. Note that we can safely do this as
this operation takes exclusive lock (similar to name-lock) which blocks both
DML and DDL on the table being altered.
It also introduces mysql-test/include/wait_show_pattern.inc helper script
which is used to make test-case for this bug robust enough.
After fix for bug#21798 JOIN stores the pointer to the buffer for sorting
fields. It is used while sorting for grouping and for ordering. If ORDER BY
clause has more elements then the GROUP BY clause then a memory overrun occurs.
Now the length of the ORDER BY list is always passed to the
make_unireg_sortorder() function and it allocates buffer big enough to be
used for bigger list.
Fixed compiler warnings.
In set_var.cc, the code was not properly returning an error code
if close_cached_tables() failed.
In sql_tables.cc, the code was not returning properly an error code
if lock_table_names() failed.
Both cases are bugs, introduced in 5.1 only by recent changes.
- Removed not used variables and functions
- Added #ifdef around code that is not used
- Renamed variables and functions to avoid conflicts
- Removed some not used arguments
Fixed some class/struct warnings in ndb
Added define IS_LONGDATA() to simplify code in libmysql.c
I did run gcov on the changes and added 'purecov' comments on almost all lines that was not just variable name changes
