New prototype for get_auto_increment() (but new arguments not yet used), to be able
to reserve a finite interval of auto_increment values from cooperating engines.
A hint on how many values to reserve is found in handler::estimation_rows_to_insert,
filled by ha_start_bulk_insert(), new wrapper around start_bulk_insert().
NOTE: this patch changes nothing, for all engines. But it makes the API ready for those
engines which will want to do reservation.
More csets will come to complete WL#3146.
duplicate fields removed, st_mysql_storage_engine added to support
run-time handlerton initialization (no compiler warnings), handler API
is now tied to MySQL version, handlerton->plugin mapping added
(slot-based), dummy default_hton removed, plugin-type-specific
initialization generalized, built-in plugins are now initialized too,
--default-storage-engine no longer needs a list of storage engines
in handle_options().
mysql-test-run.pl bugfixes
Added support for key_block_size to MyISAM.
Simplify interface to 'new Key' to make it easier to add new key options.
mysqld option --new is used to define where key options are printed.
(In 5.3 we should move all key options to after key part definition to avoid problem with reserved names)
Fixed some compiler warnings and a memory leak in ssl
Added missing DBUG_xxx_RETURN statements
Fixed some usage of not initialized variables (as found by valgrind)
Ensure that we don't remove locked tables used as name locks from open table cache until unlock_table_names() are called.
This was fixed by having drop_locked_name() returning any table used as a name lock so that we can free it in unlock_table_names()
This will allow Tomas to continue with his work to use namelocks to syncronize things.
Note: valgrind still produces a lot of warnings about using not initialized code and shows memory loss errors when running the ndb tests
and new binlog format called "mixed" (which is statement-based except if only row-based is correct,
in this cset it means if UDF or UUID is used; more cases could be added in later 5.1 release):
SET GLOBAL|SESSION BINLOG_FORMAT=row|statement|mixed|default;
the global default is statement unless cluster is enabled (then it's row) as in 5.1-alpha.
It's not possible to use SET on this variable if a session is currently in row-based mode and has open temporary tables (because CREATE
TEMPORARY TABLE was not binlogged so temp table is not known on slave), or if NDB is enabled (because
NDB does not support such change on-the-fly, though it will later), of if in a stored function (see below).
The added tests test the possibility or impossibility to SET, their effects, and the mixed mode,
including in prepared statements and in stored procedures and functions.
Caveats:
a) The mixed mode will not work for stored functions: in mixed mode, a stored function will
always be binlogged as one call and in a statement-based way (e.g. INSERT VALUES(myfunc()) or SELECT myfunc()).
b) for the same reason, changing the thread's binlog format inside a stored function is
refused with an error message.
c) the same problems apply to triggers; implementing b) for triggers will be done later (will ask
Dmitri).
Additionally, as the binlog format is now changeable by each user for his session, I remove the implication
which was done at startup, where row-based automatically set log-bin-trust-routine-creators to 1
(not possible anymore as a user can now switch to stmt-based and do nasty things again), and automatically
set --innodb-locks-unsafe-for-binlog to 1 (was anyway theoretically incorrect as it disabled
phantom protection).
Plus fixes for compiler warnings.
Optimised version of ADD/DROP/REORGANIZE partitions for
non-NDB storage engines.
New syntax to handle REBUILD/OPTIMIZE/ANALYZE/CHECK/REPAIR partitions
Quite a few bug fixes
NDB cluster is not fully supported. This will be added with
WL 1892 (NDB Handler: Add support for CREATE/DROP INDEX).
Some preparatory code for this is already present though.
A change for the "duplicate key" error message is planned
for another changeset.
(MySQL plugin interface: status variables)
adding SHOW_FUNC, removing SHOW_some_specific_value,
only generic SHOW_LONG/SHOW_CHAR/etc are recognized.
changing to use SHOW_FUNC instead of ha_update_statistics
- Encoding itself, implemented as a charset
"filename". Originally planned to use '.'
as an escape character, but now changed to '@'
for two reasons: "ls" does not return
file names starting with '.' considering them
as a kind of hidden files; some platforms
do not allow several dots in a file name.
- replacing many calls of my_snprintf() and
strnxmov() to the new build_table_filename().
- Adding MY_APPEND_EXT mysys flag, to append
an extention rather that replace it.
- Replacing all numeric constants in fn_format
flag arguments to their mysys definitions, e.g.
MY_UNPACK_FILENAME,
- Predictability in several function/methods:
when a table name can appear with or withot .frm
extension. Some functions/methods were changed
so accept names strictly with .frm, other - strictly
without .frm extensions. Several DBUG_ASSERTs were
added to check whether an extension is passed.
Many files:
table name to file name encoding
mysql_priv.h:
Prototypes for new table name encoding tools.
ctype-utf8.c:
Implementing "filename" charset for
table name to file name encoding.
row0mysql.c:
Fixing table name prefix.
mf_format.c:
Adding MY_APPEND_EXT processing.
Many files:
Fixing tests.
my_sys.h:
Adding new flag to append rather than replace an extension.
m_ctype.h:
Adding "filename" charset definition.
in NDB that the fix would cause due to improper registration of a transaction
that isn't meant to be seen by the handler layer. (Bug #14212)
Thanks to Sergei Golubchik for helping with this.
The table opening process now works the following way:
- Create common TABLE_SHARE object
- Read the .frm file and unpack it into the TABLE_SHARE object
- Create a TABLE object based on the information in the TABLE_SHARE
object and open a handler to the table object
Other noteworthy changes:
- In TABLE_SHARE the most common strings are now LEX_STRING's
- Better error message when table is not found
- Variable table_cache is now renamed 'table_open_cache'
- New variable 'table_definition_cache' that is the number of table defintions that will be cached
- strxnmov() calls are now fixed to avoid overflows
- strxnmov() will now always add one end \0 to result
- engine objects are now created with a TABLE_SHARE object instead of a TABLE object.
- After creating a field object one must call field->init(table) before using it
- For a busy system this change will give you:
- Less memory usage for table object
- Faster opening of tables (if it's has been in use or is in table definition cache)
- Allow you to cache many table definitions objects
- Faster drop of table
Bad examples of usage of a string with its length fixed.
The incorrect length in the trigger file configuration descriptor
fixed (BUG#14090).
A hook for unknown keys added to the parser to support old .TRG files.
handling of savepoints in stored routines.
Fixed ha_rollback_to_savepoint()/ha_savepoint()/ha_release_savepoint()
functions to properly handle savepoints inside of stored functions and
triggers.
Also now when we invoke stored function or trigger we create new savepoint
level. We destroy it at the end of function/trigger execution and return back
to old savepoint level.
large table gives server crash": make sure that when a MyISAM temporary
table is created for a cursor, it's created in its memory root,
not the memory root of the current query.
- CHAR() now returns binary string as default
- CHAR(X*65536+Y*256+Z) is now equal to CHAR(X,Y,Z) independent of the character set for CHAR()
- Test for both ETIMEDOUT and ETIME from pthread_cond_timedwait()
(Some old systems returns ETIME and it's safer to test for both values
than to try to write a wrapper for each old system)
- Fixed new introduced bug in NOT BETWEEN X and X
- Ensure we call commit_by_xid or rollback_by_xid for all engines, even if one engine has failed
- Use octet2hex() for all conversion of string to hex
- Simplify and optimize code
Handlerton array is now created instead of using sys_table_types_st. All storage engines can now have inits and giant ifdef's are now gone for startup. No compeltely clean yet, handlertons will next be merged with sys_table_types. Federated and archive now have real cleanup if their inits fail.
The alias structure now is a bit more simple and just uses a pointer to replace with the currect name. The giant case statement should go away in the next patch.
cursor is interpreted latin1 character and Bug#9819 "Cursors: Mysql Server
Crash while fetching from table with 5 million records."
A fix for a possible memory leak when fetching into an SP cursor
in a long loop.
The patch uses a common implementation of cursors in the binary protocol and
in stored procedures and implements materialized cursors.
For implementation details, see comments in sql_cursor.cc
Added flag to Field::store(longlong) to specify if value is unsigned.
This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0)
Fixed warning from valgrind in CREATE ... SELECT
Fixed double free of mysql.options if reconnect failed
its body, but lets each statement to get/release its own locks. This allows a broader set
of statements to be executed inside PROCEDUREs (but breaks replication)
This patch should fix BUG#8072, BUG#8766, BUG#9563, BUG#11126
The idea of the patch
is that every cursor gets its own lock id for table level locking.
Thus cursors are protected from updates performed within the same
connection. Additionally a list of transient (must be closed at
commit) cursors is maintained and all transient cursors are closed
when necessary. Lastly, this patch adds support for deadlock
timeouts to TLL locking when using cursors.
+ post-review fixes.
"Triggers have the wrong namespace"
"Triggers: duplicate names allowed"
"Triggers: CREATE TRIGGER does not accept fully qualified names"
"SHOW TRIGGERS"
- Added better error messages when trying to open a table that can't be discovered or unpacked. The most likely cause of this is that it does not have any frm data, probably since it has been created from NdbApi or is a NDB system table.
- Separated functionality that was in ha_create_table_from_engine into two functions. One that checks if the table exists and another one that tries to create the table from the engine.
We should not allow explicit or implicit transaction commits inside
of stored functions or triggers (so in autocommit mode we should not
do commits after execution of sub-statement).
Also since we don't support nested statement transactions in 5.0,
we shouldn't commit or rollback stmt transactions while we are inside
stored functions or triggers. This should be fixed in later (>=5.1)
releases.
Ensure that 'null_value' is not accessed before val() is called in FIELD() functions
Fixed initialization of key maps. This fixes some problems with keys when you have more than 64 keys
Fixed that ROLLUP don't always create a temporary table. This fix ensures that func_gconcat.test results are now predictable
to read and write
Changed Server code, added new interface to handler and changed the
NDB handler, InnoDB handler and Federated handler that previously used
query_id
Bug#10202 fix (one-liner fix for memory leak)
BUG#9738 "SHOW VARIABLES still displays the deprecated 'log_update' in 5.0",
BUG#9542 "MySQL dies with signal 11 when it is using non-existent location of binary logs"
Semi-synchronous replication for InnoDB type tables; before telling the client that a commit has been processed, wait that the replication thread has returned from my_net_send() where it sends the binlog to the slave; note that TCP/IP, even with the TCP_NODELAY option does not guarantee that the slave has RECEIVED the data - this is just heuristic at the moment; this is useful in failover: in almost all cases, every transaction that has returned from the commit has been sent and processed in the slave, which makes failover to the slave simpler if the master crashes; the code does not work yet as is, because MySQL should call innobase_report_binlog_offset_and_commit() in a commit; we will most probably return that call to 5.0.x, to make InnoDB Hot Backup and group commit to work again; XA code broke them temporarily in 5.0.3
#6559 "DROP DATABASE forgets to drop triggers".
If we drop table we should also drop all triggers associated with it.
To do this we have to check for existence of .TRG file when we are
dropping table and delete it too.
Windows to call CreateFileMapping() with correct arguments, and
propogating the introduction of query_id_t to everywhere query ids are
passed around. (Bug #8826)
This changeset/patch is on top of changesets 1.1814 and 1.1846
(for bugs 8033 and 8065) and now fixes bug 8535.
These changes have been built and tested successfully on build.mysql.com
handler.cc:
Added hooks for federated_db_init() and federated_db_end(),
as done with ha_archive_db does, per suggestion by Ingo in
code review of patch 1.1846.
ha_federated.h:
declaration of federated_db_init() and federated_db_end()
ha_federated.cc:
- Fixed some indentation problems from indent-ex (mainly to do with
cases where "variablename += value"
- Added federated_db_init() and federated_db_end(), as done with
archive, which also handler more elegantly one of the memory leaks
from bug 8033 where the federated_mutex was not freed
- Removed extrenous debug messages in parse_url()
- Fixed bug 8535, caused by NULL being quoted in write_row. This used to
work (incorrectly) but a recent change was made in the server that
exposed this
Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon)
Created Field::make_field() and made Field_num::make_field() to call this
Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead
Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer.
Renamed TABLE_SHARE->real_name to table_name
Renamed TABLE->table_name to alias
Renamed TABLE_LIST->real_name to table_name
Changed my_error() to print error messages, which come from
arbitrary registered ranges of error messages. Messages can
be unregistered (and should be at end of the program).
Added registration of handler error messages.
Added a new mi_print_error() macro and a new
mi_report_error() function, which supply error
messages with a table name.
Added calls to mi_print_error() or mi_report_error()
at all places in MyISAM, where table corruption is detected.
CREATE DATABASE statement used the current database instead of the
database created when checking conditions for replication.
CREATE/DROP/ALTER DATABASE statements are now replicated based on
the manipulated database.
enabled query cache for ndb
modified engine interface somewhat
sql/ha_innodb.h
enabled query cache for ndb
modified engine interface somewhat
sql/ha_ndbcluster.cc
enabled query cache for ndb
modified engine interface somewhat
ndb will only allow caching and retrieval if running autocommit
- return false, but do not invalidate
commit count is used as engine data, i.e.
- store commit count before store of cache
- allow retrieval if commit count has not changed on a table
- invalidate if commit count has changed
sql/ha_ndbcluster.h
enabled query cache for ndb
modified engine interface somewhat
sql/handler.cc
enabled query cache for ndb
modified engine interface somewhat
sql/handler.h
enabled query cache for ndb
modified engine interface somewhat
new virtual handler method cached_table_registration called on each table before alowing store in query cache
- return TRUE - ok to cache, FALSE - not allowed to cache, invalidate queries if engine_data below has changed
- sets ulonglong (engine_data) that is stored in query cache for each table
- sets callback to be called for each table before usage of cached query, callback = 0 -> no check later
sql/mysql_priv.h
enabled query cache for ndb
modified engine interface somewhat
callcack prototype for callback to engine before query cache retrieval
sql/sql_cache.cc
enabled query cache for ndb
modified engine interface somewhat
if callback is set on table in cache, do callback to check if allowed to use cache
if not allowed to use cache, check if engine_data has changed, if so, invalidate all queries with that table
+ changes to store and pass callback and engine_data around
sql/sql_cache.h
enabled query cache for ndb
modified engine interface somewhat
changes to store callback and engine_data
sql/table.h
enabled query cache for ndb
modified engine interface somewhat
changes to store callback and engine_data
binlog coordinates corresponding to the dump".
The good news is that now mysqldump can be used to get an online backup of InnoDB *which works for
point-in-time recovery and replication slave creation*. Formerly, mysqldump --master-data --single-transaction
used to call in fact mysqldump --master-data, so the dump was not an online dump (took big lock all time of dump).
The only lock which is now taken in this patch is at the beginning of the dump: mysqldump does:
FLUSH TABLES WITH READ LOCK; START TRANSACTION WITH CONSISTENT SNAPSHOT; SHOW MASTER STATUS; UNLOCK TABLES;
so the lock time is in fact the time FLUSH TABLES WITH READ LOCK takes to return (can be 0 or very long, if
a table is undergoing a huge update).
I have done some more minor changes listed in the paragraph of mysqldump.c.
WL#2237 "WITH CONSISTENT SNAPSHOT clause for START TRANSACTION":
it's a START TRANSACTION which additionally starts a consistent read on all
capable storage engine (i.e. InnoDB). So, can serve as a replacement for
BEGIN; SELECT * FROM some_innodb_table LIMIT 1; which starts a consistent read too.
DROP DATABASE failed because of file ext not in TYPELIB of known extensions.
General solution - construct a TYPELIB at runtime instead of a static list.
not know there's rollback (if it's because of a dupl row), better warn
that it's happening. It can also be of use for a DBA killing a
connection and wondering what this connection is still doing now. Example:
| 5 | root | localhost | test | Killed | 10 | Rolling back | insert into i select * from j |
column types TIMESTAMP is NOT NULL by default, so in order to have
TIMESTAMP column holding NULL valaues you have to specify NULL as
one of its attributes (this needed for backward compatibility).
Main changes:
Replaced TABLE::timestamp_default_now/on_update_now members with
TABLE::timestamp_auto_set_type flag which is used everywhere
for determining if we should auto-set value of TIMESTAMP field
during this operation or not. We are also use Field_timestamp::set_time()
instead of handler::update_timestamp() in handlers.
* Changed the implementation of ndbcluster_find_files to be more efficient, using only one mutex lock
* Moved ha_find_files to end of mysql_find_files so that it can be passed the list that we are interested to find.
This allows one to setup a master <-> master replication with non conflicting auto-increment series.
Cleaned up binary log code to make it easyer to add new state variables.
Added simpler 'upper level' logic for artificial events (events that should not cause cleanups on slave).
Simplified binary log handling.
Changed how auto_increment works together with to SET INSERT_ID=# to make it more predictable: Now the inserted rows in a multi-row statement are set independent of the existing rows in the table. (Before only InnoDB did this correctly)
Fixed (together with Guilhem) bugs in mysqlbinlog regarding --offset
Prefix addresses with 0x for easier comparisons of debug logs
Fixed problem where MySQL choosed index-read even if there would be a much better range on the same index
This fix changed some 'index' queries to 'range' queries in the test suite
Don't create 'dummy' WHERE clause for trivial WHERE clauses where we can remove the WHERE clause.
This fix removed of a lot of 'Using where' notes in the test suite.
Give NOTE instead of WARNING if table/function doesn't exists when using DROP IF EXISTS
Give NOTE instead of WARNING for safe field-type conversions
added tests to alter table for "large" alter tables and truncates in ndbcluster
added debug printout in restart() in ndbcluster
added flag THD::transaction.on to enable/disable transaction
in a deadlock-free manner. This splits locking the global read lock in two steps.
This fixes a consequence of this bug, known as:
BUG#4953 'mysqldump --master-data may report incorrect binlog position if using InnoDB'
And a test.
Note: The following tests fails
- fulltext (Sergei has promised to fix)
- rpl_charset (Guilhem should fix)
- rpl_timezone (Dimitray has promised to fix)
Sanja needs to check out the calling of close_thread_tables() in sp_head.cc
Set default max_allowed_packet to be able to read help tables even if an my.cnf file with this option is present. (Bug #3938)
Don't use default arguments for ha_rnd_init()
Simple code cleanups since last pull
you just need to change this line of mysqld.cc:
#if MYSQL_VERSION_ID >= 40103
to, say,
#if MYSQL_VERSION_ID > 40103
I am noticing a failure of bdb.test; I believe this is not related to the
code I added yesterday and today, but I am checking. In any case I push this
changeset as it cannot by itself bring more mess that I *may* (or not)
already have brought with previous pushes.
a limitation of yesterday's implementation:
if there was an unfinished transaction (COMMIT not typed), and some MyISAM tables were
then updated, and then mysqld crashes, then at restart the server would use the too old
binlog offset known by InnoDB to cut the binlog, thus cutting the successful MyISAM
updates. We fix this by reporting the binlog offset into InnoDB even if InnoDB was not
affected at all by the update.
But the feature is still disabled until we decide if it can go into 4.1.3.
more logical table/index_flags
return HA_ERR_WRONG_COMMAND instead of abstract methods where appropriate
max_keys and other limits renamed to max_supported_keys/etc
max_keys/etc are now wrappers to max_supported_keys/etc
ha_index_init/ha_rnd_init/ha_index_end/ha_rnd_end are now wrappers to real {index,rnd}_{init,end} to enforce strict pairing
names to lower case using latin1 instead of utf-8 in sql_acl.cc if lower_case_table_names
was on. Also replaced in other such places system_charset_info with files_charset_info
for consistency.
New records_in_range() interface (similar to read_range())
Macros for faster bitmap handling
Simplify read_range() code (#WL1786)
New general key_cmp() function to compare keys
- New solution after discussions with Sergei, no handler specific code or error messages should be in sql layer.
next_result, only check for error if check is -1
Improved index_read
Added two status variables:
binlog_cache_use - counts number of transactions that used somehow
transaction temporary binary log.
binlog_cache_disk_use - counts number of transactions that required
disk I/O for storing info in this this binary log.
init the binlog_cache (THD::transaction.trans_log).
I have checked all places where trans_log is used, because as now
it may not be inited in some cases, we have to be cautious
(will forward this commit mail to Heikki).
Added more DBUG statements
Ensure that we are comparing end space with BINARY strings
Use 'any_db' instead of '' to mean any database. (For HANDLER command)
Only strip ' ' when comparing CHAR, not other space-like characters (like \t)
Added support for lower_case_table_names=2, which is to be used on case insensitive file systems.
This tells MySQL to preserve the used case of filenames and database names to make it esier to move files between cases sensitive can case insensitive file systems (like Windows and Linux)
New multi-key-cache handling. This was needed becasue the old one didn't work reliable with MERGE tables.
ALTER TABLE table_name ... CHARACTER SET ... now changes all char/varchar/text columns to the given character set
(One must use ALTER TABLE ... DEFAULT CHARACTER SET ... to change the default character set)
Fixed that have_compress is detected properly (fixes problems with func_compress.test on platforms without zlib)
New syntax for CACHE INDEX ('keys' is optional if no index name is given and one mentions the key cache name only ones)
Removed compiler warnings
Added mysql_set_server_option() to allow clients like PHP to easaily set/reset the multi-statement flag.
This commit is related to my previos one(ChangeSet 1.1583 03/08/27 18:03:39).
Note about COMMIT&ROLLBACK:
Only 'COMMIT' statement updates the binary log.
'ROLLBACK' statement doesn't update the binlog.
fix for BUG#1113 "INSERT into non-trans table SELECT ; ROLLBACK" does not send warning"
and
fix for BUG#873 "In transaction, INSERT to non-trans table is written too early to binlog".
Now we don't always write the non-trans update immediately to the binlog;
if there is something in the binlog cache we write it to the binlog cache
(because the non-trans update could depend on a trans table which was modified
earlier in the transaction); then in case of ROLLBACK, we write the binlog
cache to the binlog, wrapped with BEGIN/ROLLBACK.
This guarantees that the slave does the same updates.
For ROLLBACK TO SAVEPOINT: when we execute a SAVEPOINT command we write it
to the binlog cache. At ROLLBACK TO SAVEPOINT, if some non-trans table was updated,
we write ROLLBACK TO SAVEPOINT to the binlog cache; when the transaction
terminates (COMMIT/ROLLBACK), the binlog cache will be flushed to the binlog
(because of the non-trans update) so we'll have SAVEPOINT and ROLLBACK TO
SAVEPOINT in the binlog.
Apart from this rare case of updates of mixed table types in transaction, the
usual way is still clear the binlog cache at ROLLBACK, or chop it at
ROLLBACK TO SAVEPOINT (meaning the SAVEPOINT command is also chopped, which
is fine).
Note that BUG#873 encompasses subbugs 1) and 2) of BUG#333 "3 binlogging bugs when doing INSERT with mixed InnoDB/MyISAM".