The problem is that when statement-based replication was enabled,
statements such as INSERT INTO .. SELECT FROM .. and CREATE TABLE
.. SELECT FROM need to grab a read lock on the source table that
does not permit concurrent inserts, which would in turn be denied
if the source table is a log table because log tables can't be
locked exclusively.
The solution is to not take such a lock when the source table is
a log table as it is unsafe to replicate log tables under statement
based replication. Furthermore, the read lock that does not permits
concurrent inserts is now only taken if statement-based replication
is enabled and if the source table is not a log table.
include/thr_lock.h:
Introduce yet another lock type that my get upgraded depending
on the binary log format. This is not a optimal solution but
can be easily improved later.
mysql-test/r/log_tables.result:
Add test case result for Bug#34306
mysql-test/suite/binlog/r/binlog_stm_row.result:
Add test case result for Bug#34306
mysql-test/suite/binlog/t/binlog_stm_row.test:
Add test case for Bug#34306
mysql-test/t/log_tables.test:
Add test case for Bug#34306
sql/lock.cc:
Assert that TL_READ_DEFAULT is not a real lock type.
sql/mysql_priv.h:
Export new function.
sql/mysqld.cc:
Remove using_update_log.
sql/sql_base.cc:
Introduce function that returns the appropriate read lock type
depending on how the statement is going to be replicated. It will
only take a TL_READ_NO_INSERT log if the binary is enabled and the
binary log format is statement-based and the table is not a log table.
sql/sql_parse.cc:
Remove using_update_log.
sql/sql_update.cc:
Use new function to choose read lock type.
sql/sql_yacc.yy:
The lock type is now decided at open_tables time. This old behavior was
actually misleading as the binary log format can be dynamically switched
and this would not change for statements that have already been parsed
when the binary log format is changed (ie: prepared statements).
Problem: logging of utf8-incompatible binary strings didn't work
Fix: hex-encoding of incompatible sequences.
mysql-test/r/log_tables.result:
Adding test
mysql-test/t/log_tables.test:
Adding test
sql/field.cc:
Copying with hex escaping
sql/field.h:
New field flag
sql/log.cc:
Marking the column "general_log.argument" as hex-escaping field.
sql/sql_string.cc:
New function to copy strings with hex-encoding of incompatible characters.
sql/sql_string.h:
Prototype for the new function
Only select entries from the general_log that were issued by the current
connection.
mysql-test/r/log_tables.result:
Update test result.
mysql-test/t/log_tables.test:
Only select entries from the current thread.
tests adjusted for Windows
mysql-test/r/log_tables.result:
Be more conservative in tests (for Windows)
mysql-test/t/log_tables.test:
Be more conservative in tests (for Windows)
add 5.1-specific test showing that 'const' access increments 'examined'
counter in slow query log.
mysql-test/r/log_tables.result:
5.1-only test showing that 'const' access increments counters
mysql-test/t/log_tables.test:
5.1-only test showing that 'const' access increments counters
into anubis.xiphis.org:/usr/home/antony/work/mysql-5.1-engines.merge
configure.in:
Auto merged
mysql-test/r/heap_btree.result:
Auto merged
mysql-test/r/log_tables.result:
Auto merged
mysql-test/r/partition.result:
Auto merged
mysql-test/r/system_mysql_db.result:
Auto merged
mysql-test/t/heap_btree.test:
Auto merged
mysql-test/t/log_tables.test:
Auto merged
mysql-test/t/partition.test:
Auto merged
mysys/my_getopt.c:
Auto merged
scripts/mysql_system_tables.sql:
Auto merged
sql/sql_base.cc:
Auto merged
"CSV does not work with NULL value in datetime fields"
Attempting to insert a row with a NULL value for a DATETIME field
results in a CSV file which the storage engine cannot read.
Don't blindly assume that "0" is acceptable for all field types,
Since CSV does not support NULL, we find out from the field the
default non-null value.
Do not permit the creation of a table with a nullable columns.
mysql-test/r/csv.result:
test for bug 31473
mysql-test/r/log_tables.result:
change in results due to bugfix 31473
mysql-test/r/system_mysql_db.result:
change in results due to bugfix 31473
mysql-test/t/csv.test:
test for bug 31473
mysql-test/t/log_tables.test:
due to bug31473, all columns in CSV tables must be declared as NOT NULL
scripts/mysql_system_tables.sql:
due to bug31473, all columns in CSV tables must be declared as NOT NULL
storage/csv/ha_tina.cc:
bug31473
Don't blindly assume that "0" is acceptable for all field types,
Since CSV does not support NULL, we find out from the field the
default non-null value.
Do not permit the creation of a table with a nullable columns;
The general log write function (general_log_print) uses printf style
arguments which need to be pre-processed, meaning that the all arguments
are copied to a single buffer and the problem is that the buffer size is
constant (1022 characters) but queries can be much larger then this.
The solution is to introduce a new log write function that accepts a
buffer and it's length as arguments. The function is to be used when
a formatted output is not required, which is the case for almost all
query write-to-log calls.
This is a incompatible change with respect to the log format of prepared
statements.
mysql-test/r/log_tables.result:
Add test case result for Bug#21557
mysql-test/t/log_tables.test:
Add test case for Bug#21557
sql/log.cc:
Introduce the logger function general_log_write which is similar to
general_log_print but accepts a single buffer and the buffer length.
The function doesn't perform any formatting and sends the buffer
directly to the underlying log handlers.
sql/log.h:
Introduce the logger function general_log_write.
sql/log_event.cc:
Pass the query buffer directly to the logger function, formatting
is not required on this case.
sql/mysql_priv.h:
Prototype for the logger function general_log_write.
sql/sp_head.cc:
Pass the query buffer directly to the logger function, formatting
is not required on this case.
sql/sql_parse.cc:
Pass the buffer directly to the logger function when formatting
is not required.
sql/sql_prepare.cc:
Don't log the statement id, it avoids making a extra copy of the query
and the query is not truncated anymore if it exceeds the limit.
Before this patch, failures to write to the log tables (mysql.slow_log
and mysql.general_log) were improperly printed (the time was printed twice),
or not printed at all.
With this patch, failures to write to the log tables is reported in the
error log, for all cases of failures.
mysql-test/r/log_tables.result:
Bug#27858 (Failing to log to a log table doesn't log anything to error log)
mysql-test/t/log_tables.test:
Bug#27858 (Failing to log to a log table doesn't log anything to error log)
sql/log.cc:
Bug#27858 (Failing to log to a log table doesn't log anything to error log)
mysql-test/r/log_tables.result:
Update results.
mysql-test/t/log_tables.test:
Silence a race condition: TRUNCATE code issues mysql_frm_type without
a metadata lock, and finds no table if hits the moment when
ALTER is swapping two tables.
Bug#25422 (Hang with log tables)
Bug 17876 (Truncating mysql.slow_log in a SP after using cursor locks the
thread)
Bug 23044 (Warnings on flush of a log table)
Bug 29129 (Resetting general_log while the GLOBAL READ LOCK is set causes
a deadlock)
Prior to this fix, the server would hang when performing concurrent
ALTER TABLE or TRUNCATE TABLE statements against the LOG TABLES,
which are mysql.general_log and mysql.slow_log.
The root cause traces to the following code:
in sql_base.cc, open_table()
if (table->in_use != thd)
{
/* wait_for_condition will unlock LOCK_open for us */
wait_for_condition(thd, &LOCK_open, &COND_refresh);
}
The problem with this code is that the current implementation of the
LOGGER creates 'fake' THD objects, like
- Log_to_csv_event_handler::general_log_thd
- Log_to_csv_event_handler::slow_log_thd
which are not associated to a real thread running in the server,
so that waiting for these non-existing threads to release table locks
cause the dead lock.
In general, the design of Log_to_csv_event_handler does not fit into the
general architecture of the server, so that the concept of general_log_thd
and slow_log_thd has to be abandoned:
- this implementation does not work with table locking
- it will not work with commands like SHOW PROCESSLIST
- having the log tables always opened does not integrate well with DDL
operations / FLUSH TABLES / SET GLOBAL READ_ONLY
With this patch, the fundamental design of the LOGGER has been changed to:
- always open and close a log table when writing a log
- remove totally the usage of fake THD objects
- clarify how locking of log tables is implemented in general.
See WL#3984 for details related to the new locking design.
Additional changes (misc bugs exposed and fixed):
1)
mysqldump which would ignore some tables in dump_all_tables_in_db(),
but forget to ignore the same in dump_all_views_in_db().
2)
mysqldump would also issue an empty "LOCK TABLE" command when all the tables
to lock are to be ignored (numrows == 0), instead of not issuing the query.
3)
Internal errors handlers could intercept errors but not warnings
(see sql_error.cc).
4)
Implementing a nested call to open tables, for the performance schema tables,
exposed an existing bug in remove_table_from_cache(), which would perform:
in_use->some_tables_deleted=1;
against another thread, without any consideration about thread locking.
This call inside remove_table_from_cache() was not required anyway,
since calling mysql_lock_abort() takes care of aborting -- cleanly -- threads
that might hold a lock on a table.
This line (in_use->some_tables_deleted=1) has been removed.
sql/handler.cc:
Moved logic for system / log tables in the SQL layer.
sql/handler.h:
Moved logic for system / log tables in the SQL layer.
sql/lock.cc:
Revised locking of log tables
sql/log.cc:
Major cleanup: changed how log tables are locked / written to.
sql/log.h:
Major cleanup: changed how log tables are locked / written to.
sql/mysql_priv.h:
performance schema helpers
sql/slave.cc:
open_ltable() lock flags
sql/sp.cc:
open_ltable() lock flags
sql/sql_acl.cc:
open_ltable() lock flags
sql/sql_class.h:
performance schema helpers
sql/sql_delete.cc:
log tables cleanup in TRUNCATE
sql/sql_error.cc:
Internal handlers can also intercept warnings
sql/sql_insert.cc:
open_ltable() lock flags
sql/sql_parse.cc:
performance schema helpers
sql/sql_plugin.cc:
open_ltable() lock flags
sql/sql_rename.cc:
log tables cleanup in RENAME
sql/sql_servers.cc:
open_ltable() lock flags
sql/sql_show.cc:
Move INFORMATION_SCHEMA_NAME to table.cc
sql/sql_table.cc:
log tables cleanup (admin operations, ALTER TABLE)
sql/sql_udf.cc:
open_ltable() lock flags
sql/table.cc:
Implemented TABLE_CATEGORY.
sql/share/errmsg.txt:
Changed the wording and name of ER_CANT_READ_LOCK_LOG_TABLE
sql/table.h:
Implemented TABLE_CATEGORY.
storage/csv/ha_tina.cc:
Moved logic for system / log tables in the SQL layer.
storage/csv/ha_tina.h:
Moved logic for system / log tables in the SQL layer.
storage/myisam/ha_myisam.cc:
Moved logic for system / log tables in the SQL layer.
storage/myisam/ha_myisam.h:
Moved logic for system / log tables in the SQL layer.
client/mysqldump.c:
Don't lock tables in the ignore list.
Don't issue empty LOCK TABLES queries.
sql/sql_base.cc:
log tables cleanup
performance schema helpers
mysql-test/r/ps.result:
Adjust test results
mysql-test/r/show_check.result:
Adjust test results
mysql-test/r/status.result:
Adjust test results
mysql-test/t/log_state.test:
Added tests for Bug#29129
mysql-test/t/ps.test:
Make the test output deterministic
mysql-test/t/show_check.test:
Make the test output deterministic
mysql-test/r/log_state.result:
Changed the default location of the log output to LOG_FILE,
for backward compatibility with MySQL 5.0
---
Adjust test results
mysql-test/r/log_tables.result:
cleanup for -ps-protocol
mysql-test/t/log_tables.test:
cleanup for -ps-protocol
sql/set_var.cc:
Changed the default location of the log output to LOG_FILE,
for backward compatibility with MySQL 5.0
---
log tables cleanup
numbers)
Before this patch, the code in the class Log_to_csv_event_handler, which is
used by the global LOGGER object to write to the tables mysql.slow_log and
mysql_general_log, was supporting only records of the format defined for
these tables in the database creation scripts.
Also before this patch, the server would allow, with certain limitations,
to perform ALTER TABLE on the LOG TABLES.
As implemented, the behavior of the server, with regards to LOG TABLES,
is inconsistent:
- either ALTER TABLES on LOG TABLES should be prohibited,
and the code writing to these tables can make assumptions on the record
format,
- or ALTER TABLE on LOG TABLES is permitted, in which case the code
writing a record to these tables should be more flexible and honor
new fields.
In particular, adding an AUTO_INCREMENT column to the logs,
does not work as expected (per the bug report).
Given that the ALTER TABLE on log tables statement has been explicitly
implemented to check that the log should be off to perform the operation,
and that current test cases already cover this, the user expectation is
already set that this is a "feature" and should be supported.
With this patch, the server will:
- populate AUTO INCREMENT columns if present,
- populate any additional column with it's default value
when writing a record to the LOG TABLES.
Tests are provided, that detail the precise sequence of statements
a SUPER user might want to perform to add more columns to the log tables.
mysql-test/r/log_tables.result:
Test case for bug#27857
mysql-test/t/log_tables.test:
Test case for bug#27857
sql/log.cc:
Set extra fields in log tables with default values.
"Server Variables for Plugins"
Implement support for plugins to declare server variables.
Demonstrate functionality by removing InnoDB specific code from sql/*
New feature for HASH - HASH_UNIQUE flag
New feature for DYNAMIC_ARRAY - initializer accepts preallocated ptr.
Completed support for plugin reference counting.
include/hash.h:
New flag for HASH
HASH_UNIQUE
include/my_getopt.h:
New data types for options: ENUM and SET.
Use typelib to enumerate possible values.
New flag variable:
my_getopt_skip_unknown
include/my_sys.h:
change to DYNAMIC_ARRAY init functions to support pre-allocated buffers
include/mysql.h:
relocate inclusion of typelib due to longlong requirement
include/mysql/plugin.h:
wl2936
New declarations for plugin server variable support.
New functions for use by plugins
include/mysys_err.h:
new my_getopt return value: EXIT_ARGUMENT_INVALID
include/typelib.h:
new typelib function: find_typeset(), returns an int which is a SET of
the elements in the typelib
mysql-test/r/im_utils.result:
change to more specific command line settings
--skip-innodb => --skip-plugin-innodb
etc.
mysql-test/r/log_tables.result:
set default storage engine to MEMORY so that test will succeed even
when some of the other named storage engines are not present
mysql-test/r/ndb_dd_basic.result:
change in error message
mysql-test/r/partition_innodb.result:
change in results
mysql-test/r/ps_1general.result:
bdb doesn't exist, use myisam for a non-transactional engine
mysql-test/r/variables.result:
information schema doesn't sort row results for server variables.
mysql-test/t/log_tables.test:
set default storage engine to MEMORY so that test will succeed even
when some of the other named storage engines are not present
mysql-test/t/ndb_dd_basic.test:
ALTER LOGFILE GROUP no longer silently fail here
mysql-test/t/partition_innodb.test:
ALTER TABLE no longer silently fails for unknown storage engine
mysql-test/t/ps_1general.test:
remove unneccessary parts
use myisam as it is an always present non-transactional engine
mysql-test/t/variables.test:
information schema doesn't sort row results for server variables.
mysql-test/t/warnings_engine_disabled-master.opt:
use the new style command line option
mysys/array.c:
change to DYNAMIC_ARRAY init functions to support pre-allocated buffers
mysys/hash.c:
New flag for HASH
HASH_UNIQUE
Implement HASH_UNIQUE functionality by performing a hash_search
mysys/my_getopt.c:
New data types for options: ENUM and SET.
Use typelib to enumerate possible values.
New flag variable:
my_getopt_skip_unknown
mysys/typelib.c:
new typelib function: find_typeset(), returns an int which is a SET of
the elements in the typelib
sql/ha_ndbcluster.cc:
use ha_statistic_increment() method instead of
statistic_increment() function
ha_ndbcluster variable has gone away.
sql/ha_partition.cc:
fix for reference counting
sql/ha_partition.h:
fix for reference counting
sql/handler.cc:
fixes for reference counting
sql/handler.h:
fixes for reference counting
some new methods to aid storage engine writers
sql/item_func.cc:
find_sys_var() function now requires thd
sql/item_sum.cc:
fixes for ref counting
sql/mysql_priv.h:
remove unneccessary globals.
new lock: LOCK_system_variables_hash
sql/mysqld.cc:
Remove InnoBase specific code.
Support plugin command line processing.
sql/set_var.cc:
Remove InnoBase specific declarations
Remove redundant declarations
changes to permit new variables at run time
changes for ref counting
sql/set_var.h:
changes to permit new variables at run time
changes for ref counting
sql/sql_base.cc:
changes for ref counting
sql/sql_cache.cc:
mark code as needing work in the future
sql/sql_class.cc:
new functions to aid plugin authors
initialize variables for dynamic plugin variables
sql/sql_class.h:
remove InnoBase specific declarations
New declarations for plugin variables.
sql/sql_connect.cc:
initialization and cleanup of plugin variables
sql/sql_delete.cc:
change for ref counting
sql/sql_insert.cc:
change for ref counting
sql/sql_lex.cc:
changes for ref counting and plugin variables
sql/sql_lex.h:
add properties for plugin ref counting,
add to distructor to clean up
sql/sql_partition.cc:
changes for ref counting
sql/sql_plugin.cc:
WL2936
Plugin Variables
New methods and code to support server variables for plugins.
New code to complete plugin reference counting
Debug code adds further indirection so that malloc debugging can be
used to aid finding plugin ref count leaks
sql/sql_plugin.h:
WL2936
Plugin Variables
New methods and code to support server variables for plugins.
New code to complete plugin reference counting
Debug code adds further indirection so that malloc debugging can be
used to aid finding plugin ref count leaks
sql/sql_repl.cc:
replication system variables moved here from set_var.cc
sql/sql_repl.h:
new function to initialise replication server variables
sql/sql_select.cc:
changes for ref counting
sql/sql_show.cc:
changes for ref counting
sql/sql_table.cc:
changes for ref counting
sql/sql_tablespace.cc:
use supplied functions instead of digging into data structures manually
sql/sql_yacc.yy:
changes for ref counting
find_sys_var() now requires thd parameter
changes on reporting errors to keep user-visible behaviour the same.
sql/structs.h:
changes for ref counting
sql/table.cc:
changes for ref counting
sql/table.h:
changes for ref counting
storage/federated/ha_federated.cc:
use ha_statistic_increment() method instead of statistic_increment()
function
storage/heap/ha_heap.cc:
use ha_statistic_increment() method instead of statistic_increment()
function
storage/innobase/handler/ha_innodb.cc:
use ha_statistic_increment() method instead of statistic_increment()
function
WL2936
Move InnoBase specific code out of mysqld.cc and into here
Declare all required server variables for InnoBase
storage/innobase/include/trx0trx.h:
store a bit more state so that InnoBase does not have to dig into
mysqld internal data structures.
storage/myisam/ha_myisam.cc:
use ha_statistic_increment() method instead of statistic_increment()
function
storage/myisammrg/ha_myisammrg.cc:
use ha_statistic_increment() method instead of statistic_increment()
function
into mysql.com:/home/cps/mysql/devel/5.1-rename-bug
mysql-test/r/log_tables.result:
Auto merged
mysql-test/t/log_tables.test:
Auto merged
sql/handler.cc:
Auto merged
sql/handler.h:
Auto merged
sql/log.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/sql_delete.cc:
Auto merged
sql/sql_rename.cc:
Auto merged
sql/table.cc:
Auto merged
storage/csv/ha_tina.cc:
Auto merged
storage/myisam/ha_myisam.cc:
Auto merged
sql/log.cc:
manual merge
sql/share/errmsg.txt:
manual merge
sql/sql_table.cc:
manual merge
Bug #21785 "Server crashes after rename of the log table" and
Bug #21966 "Strange warnings on create like/repair of the log
tables"
According to the patch, from now on, one should use RENAME to
perform a log table rotation (this should also be reflected in
the manual).
Here is a sample:
use mysql;
CREATE TABLE IF NOT EXISTS general_log2 LIKE general_log;
RENAME TABLE general_log TO general_log_backup, general_log2 TO general_log;
The rules for Rename of the log tables are following:
IF 1. Log tables are enabled
AND 2. Rename operates on the log table and nothing is being
renamed to the log table.
DO 3. Throw an error message.
ELSE 4. Perform rename.
The very RENAME query will go the the old (backup) table. This is
consistent with the behavoiur we have with binlog ROTATE LOGS
statement.
Other problems, which are solved by the patch are:
1) Now REPAIR of the log table is exclusive operation (as it should be), this
also eliminates lock-related warnings. and
2) CREATE LIKE TABLE now usese usual read lock on the source table rather
then name lock, which is too restrictive. This way we get rid of another
log table-related warning, which occured because of the above fact
(as a side-effect, name lock resulted in a warning).
mysql-test/r/log_tables.result:
update result file
mysql-test/t/log_tables.test:
Add tests for the bugs
sql/handler.cc:
update comment
sql/handler.h:
update function to reflect changes in log tables
locking logic.
sql/lock.cc:
Now we allow locking of the log tables for "privileged" threads
Privileged thread must explicitly close and lock log tables. This
is required for admin operations such as REPAIR.
sql/log.cc:
Changes to the file:
1) Add checks for table schema. It's more important now,
as we allow rename of the log tables. Since we should
check for schema when writing to a log table.
E.g. if one created a table with one-only comlumn and
renamed it to general_log, the server should cope with
it.
2) refactor LOGGER::flush(), so that we can now use the same
machinery as we use in FLUSH LOGS in other statements:
whenever we have to perform a serious operation on the log
tables, we have to
(a) lock logger, which blocks other concurrent statements (such
as selects) (b) close logs. Then perform an
exclusive operation, c) reenable logs and d) unlock logger.
3) Add a function to check if a given table is a log table.
4) Add support for "privileged" thread
5) merge is_[general/slow]_log_table_enabled() into one function.
6) Add new function: reopen _log_tables, which reopens the tables,
which were enabled (after temporary close, required for admin
operation)
sql/log.h:
1) add a new call close_n_lock_tables(). Now we use it instead of
LOGGER::flush() in FLUSH LOGS implementation.
2) add a prototype for the function to check if a given
table is a log table;
3) add privileged table flag to table logger
4) merge is_[general/slow]_log_table_enabled()
into one function.
sql/mysql_priv.h:
move log table defines to log.h
sql/sql_delete.cc:
use new function check_if_log_table() instead of direct strcmp
sql/sql_rename.cc:
Traverse the list of tables in mysql_rename_tables
to make sure that log tables are processed correctly
(that is, according to the rules specified in the
main CS comment)
sql/sql_table.cc:
1) mysql_admin_table() should disable logs if it performs
exclusive admin operation on a log table. This way we
also eliminate warning on REPAIR of the log table.
2) mysql_create_like_table should read-lock the source table
instead getting name lock on it. Name lock is too restrictive
in this case.
sql/share/errmsg.txt:
Add a new error message for rename of the log tables
sql/table.cc:
use new function instead of direct strcmp.
change my_strcasecmp() -> strcmp(), when
comparing system db and table names
storage/csv/ha_tina.cc:
update function to reflect changes in log tables
locking logic.
storage/myisam/ha_myisam.cc:
update function to reflect changes in log tables
locking logic.
Bug #18559 "log tables cannot change engine, and
gets deadlocked when dropping w/ log on":
1) Add more generic error messages
2) Add new handlerton flag for engines, which support
log tables
3) Remove (log-tables related) mutex lock in myisam to
improve performance
mysql-test/r/log_tables.result:
update result file to use new error messages
mysql-test/t/log_tables.test:
update test file with new error messages
sql/handler.h:
Add new handlerton flag, to mark whether engine
supports log tables
sql/share/errmsg.txt:
Add more generic error messages
sql/sql_table.cc:
Update error messages
storage/csv/ha_tina.cc:
CSV supports log tables
storage/myisam/ha_myisam.cc:
MyISAM supports log tables
storage/myisam/mi_write.c:
remove mutex lock to improve performance
into mysql.com:/home/cps/mysql/devel/5.1-curs-bug
sql/ha_myisam.cc:
Auto merged
sql/handler.h:
Auto merged
sql/sql_table.cc:
Auto merged
mysql-test/r/log_tables.result:
SCCS merged
mysql-test/t/log_tables.test:
SCCS merged
gets deadlocked when dropping w/ log on"
Log tables rely on concurrent insert machinery to add data.
This means that log tables are always opened and locked by
special (artificial) logger threads. Because of this, the thread
which tries to drop a log table starts to wait for the table
to be unlocked. Which will happen only if the log table is disabled.
Alike situation happens if one tries to alter a log table.
However in addition to the problem above, alter table calls
check_if_locking_is_allowed() routine for the engine. The
routine does not allow alter for the log tables. So, alter
doesn't start waiting forever for logs to be disabled, but
returns with an error.
Another problem is that not all engines could be used for
the log tables. That's because they need concurrent insert.
In this patch we:
(1) Explicitly disallow to drop/alter a log table if it
is currently used by the logger.
(2) Update MyISAM to support log tables
(3) Allow to drop log tables/alter log tables if log is
disabled
At the same time we (4) Disallow to alter log tables to
unsupported engine (after this patch CSV and MyISAM are
alowed)
Recommit with review fixes.
mysql-test/r/log_tables.result:
Update result file.
Note: there are warnings in result file. This is because of CSV
bug (Bug #21328). They should go away after it is fixed.
mysql-test/t/log_tables.test:
Add a test for the bug
sql/ha_myisam.cc:
Add log table handling to myisam: as log tables
use concurrent insert, they are typically
locked with TL_CONCURRERENT_INSERT lock. So,
disallow other threads to attempt locking of
the log tables in incompatible modes. Because
otherwise the threads will wait for the tables
to be unlocked forever.
sql/handler.cc:
Add a function to check if a table we're going to lock
is a log table and if the lock mode we want allowed
sql/handler.h:
Add a new function to check compatibility of the locking
sql/log.cc:
we shouldn't close the log table if and only
if this particular table is already closed
sql/log.h:
add new functions to check if a log is enabled
sql/share/errmsg.txt:
add new error messages
sql/sql_table.cc:
DROP and ALTER TABLE should not work on log
tables if the log tables are enabled
storage/csv/ha_tina.cc:
move function to check if the locking for the log
tables allowed to handler class, so that we can
reuse it in other engines.
storage/myisam/mi_extra.c:
add new ::extra() flag processing to myisam
storage/myisam/mi_open.c:
init log table flag
storage/myisam/mi_write.c:
update status after each write if it's a log table
storage/myisam/myisamdef.h:
Add new log table flag to myisam share.
We need it to distinguish between usual
and log tables, as for the log tables we
should provide concurrent insert in a
different way than for usual tables: we
want new rows to be immediately visible
to other threads.
Due to incorrect handling of FLUSH TABLES, log tables were marked for flush,
but not reopened. Later we started to wait for the log table to be closed
(disabled) after the flush. And as nobody disabled logs in concurrent treads,
the command lasted forever.
After internal consultations it was decided to skip logs during FLUSH TABLES.
The reasoning is that logging is done in the "log device", whatever it is
which is always active and controlled by FLUSH LOGS. So, to flush logs
one should use FLUSH LOGS, and not FLUSH TABLES.
mysql-test/r/log_tables.result:
update result file
mysql-test/t/log_tables.test:
add a test for the bug
sql/sql_base.cc:
Skip log tables during FLUSH TABLES
This test included hostname information in the output, which varies
between platforms.
mysql-test/r/log_tables.result:
Update results
mysql-test/t/log_tables.test:
Replace user_host selected from logs to avoid platform differences.
Also, fix use of replace_column in test for bug 17600.
Move plugin declarations after system functions have been checked
(Fixes problem with ndb_config failing becasue SHM is not declared)
Fixed some memory leaks
configure.in:
Move plugin declarations after system functions have been checked
(Fixes problem with ndb_config failing becasue SHM is not declared)
mysql-test/t/disabled.def:
Disabled ndb_load, as it is in 5.0
(Fails randomly in binlog_close_connection())
mysql-test/t/log_tables.test:
Update error numbers
mysql-test/t/ndb_config.test:
Moved test depending on SHM to ndb_config2.test
mysql-test/t/ndb_partition_error.test:
Update error numbers
sql/event_timed.cc:
Removed not needed line
sql/ha_berkeley.cc:
Fixed compiler warning
sql/ha_heap.cc:
Fixed compiler warning
sql/ha_innodb.cc:
Fixed compiler warning
sql/ha_myisam.cc:
Fixed compiler warning
sql/ha_myisammrg.cc:
Fixed compiler warning
sql/ha_ndbcluster.cc:
Fixed compiler warning
sql/ha_partition.cc:
Fixed compiler warning
sql/mysql_priv.h:
After merge fix
sql/mysqld.cc:
Fixed memory leak in skip_grant.test
sql/share/errmsg.txt:
Removed wrong error messages
sql/sql_lex.cc:
More debugging
Faster lex_end()
sql/sql_partition.cc:
Fixed memory leak
sql/sql_view.cc:
Fixed memory leak
mysql-test/r/ndb_config2.result:
New BitKeeper file ``mysql-test/r/ndb_config2.result''
mysql-test/t/ndb_config2.test:
New BitKeeper file ``mysql-test/t/ndb_config2.test''
Recommit with post-review fixes.
mysql-test/r/log_tables.result:
update result
mysql-test/t/log_tables.test:
add a test
sql/log.cc:
Fix NULL handling in log tables
storage/csv/ha_tina.cc:
Explicitly check fo NULLs, when writing a row.
We should not hope, that the null field was
cleaned up.
Though usually we do call Field::reset() or
restore_row(), before calling Field::set_null(),
this depency is neither documented nor enforced
by other means.
mysql-test/t/log_tables-master.opt:
New BitKeeper file ``mysql-test/t/log_tables-master.opt''
mysql-test/r/log_tables.result:
update result
mysql-test/t/log_tables.test:
add a testcase
sql/log.cc:
take into account client charset info, while logging
into a table.
sql/log.h:
pass the charset info along with the log messages
to log them correctly
The log content is obviously different in two modes,
as Queries are translated into Prepare and Execute
commands. Thus we should use test only in one mode
to get the match with result.
mysql-test/t/log_tables.test:
disable ps-protocol in log_tables test
libmysqld/lib_sql.cc:
logger initialization added
mysql-test/mysql-test-run.pl:
ndbcluster and ssl disabled in embedded server
mysql-test/mysql-test-run.sh:
ndbcluster disabled in embedded server
mysql-test/r/mysqltest.result:
result fixed
mysql-test/r/sp_notembedded.result:
result fixed
mysql-test/t/log_tables.test:
test disabled in embedded server
mysql-test/t/sp_notembedded.test:
test fixed
tree to get rid of multiple typos in CS comments and
unify the patch.
configure.in:
CSV is compiled in by default now
include/my_base.h:
add new ha_extra flag for the log tables
mysql-test/include/im_check_os.inc:
we should only run im tests if csv is on for now: im relies
on mysqld options available only in csv build.
mysql-test/include/system_db_struct.inc:
check log tables structure
mysql-test/lib/init_db.sql:
create log tables when running tests.
mysql-test/mysql-test-run.pl:
Add old logs flag to IM tests. As IM could only deal with
old logs (this feature is not needed with log tables)
mysql-test/r/connect.result:
update result
mysql-test/r/csv.result:
update result
mysql-test/r/im_utils.result:
update result
mysql-test/r/information_schema.result:
update result
mysql-test/r/mysqlcheck.result:
update result
mysql-test/r/show_check.result:
update result
mysql-test/r/system_mysql_db.result:
update result
mysql-test/t/connect.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
mysql-test/t/csv.test:
add tests for concurrent insert (the functionality is added
to CSV in this patch)
mysql-test/t/information_schema.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
mysql-test/t/mysqlcheck.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
mysql-test/t/show_check.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
mysql-test/t/system_mysql_db.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
mysql-test/t/system_mysql_db_fix.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
scripts/mysql_create_system_tables.sh:
new system tables: slow_log and general_log
scripts/mysql_fix_privilege_tables.sql:
add new log tables: use an SP to create them for
non-csv build to work fine.
sql/ha_myisam.cc:
move locking-related checks to the hanlder
sql/ha_myisam.h:
new function declared
sql/handler.h:
new virtual function is added: we should check for handler-related
locking issues in the handler
sql/lock.cc:
from now on we check for handler-related locking issues
in the handler itself rather then in lock.cc
sql/log.cc:
Add log tables support, refactoring: there are log event
handlers with common interface. They are used by the LOGGER
class, which is responsible for their initialization, cleanup
and managment. Logging to the tables provided by one of the
log event handler types.
sql/log.h:
declare new log classes
sql/log_event.cc:
convert old logging routines calls to use new API
sql/mysql_priv.h:
define common log routines and objects
sql/mysqld.cc:
Add support for the log tables. Their initalization, cleanup
and specific options.
sql/share/errmsg.txt:
add new error messages for the log tables
sql/slave.cc:
convert old logging routines calls to use new API
sql/sql_base.cc:
TABLE objects used by the logger should be skipped
during refreshes (as log tables are always opened
and locked). fix table_is_used to skip them. This
is needed for FLUSH LOGS to work
sql/sql_db.cc:
convert old logging routines calls to use new API
sql/sql_delete.cc:
fix TRUNCATE to work with log tables
sql/sql_parse.cc:
command_name is now an array of LEX_STRINGs
sql/sql_prepare.cc:
convert old logging routines calls to use new API
sql/sql_show.cc:
convert old logging routines calls to use new API
sql/sql_table.cc:
don't reoped the log tables for admin purposes
sql/table.cc:
mark log tables as such during the open
sql/table.h:
add log-related info
storage/csv/ha_tina.cc:
add support for concurrent insert (see bk commit - 5.1 tree
(petr:1.1910) for standalone patch), add log tables-specific
csv table handling.
storage/csv/ha_tina.h:
enable concurrent insert for CSV, add log table flag
mysql-test/r/log_tables.result:
New BitKeeper file ``mysql-test/r/log_tables.result''
mysql-test/t/log_tables.test:
New BitKeeper file ``mysql-test/t/log_tables.test''
