numbers)
Before this patch, the code in the class Log_to_csv_event_handler, which is
used by the global LOGGER object to write to the tables mysql.slow_log and
mysql_general_log, was supporting only records of the format defined for
these tables in the database creation scripts.
Also before this patch, the server would allow, with certain limitations,
to perform ALTER TABLE on the LOG TABLES.
As implemented, the behavior of the server, with regards to LOG TABLES,
is inconsistent:
- either ALTER TABLES on LOG TABLES should be prohibited,
and the code writing to these tables can make assumptions on the record
format,
- or ALTER TABLE on LOG TABLES is permitted, in which case the code
writing a record to these tables should be more flexible and honor
new fields.
In particular, adding an AUTO_INCREMENT column to the logs,
does not work as expected (per the bug report).
Given that the ALTER TABLE on log tables statement has been explicitly
implemented to check that the log should be off to perform the operation,
and that current test cases already cover this, the user expectation is
already set that this is a "feature" and should be supported.
With this patch, the server will:
- populate AUTO INCREMENT columns if present,
- populate any additional column with it's default value
when writing a record to the LOG TABLES.
Tests are provided, that detail the precise sequence of statements
a SUPER user might want to perform to add more columns to the log tables.
mysql-test/r/log_tables.result:
Test case for bug#27857
mysql-test/t/log_tables.test:
Test case for bug#27857
sql/log.cc:
Set extra fields in log tables with default values.
"Server Variables for Plugins"
Implement support for plugins to declare server variables.
Demonstrate functionality by removing InnoDB specific code from sql/*
New feature for HASH - HASH_UNIQUE flag
New feature for DYNAMIC_ARRAY - initializer accepts preallocated ptr.
Completed support for plugin reference counting.
include/hash.h:
New flag for HASH
HASH_UNIQUE
include/my_getopt.h:
New data types for options: ENUM and SET.
Use typelib to enumerate possible values.
New flag variable:
my_getopt_skip_unknown
include/my_sys.h:
change to DYNAMIC_ARRAY init functions to support pre-allocated buffers
include/mysql.h:
relocate inclusion of typelib due to longlong requirement
include/mysql/plugin.h:
wl2936
New declarations for plugin server variable support.
New functions for use by plugins
include/mysys_err.h:
new my_getopt return value: EXIT_ARGUMENT_INVALID
include/typelib.h:
new typelib function: find_typeset(), returns an int which is a SET of
the elements in the typelib
mysql-test/r/im_utils.result:
change to more specific command line settings
--skip-innodb => --skip-plugin-innodb
etc.
mysql-test/r/log_tables.result:
set default storage engine to MEMORY so that test will succeed even
when some of the other named storage engines are not present
mysql-test/r/ndb_dd_basic.result:
change in error message
mysql-test/r/partition_innodb.result:
change in results
mysql-test/r/ps_1general.result:
bdb doesn't exist, use myisam for a non-transactional engine
mysql-test/r/variables.result:
information schema doesn't sort row results for server variables.
mysql-test/t/log_tables.test:
set default storage engine to MEMORY so that test will succeed even
when some of the other named storage engines are not present
mysql-test/t/ndb_dd_basic.test:
ALTER LOGFILE GROUP no longer silently fail here
mysql-test/t/partition_innodb.test:
ALTER TABLE no longer silently fails for unknown storage engine
mysql-test/t/ps_1general.test:
remove unneccessary parts
use myisam as it is an always present non-transactional engine
mysql-test/t/variables.test:
information schema doesn't sort row results for server variables.
mysql-test/t/warnings_engine_disabled-master.opt:
use the new style command line option
mysys/array.c:
change to DYNAMIC_ARRAY init functions to support pre-allocated buffers
mysys/hash.c:
New flag for HASH
HASH_UNIQUE
Implement HASH_UNIQUE functionality by performing a hash_search
mysys/my_getopt.c:
New data types for options: ENUM and SET.
Use typelib to enumerate possible values.
New flag variable:
my_getopt_skip_unknown
mysys/typelib.c:
new typelib function: find_typeset(), returns an int which is a SET of
the elements in the typelib
sql/ha_ndbcluster.cc:
use ha_statistic_increment() method instead of
statistic_increment() function
ha_ndbcluster variable has gone away.
sql/ha_partition.cc:
fix for reference counting
sql/ha_partition.h:
fix for reference counting
sql/handler.cc:
fixes for reference counting
sql/handler.h:
fixes for reference counting
some new methods to aid storage engine writers
sql/item_func.cc:
find_sys_var() function now requires thd
sql/item_sum.cc:
fixes for ref counting
sql/mysql_priv.h:
remove unneccessary globals.
new lock: LOCK_system_variables_hash
sql/mysqld.cc:
Remove InnoBase specific code.
Support plugin command line processing.
sql/set_var.cc:
Remove InnoBase specific declarations
Remove redundant declarations
changes to permit new variables at run time
changes for ref counting
sql/set_var.h:
changes to permit new variables at run time
changes for ref counting
sql/sql_base.cc:
changes for ref counting
sql/sql_cache.cc:
mark code as needing work in the future
sql/sql_class.cc:
new functions to aid plugin authors
initialize variables for dynamic plugin variables
sql/sql_class.h:
remove InnoBase specific declarations
New declarations for plugin variables.
sql/sql_connect.cc:
initialization and cleanup of plugin variables
sql/sql_delete.cc:
change for ref counting
sql/sql_insert.cc:
change for ref counting
sql/sql_lex.cc:
changes for ref counting and plugin variables
sql/sql_lex.h:
add properties for plugin ref counting,
add to distructor to clean up
sql/sql_partition.cc:
changes for ref counting
sql/sql_plugin.cc:
WL2936
Plugin Variables
New methods and code to support server variables for plugins.
New code to complete plugin reference counting
Debug code adds further indirection so that malloc debugging can be
used to aid finding plugin ref count leaks
sql/sql_plugin.h:
WL2936
Plugin Variables
New methods and code to support server variables for plugins.
New code to complete plugin reference counting
Debug code adds further indirection so that malloc debugging can be
used to aid finding plugin ref count leaks
sql/sql_repl.cc:
replication system variables moved here from set_var.cc
sql/sql_repl.h:
new function to initialise replication server variables
sql/sql_select.cc:
changes for ref counting
sql/sql_show.cc:
changes for ref counting
sql/sql_table.cc:
changes for ref counting
sql/sql_tablespace.cc:
use supplied functions instead of digging into data structures manually
sql/sql_yacc.yy:
changes for ref counting
find_sys_var() now requires thd parameter
changes on reporting errors to keep user-visible behaviour the same.
sql/structs.h:
changes for ref counting
sql/table.cc:
changes for ref counting
sql/table.h:
changes for ref counting
storage/federated/ha_federated.cc:
use ha_statistic_increment() method instead of statistic_increment()
function
storage/heap/ha_heap.cc:
use ha_statistic_increment() method instead of statistic_increment()
function
storage/innobase/handler/ha_innodb.cc:
use ha_statistic_increment() method instead of statistic_increment()
function
WL2936
Move InnoBase specific code out of mysqld.cc and into here
Declare all required server variables for InnoBase
storage/innobase/include/trx0trx.h:
store a bit more state so that InnoBase does not have to dig into
mysqld internal data structures.
storage/myisam/ha_myisam.cc:
use ha_statistic_increment() method instead of statistic_increment()
function
storage/myisammrg/ha_myisammrg.cc:
use ha_statistic_increment() method instead of statistic_increment()
function
into mysql.com:/home/cps/mysql/devel/5.1-rename-bug
mysql-test/r/log_tables.result:
Auto merged
mysql-test/t/log_tables.test:
Auto merged
sql/handler.cc:
Auto merged
sql/handler.h:
Auto merged
sql/log.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/sql_delete.cc:
Auto merged
sql/sql_rename.cc:
Auto merged
sql/table.cc:
Auto merged
storage/csv/ha_tina.cc:
Auto merged
storage/myisam/ha_myisam.cc:
Auto merged
sql/log.cc:
manual merge
sql/share/errmsg.txt:
manual merge
sql/sql_table.cc:
manual merge
Bug #21785 "Server crashes after rename of the log table" and
Bug #21966 "Strange warnings on create like/repair of the log
tables"
According to the patch, from now on, one should use RENAME to
perform a log table rotation (this should also be reflected in
the manual).
Here is a sample:
use mysql;
CREATE TABLE IF NOT EXISTS general_log2 LIKE general_log;
RENAME TABLE general_log TO general_log_backup, general_log2 TO general_log;
The rules for Rename of the log tables are following:
IF 1. Log tables are enabled
AND 2. Rename operates on the log table and nothing is being
renamed to the log table.
DO 3. Throw an error message.
ELSE 4. Perform rename.
The very RENAME query will go the the old (backup) table. This is
consistent with the behavoiur we have with binlog ROTATE LOGS
statement.
Other problems, which are solved by the patch are:
1) Now REPAIR of the log table is exclusive operation (as it should be), this
also eliminates lock-related warnings. and
2) CREATE LIKE TABLE now usese usual read lock on the source table rather
then name lock, which is too restrictive. This way we get rid of another
log table-related warning, which occured because of the above fact
(as a side-effect, name lock resulted in a warning).
mysql-test/r/log_tables.result:
update result file
mysql-test/t/log_tables.test:
Add tests for the bugs
sql/handler.cc:
update comment
sql/handler.h:
update function to reflect changes in log tables
locking logic.
sql/lock.cc:
Now we allow locking of the log tables for "privileged" threads
Privileged thread must explicitly close and lock log tables. This
is required for admin operations such as REPAIR.
sql/log.cc:
Changes to the file:
1) Add checks for table schema. It's more important now,
as we allow rename of the log tables. Since we should
check for schema when writing to a log table.
E.g. if one created a table with one-only comlumn and
renamed it to general_log, the server should cope with
it.
2) refactor LOGGER::flush(), so that we can now use the same
machinery as we use in FLUSH LOGS in other statements:
whenever we have to perform a serious operation on the log
tables, we have to
(a) lock logger, which blocks other concurrent statements (such
as selects) (b) close logs. Then perform an
exclusive operation, c) reenable logs and d) unlock logger.
3) Add a function to check if a given table is a log table.
4) Add support for "privileged" thread
5) merge is_[general/slow]_log_table_enabled() into one function.
6) Add new function: reopen _log_tables, which reopens the tables,
which were enabled (after temporary close, required for admin
operation)
sql/log.h:
1) add a new call close_n_lock_tables(). Now we use it instead of
LOGGER::flush() in FLUSH LOGS implementation.
2) add a prototype for the function to check if a given
table is a log table;
3) add privileged table flag to table logger
4) merge is_[general/slow]_log_table_enabled()
into one function.
sql/mysql_priv.h:
move log table defines to log.h
sql/sql_delete.cc:
use new function check_if_log_table() instead of direct strcmp
sql/sql_rename.cc:
Traverse the list of tables in mysql_rename_tables
to make sure that log tables are processed correctly
(that is, according to the rules specified in the
main CS comment)
sql/sql_table.cc:
1) mysql_admin_table() should disable logs if it performs
exclusive admin operation on a log table. This way we
also eliminate warning on REPAIR of the log table.
2) mysql_create_like_table should read-lock the source table
instead getting name lock on it. Name lock is too restrictive
in this case.
sql/share/errmsg.txt:
Add a new error message for rename of the log tables
sql/table.cc:
use new function instead of direct strcmp.
change my_strcasecmp() -> strcmp(), when
comparing system db and table names
storage/csv/ha_tina.cc:
update function to reflect changes in log tables
locking logic.
storage/myisam/ha_myisam.cc:
update function to reflect changes in log tables
locking logic.
Bug #18559 "log tables cannot change engine, and
gets deadlocked when dropping w/ log on":
1) Add more generic error messages
2) Add new handlerton flag for engines, which support
log tables
3) Remove (log-tables related) mutex lock in myisam to
improve performance
mysql-test/r/log_tables.result:
update result file to use new error messages
mysql-test/t/log_tables.test:
update test file with new error messages
sql/handler.h:
Add new handlerton flag, to mark whether engine
supports log tables
sql/share/errmsg.txt:
Add more generic error messages
sql/sql_table.cc:
Update error messages
storage/csv/ha_tina.cc:
CSV supports log tables
storage/myisam/ha_myisam.cc:
MyISAM supports log tables
storage/myisam/mi_write.c:
remove mutex lock to improve performance
into mysql.com:/home/cps/mysql/devel/5.1-curs-bug
sql/ha_myisam.cc:
Auto merged
sql/handler.h:
Auto merged
sql/sql_table.cc:
Auto merged
mysql-test/r/log_tables.result:
SCCS merged
mysql-test/t/log_tables.test:
SCCS merged
gets deadlocked when dropping w/ log on"
Log tables rely on concurrent insert machinery to add data.
This means that log tables are always opened and locked by
special (artificial) logger threads. Because of this, the thread
which tries to drop a log table starts to wait for the table
to be unlocked. Which will happen only if the log table is disabled.
Alike situation happens if one tries to alter a log table.
However in addition to the problem above, alter table calls
check_if_locking_is_allowed() routine for the engine. The
routine does not allow alter for the log tables. So, alter
doesn't start waiting forever for logs to be disabled, but
returns with an error.
Another problem is that not all engines could be used for
the log tables. That's because they need concurrent insert.
In this patch we:
(1) Explicitly disallow to drop/alter a log table if it
is currently used by the logger.
(2) Update MyISAM to support log tables
(3) Allow to drop log tables/alter log tables if log is
disabled
At the same time we (4) Disallow to alter log tables to
unsupported engine (after this patch CSV and MyISAM are
alowed)
Recommit with review fixes.
mysql-test/r/log_tables.result:
Update result file.
Note: there are warnings in result file. This is because of CSV
bug (Bug #21328). They should go away after it is fixed.
mysql-test/t/log_tables.test:
Add a test for the bug
sql/ha_myisam.cc:
Add log table handling to myisam: as log tables
use concurrent insert, they are typically
locked with TL_CONCURRERENT_INSERT lock. So,
disallow other threads to attempt locking of
the log tables in incompatible modes. Because
otherwise the threads will wait for the tables
to be unlocked forever.
sql/handler.cc:
Add a function to check if a table we're going to lock
is a log table and if the lock mode we want allowed
sql/handler.h:
Add a new function to check compatibility of the locking
sql/log.cc:
we shouldn't close the log table if and only
if this particular table is already closed
sql/log.h:
add new functions to check if a log is enabled
sql/share/errmsg.txt:
add new error messages
sql/sql_table.cc:
DROP and ALTER TABLE should not work on log
tables if the log tables are enabled
storage/csv/ha_tina.cc:
move function to check if the locking for the log
tables allowed to handler class, so that we can
reuse it in other engines.
storage/myisam/mi_extra.c:
add new ::extra() flag processing to myisam
storage/myisam/mi_open.c:
init log table flag
storage/myisam/mi_write.c:
update status after each write if it's a log table
storage/myisam/myisamdef.h:
Add new log table flag to myisam share.
We need it to distinguish between usual
and log tables, as for the log tables we
should provide concurrent insert in a
different way than for usual tables: we
want new rows to be immediately visible
to other threads.
Due to incorrect handling of FLUSH TABLES, log tables were marked for flush,
but not reopened. Later we started to wait for the log table to be closed
(disabled) after the flush. And as nobody disabled logs in concurrent treads,
the command lasted forever.
After internal consultations it was decided to skip logs during FLUSH TABLES.
The reasoning is that logging is done in the "log device", whatever it is
which is always active and controlled by FLUSH LOGS. So, to flush logs
one should use FLUSH LOGS, and not FLUSH TABLES.
mysql-test/r/log_tables.result:
update result file
mysql-test/t/log_tables.test:
add a test for the bug
sql/sql_base.cc:
Skip log tables during FLUSH TABLES
This test included hostname information in the output, which varies
between platforms.
mysql-test/r/log_tables.result:
Update results
mysql-test/t/log_tables.test:
Replace user_host selected from logs to avoid platform differences.
Also, fix use of replace_column in test for bug 17600.
Move plugin declarations after system functions have been checked
(Fixes problem with ndb_config failing becasue SHM is not declared)
Fixed some memory leaks
configure.in:
Move plugin declarations after system functions have been checked
(Fixes problem with ndb_config failing becasue SHM is not declared)
mysql-test/t/disabled.def:
Disabled ndb_load, as it is in 5.0
(Fails randomly in binlog_close_connection())
mysql-test/t/log_tables.test:
Update error numbers
mysql-test/t/ndb_config.test:
Moved test depending on SHM to ndb_config2.test
mysql-test/t/ndb_partition_error.test:
Update error numbers
sql/event_timed.cc:
Removed not needed line
sql/ha_berkeley.cc:
Fixed compiler warning
sql/ha_heap.cc:
Fixed compiler warning
sql/ha_innodb.cc:
Fixed compiler warning
sql/ha_myisam.cc:
Fixed compiler warning
sql/ha_myisammrg.cc:
Fixed compiler warning
sql/ha_ndbcluster.cc:
Fixed compiler warning
sql/ha_partition.cc:
Fixed compiler warning
sql/mysql_priv.h:
After merge fix
sql/mysqld.cc:
Fixed memory leak in skip_grant.test
sql/share/errmsg.txt:
Removed wrong error messages
sql/sql_lex.cc:
More debugging
Faster lex_end()
sql/sql_partition.cc:
Fixed memory leak
sql/sql_view.cc:
Fixed memory leak
mysql-test/r/ndb_config2.result:
New BitKeeper file ``mysql-test/r/ndb_config2.result''
mysql-test/t/ndb_config2.test:
New BitKeeper file ``mysql-test/t/ndb_config2.test''
Recommit with post-review fixes.
mysql-test/r/log_tables.result:
update result
mysql-test/t/log_tables.test:
add a test
sql/log.cc:
Fix NULL handling in log tables
storage/csv/ha_tina.cc:
Explicitly check fo NULLs, when writing a row.
We should not hope, that the null field was
cleaned up.
Though usually we do call Field::reset() or
restore_row(), before calling Field::set_null(),
this depency is neither documented nor enforced
by other means.
mysql-test/t/log_tables-master.opt:
New BitKeeper file ``mysql-test/t/log_tables-master.opt''
mysql-test/r/log_tables.result:
update result
mysql-test/t/log_tables.test:
add a testcase
sql/log.cc:
take into account client charset info, while logging
into a table.
sql/log.h:
pass the charset info along with the log messages
to log them correctly
The log content is obviously different in two modes,
as Queries are translated into Prepare and Execute
commands. Thus we should use test only in one mode
to get the match with result.
mysql-test/t/log_tables.test:
disable ps-protocol in log_tables test
libmysqld/lib_sql.cc:
logger initialization added
mysql-test/mysql-test-run.pl:
ndbcluster and ssl disabled in embedded server
mysql-test/mysql-test-run.sh:
ndbcluster disabled in embedded server
mysql-test/r/mysqltest.result:
result fixed
mysql-test/r/sp_notembedded.result:
result fixed
mysql-test/t/log_tables.test:
test disabled in embedded server
mysql-test/t/sp_notembedded.test:
test fixed
tree to get rid of multiple typos in CS comments and
unify the patch.
configure.in:
CSV is compiled in by default now
include/my_base.h:
add new ha_extra flag for the log tables
mysql-test/include/im_check_os.inc:
we should only run im tests if csv is on for now: im relies
on mysqld options available only in csv build.
mysql-test/include/system_db_struct.inc:
check log tables structure
mysql-test/lib/init_db.sql:
create log tables when running tests.
mysql-test/mysql-test-run.pl:
Add old logs flag to IM tests. As IM could only deal with
old logs (this feature is not needed with log tables)
mysql-test/r/connect.result:
update result
mysql-test/r/csv.result:
update result
mysql-test/r/im_utils.result:
update result
mysql-test/r/information_schema.result:
update result
mysql-test/r/mysqlcheck.result:
update result
mysql-test/r/show_check.result:
update result
mysql-test/r/system_mysql_db.result:
update result
mysql-test/t/connect.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
mysql-test/t/csv.test:
add tests for concurrent insert (the functionality is added
to CSV in this patch)
mysql-test/t/information_schema.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
mysql-test/t/mysqlcheck.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
mysql-test/t/show_check.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
mysql-test/t/system_mysql_db.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
mysql-test/t/system_mysql_db_fix.test:
disable test if CSV engine is not in: result depends on the
presence of CSV-based log tables
scripts/mysql_create_system_tables.sh:
new system tables: slow_log and general_log
scripts/mysql_fix_privilege_tables.sql:
add new log tables: use an SP to create them for
non-csv build to work fine.
sql/ha_myisam.cc:
move locking-related checks to the hanlder
sql/ha_myisam.h:
new function declared
sql/handler.h:
new virtual function is added: we should check for handler-related
locking issues in the handler
sql/lock.cc:
from now on we check for handler-related locking issues
in the handler itself rather then in lock.cc
sql/log.cc:
Add log tables support, refactoring: there are log event
handlers with common interface. They are used by the LOGGER
class, which is responsible for their initialization, cleanup
and managment. Logging to the tables provided by one of the
log event handler types.
sql/log.h:
declare new log classes
sql/log_event.cc:
convert old logging routines calls to use new API
sql/mysql_priv.h:
define common log routines and objects
sql/mysqld.cc:
Add support for the log tables. Their initalization, cleanup
and specific options.
sql/share/errmsg.txt:
add new error messages for the log tables
sql/slave.cc:
convert old logging routines calls to use new API
sql/sql_base.cc:
TABLE objects used by the logger should be skipped
during refreshes (as log tables are always opened
and locked). fix table_is_used to skip them. This
is needed for FLUSH LOGS to work
sql/sql_db.cc:
convert old logging routines calls to use new API
sql/sql_delete.cc:
fix TRUNCATE to work with log tables
sql/sql_parse.cc:
command_name is now an array of LEX_STRINGs
sql/sql_prepare.cc:
convert old logging routines calls to use new API
sql/sql_show.cc:
convert old logging routines calls to use new API
sql/sql_table.cc:
don't reoped the log tables for admin purposes
sql/table.cc:
mark log tables as such during the open
sql/table.h:
add log-related info
storage/csv/ha_tina.cc:
add support for concurrent insert (see bk commit - 5.1 tree
(petr:1.1910) for standalone patch), add log tables-specific
csv table handling.
storage/csv/ha_tina.h:
enable concurrent insert for CSV, add log table flag
mysql-test/r/log_tables.result:
New BitKeeper file ``mysql-test/r/log_tables.result''
mysql-test/t/log_tables.test:
New BitKeeper file ``mysql-test/t/log_tables.test''
