The mysqld server and all clients now reads the new client-server section
Fixed that mysqldumpslow supports new slow log formats and new mysqld --slow- options
client/mysql.cc:
Read also client-server and client-mariadb sections.
client/mysql_upgrade.c:
Read also client-server and client-mariadb sections.
client/mysqladmin.cc:
Read also client-server and client-mariadb sections.
client/mysqlbinlog.cc:
Read also client-server and client-mariadb sections.
client/mysqlcheck.c:
Read also client-server and client-mariadb sections.
client/mysqldump.c:
Read also client-server and client-mariadb sections.
client/mysqlimport.c:
Read also client-server and client-mariadb sections.
client/mysqlshow.c:
Read also client-server and client-mariadb sections.
client/mysqltest.cc:
Read also client-server and client-mariadb sections.
extra/my_print_defaults.c:
Updated help text
scripts/mysql_fix_privilege_tables.sh:
Read also sections client client-server client-mariadb
scripts/mysql_install_db.pl.in:
Also allow --data=* option
Read also groups mariadb, server and client-server.
scripts/mysql_install_db.sh:
Also allow --data=* option
Read also groups mariadb, server and client-server.
Added --lose-skip-pbxt to bootstrap
scripts/mysql_secure_installation.sh:
Read also groups client-server and client-mariadb
scripts/mysqld_multi.sh:
Read also group mariadb
scripts/mysqld_safe.sh:
Read also groups mariadb server and client-server
scripts/mysqldumpslow.sh:
Fixed to support new slow log formats
Added sorting on -ae (aggregated number of retreived rows) and e (retrieved rows)
Read also group 'mariadb'
If there is many instances of same option, use last one.
Get slow log file from options log-slow-queries=filename or query-log-file=filename
Added support for future --log-basename option
sql-common/client.c:
Read also groups 'client-server' and 'client-mariadb'
tests/mysql_client_test.c:
Read also groups 'client-server' and 'client-mariadb'
tests/thread_test.c:
Read also groups 'client-server' and 'client-mariadb'
This is a merge of percona patch mysqlbinlog_fix.patch.
Apparently, MySQL 5.0 does not flag BEGIN/SAVEPOINT/COMMIT/ROLLBACK statements
in its binlogs with LOG_EVENT_SUPPRESS_USE_F like 5.1+ does. This causes
unnecessary `use` statements around such statements when the binlog is dumped
by mysqlbinlog.
Fix by always suppressing the output of `use` for these statements.
This patch adds options to annotate the binlog (and the mysqlbinlog
output) with the original SQL query for queries that are logged
using row-based replication.
strict aliasing violations.
Essentially, the problem is that large parts of the server were
developed in simpler times (last decades, pre C99 standard) when
strict aliasing and compilers supporting such optimizations were
rare to non-existent. Thus, when compiling the server with a modern
compiler that uses strict aliasing rules to perform optimizations,
there are several places in the code that might trigger undefined
behavior.
As evinced by some recent bugs, GCC does a somewhat good of job
misoptimizing such code, but on the other hand also gives warnings
about suspicious code. One problem is that the warnings aren't
always accurate, yet we can't afford to just shut them off as we
might miss real cases. False-positive cases are aggravated mostly
by casts that are likely to trigger undefined behavior.
The solution is to start a cleanup process focused on fixing and
reducing the amount of strict-aliasing related warnings produced
by GCC and others compilers. A good deal of noise reduction can
be achieved by just removing useless casts that are product of
historical cruft and are likely to trigger undefined behavior if
dereferenced.
client/mysql.cc:
Remove now-unnecessary casts.
Break up large strings.
client/mysql_upgrade.c:
Remove now-unnecessary casts.
client/mysqladmin.cc:
Remove now-unnecessary casts.
Break up large strings.
client/mysqlbinlog.cc:
Remove now-unnecessary casts.
client/mysqlcheck.c:
Remove now-unnecessary casts.
client/mysqldump.c:
Remove now-unnecessary casts.
client/mysqlimport.c:
Remove now-unnecessary casts.
client/mysqlshow.c:
Remove now-unnecessary casts.
client/mysqlslap.c:
Remove now-unnecessary casts.
client/mysqltest.cc:
Remove now-unnecessary casts.
extra/comp_err.c:
Remove now-unnecessary casts.
extra/my_print_defaults.c:
Remove now-unnecessary casts.
Break up large strings.
extra/mysql_waitpid.c:
Remove now-unnecessary casts.
extra/perror.c:
Remove now-unnecessary casts.
extra/resolve_stack_dump.c:
Remove now-unnecessary casts.
extra/resolveip.c:
Remove now-unnecessary casts.
include/my_getopt.h:
Use a void pointer type as the opaque type to avoid problems with type
incompatibility -- GCC issues warnings when the type name is not type
compatible with a operand. As a side bonus, a explicit cast won't be
necessary anymore.
include/sslopt-longopts.h:
Remove now-unnecessary casts.
Break up large strings.
mysys/my_getopt.c:
Update opaque type and introduce a type definition for the
argument to my_getopt_register_get_addr.
server-tools/instance-manager/options.cc:
Remove now-unnecessary casts.
sql/mysqld.cc:
Remove now-unnecessary casts.
Break up large strings.
Update mysql_getopt_value prototype (the old prototype
was different from the definition anyway).
sql/sql_plugin.cc:
The type of a pointer to a function must be compatible with the
pointed-to function type, otherwise the behavior is undefined.
sql/table.cc:
The variable buf pointer to pointer to pointer to constant char
could improperly alias a incompatible type in call to fix_type_
pointers. Since this was actually dead code, it is simply removed.
sql/unireg.cc:
Remove call to get_form_pos. The code creates a new FRM file which
is always truncated and writes the form position as 0. Hence, no
need to retrieve it, we now for sure it is 0.
storage/archive/archive_reader.c:
Remove now-unnecessary casts.
storage/myisam/ft_nlq_search.c:
Read weight directly from the buffer.
storage/myisam/fulltext.h:
Add explanation about the type duality of a key buffer.
Add accessor macro to retrieve a FT float value.
storage/myisam/mi_test1.c:
Remove now-unnecessary casts.
storage/myisam/myisam_ftdump.c:
Read weight directly from the buffer.
storage/myisam/myisamchk.c:
Remove now-unnecessary casts.
storage/myisam/myisamlog.c:
A pointer to char was used to alias a pointer to pointer to
unsigned char, thus violating strict aliasing rules.
storage/myisam/myisampack.c:
Remove now-unnecessary casts.
strings/decimal.c:
Remove aliasing violation, printing the value is enough for
debugging purposes.
tests/mysql_client_test.c:
Remove now-unnecessary casts.
When mysqlbinlog was given the --database=X flag, it always printed
'ROLLBACK TO', but the corresponding 'SAVEPOINT' statement was not
printed. The replicated filter(replicated-do/ignore-db) and binlog
filter (binlog-do/ignore-db) has the same problem. They are solved
in this patch together.
After this patch, We always check whether the query is 'SAVEPOINT'
statement or not. Because this is a literal check, 'SAVEPOINT' and
'ROLLBACK TO' statements are also binlogged in uppercase with no
any comments.
The binlog before this patch can be handled correctly except one case
that any comments are in front of the keywords. for example:
/* bla bla */ SAVEPOINT a;
/* bla bla */ ROLLBACK TO a;
I found three issues during the analysis:
1. Memory leak caused by temp_buf not being freed;
2. Memory leak caused when handling argv;
3. Conditional jump that depended on unitialized values.
Issue #1
--------
DESCRIPTION: when mysqlbinlog is reading from a remote location
the event temp_buf references the incoming stream (in NET
object), which is not freed by mysqlbinlog explicitly. On the
other hand, when it is reading local binary log, it points to a
temporary buffer that needs to be explicitly freed. For both
cases, the temp_buf was not freed by mysqlbinlog, instead was
set to 0. This clearly disregards the free required in the
second case, thence creating a memory leak.
FIX: we make temp_buf to be conditionally freed depending on
the value of remote_opt. Found out that similar fix is already
in most recent codebases.
Issue #2
--------
DESCRIPTION: load_defaults is called by parse_args, and it
reads default options from configuration files and put them
BEFORE the arguments that are already in argc and argv. This is
done resorting to MEM_ROOT. However, parse_args calls
handle_options immediately after which changes argv. Later when
freeing the defaults, pointers to MEM_ROOT won't match, causing
the memory not to be freed:
void free_defaults(char **argv)
{
MEM_ROOT ptr
memcpy_fixed((char*) &ptr,(char *) argv - sizeof(ptr), sizeof(ptr));
free_root(&ptr,MYF(0));
}
FIX: we remove load_defaults from parse_args and call it
before. Then we save argv with defaults in defaults_argv BEFORE
calling parse_args (which inside can then call handle_options
at will). Actually, found out that this is in fact kind of a
backport for BUG#38468 into 5.1, so I merged in the test case
as well and added error check for load_defaults call.
Fix based on:
revid:zhenxing.he@sun.com-20091002081840-uv26f0flw4uvo33y
Issue #3
--------
DESCRIPTION: the structure st_print_event_info constructor
would not initialize the sql_mode member, although it did for
sql_mode_inited (set to false). This would later raise the
warning in valgrind when printing the sql_mode in the event
header, as this print out is protected by a check against
sql_mode_inited and sql_mode variables. Given that sql_mode was
not initialized valgrind would output the warning.
FIX: we add initialization of sql_mode to the
st_print_event_info constructor.
client/mysqlbinlog.cc:
- Conditionally free ev->temp_buf.
- save defaults_argv before handle_options is called.
mysql-test/t/mysqlbinlog.test:
Added test case from BUG#38468.
sql/log_event.cc:
Added initialization of sql_mode for st_print_event_info.
Several items said to be deprecated in the 4.1 manual
have never been removed. This worklog adds deprecation
warnings when these items are used, and warns the user
that the items will be removed in MySQL 5.6.
A couple of previously deprecation decision have been
reversed (see single file comments)
client/client_priv.h:
Macro similar to the one in the server (mysql_priv.h)
for printing a deprecation warning message
client/mysql.cc:
no-auto-rehash will not be deprecated
skip-line-numbers will not be deprecated
skip-column-names will not be deprecated
no-pager is deprecated
set-variable is deprecated
no-named-commands is deprecated
client/mysqladmin.cc:
set-variable is deprecated
client/mysqlbinlog.cc:
position is deprecated
client/mysqldump.c:
first-slave is deprecated
no-set-names is deprecated
set-variable is deprecated
mysql-test/r/mysqlbinlog.result:
Adding the [Warning] to the test case, just to show that the
deprecation works.
The test case will be changed in Celosia to use --start-position.
mysys/my_getopt.c:
set-variable (include -O) is deprecated
scripts/mysqld_multi.sh:
Warning for mysqld_multi
sql/mysqld.cc:
default-collation is deprecated
log-bin-trust-routine-creators is deprecated
set-variable is deprecated
default-character-set is deprecated
safe-show-database is deprecated
sql/share/errmsg.txt:
Added version number for sql_log_update deprecation message.
The reason for the bug is that mysqtest as well as other client tools
running in test suite (mysqlbinlog, mysqldump) will first try to connect
whatever database has created shared memory with default base name
"MySQL" and use this. (Same effect could be seen on Unix if mtr would
not care to calculate "port" and "socket" parameter).
The fix ensures that all client tools and running in mtr use unique
per-database shared memory base parameters, so there is no possibility
to clash with already installed one. We use socket name for shared memory
base (it's known to be unique). This shared-memory-base is written to the
MTR config file to the [client] and [mysqld] sections. Fix made also made
sure all client tools understand and correctly handle --shared-memory-base.
Prior to this patch it was not the case for mysqltest, mysqlbinlog and
mysql_client_test.
All new connections done from mtr scripts via connect() will by default
set shared-memory-base. And finally, there is a possibility to force
shared memory or pipe connection and overwrite shared memory/pipe base name
from within mtr scripts via optional PIPE or SHM modifier. This functionality
was manually backported from 6.0
(original patch http://lists.mysql.com/commits/74749)
- Review fixes
client/Makefile.am:
- Make it build on Linux
client/mysqlbinlog.cc:
- Coding style fixes
- Better/more comments
- Use client/sql_string.*, not server's sql/sql_string.*.
- Don't declare a dummy TABLE_LIST structure in the client.
client/sql_string.h:
- Use client/sql_string.*, not server's sql/sql_string.*.
sql/log_event.cc:
= Fix coding style
= Introduce Log_event::event_owns_temp_buf which tells whether Log_event::temp_buf is 'owned' by the Log_event object and should be my_free'd on return.
This is needed because rewrite_db() needs to dispose of the buffer, and
- when mysqlbinlog is reading directly from binlog file, the buffer
should be freed
- when mysqlbinlog is reading from a server, the buffer is a part of network
buffer and shouldn't be freed.
sql/log_event.h:
Introduce Log_event::event_owns_temp_buf which tells whether Log_event::temp_buf is 'owned' by the Log_event object and should be my_free'd on return.
This is needed because rewrite_db() needs to dispose of the buffer, and
- when mysqlbinlog is reading directly from binlog file, the buffer
should be freed
- when mysqlbinlog is reading from a server, the buffer is a part of network
buffer and shouldn't be freed.
sql/mysqld.cc:
- Better/more comments
sql/rpl_filter.cc:
- #ifdef-out Rpl_filter::tables_ok from the client. This allows not
to define dummy TABLE_LIST on the client
sql/rpl_filter.h:
- #ifdef-out Rpl_filter::tables_ok from the client. This allows not
to define dummy TABLE_LIST on the client
sql/sql_string.cc:
- Use client/sql_string.*, not server's sql/sql_string.*.
sql/sql_string.h:
- Use client/sql_string.*, not server's sql/sql_string.*.
There are some problems about help text:
- It is stated that "auto" is the default twice. It need be stated only once.
- It is stated that --base64-output is short for --base64-output=always. But that sounds
like the default is "always", not "auto".
Make the help text clear as following:
Determine when the output statements should be base64-encoded BINLOG
statements: 'never' disables it and works only for binlogs without
row-based events; 'auto' prints base64 only when necessary (i.e.,
for row-based events and format description events); 'always' prints
base64 whenever possible. 'always' is for debugging only and should
not be used in a production system. If this argument is not given,
the default is 'auto'; if it is given with no argument, 'always' is used.
Add Rpl_filter to mysqlbinlog.cc
Note. Though within MWL#36 we are going to use only two Rpl_filter's
methods (add_db_rewrite and get_rewrite_db), we look forward for
MWL#40 where Rpl_filter is likely to be used to its more extent.
Note. Within MWL#36 we will not use Rpl_filter for supporting --database
option: this option allows to specify only one database what
doesn't correlate with Rpl_filter::add_do_db() (using this method
will either appear "artificial" or require changing --database
semantics). To be discussed within MWL#40.
To add Rpl_filter we need:
1. include sql_string.h
There are two instances of sql_string.* files - in sql and in client
directories. We need to use the ones from the sql dir.
2. include sql_list.h
This requires to define a client version of sql_alloc() function.
3. include rpl_filter.h
This requires a definition of system_charset_info variable.
Besides, Rpl_filter::tables_ok() refers to a TABLE_LIST structure which
encounts deep non-client dependencies and can't be used here as is. On
the other hand, tables_ok() make use only few TABLE_LIST's members and
none of them depends on specific server context. This allows to redefine
TABLE_LIST in a client context so that tables_ok() becomes admissible
(surely it's a kind of hack but (at least currently) it's better than
#ifndef'ing this method in Rpl_filter definition).
Also add Rpl_filter::rewrite_db_is_empty() method. This is needed to be
able to check that --rewrite-db is not used jointly with --base64-output=
always (this is not supported - at least currently).
The 'BEGIN/COMMIT/ROLLBACK' log event could be filtered out if the
database is not selected by --database option of mysqlbinlog command.
This can result in problem if there are some statements in the
transaction are not filtered out.
To fix the problem, mysqlbinlog will output 'BEGIN/ROLLBACK/COMMIT'
in regardless of the database filtering rules.
client/mysqlbinlog.cc:
Skip the database check for BEGIN/COMMIT/ROLLBACK log events.
mysql-test/r/mysqlbinlog.result:
Test result for bug#46998
mysql-test/suite/binlog/t/binlog_row_mysqlbinlog_db_filter.test:
The test case is updated duo to the patch of bug#46998
mysql-test/t/mysqlbinlog.test:
Added test to verify if the 'BEGIN', 'COMMIT' and 'ROLLBACK' are output
in regardless of database filtering
The 'BEGIN/COMMIT/ROLLBACK' log event could be filtered out if the
database is not selected by --database option of mysqlbinlog command.
This can result in problem if there are some statements in the
transaction are not filtered out.
To fix the problem, mysqlbinlog will output 'BEGIN/ROLLBACK/COMMIT'
in regardless of the database filtering rules.
client/mysqlbinlog.cc:
Skip the database check for BEGIN/COMMIT/ROLLBACK log events.
mysql-test/r/mysqlbinlog.result:
Test result for bug#46998
mysql-test/t/mysqlbinlog.test:
Added test to verify if the 'BEGIN', 'COMMIT' and 'ROLLBACK' are output
in regardless of database filtering
mysqlbinlog --database parameter was being ignored when processing
row events. As such no event filtering would take place.
This patch addresses this by deploying a call to shall_skip_database
when table_map_events are handled (as these contain also the name of
the database). All other rows events referencing the table id for the
filtered map event, will also be skipped.
client/mysqlbinlog.cc:
Added shall_skip_database call to the part of the code that handles
Table_map_log_events. It inspects the database name and decides whether
to filter the event or not. Furthermore, if table map event is filtered
next events referencing the table id in the table map event, will also
be filtered.
mysql-test/suite/binlog/t/binlog_row_mysqlbinlog_db_filter.test:
Test case that checks if row events are actually filtered out.
sql/log_event.h:
Added a map for holding the currently ignored table map events.
Table map events are inserted when they shall be skipped and removed
once the last row event in the statement is processed.
The --hexdump option crashed mysqlbinlog when used together
with the --read-from-remote-server option due to use of
uninitialized memory.
Since Log_event::print_header() relies on temp_buf to be
initialized when the --hexdump option is present,
dump_remote_log_entries() was fixed to setup temp_buf to point
to the start of a binlog event as done in
dump_local_log_entries().
The root cause of this bug is identical to the one for
bug #17654. The latter was fixed in 5.1 and up, so this
patch is backport of the patches for bug #17654 to 5.0.
Only 5.0 needs a changelog entry.
client/mysqlbinlog.cc:
Fixed dump_remote_log_entries() so that temp_buf is initialized
as it may be used later by Log_event::print_header() if the
--hexdump option is present.
mysql-test/r/mysqlbinlog.result:
Added a test case for bug #41943.
mysql-test/t/mysqlbinlog.test:
Added a test case for bug #41943.
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
- Implementing --base64-format=decode-rows, to display
SQL-alike decoded row events without their BINLOG statements.
- Adding --base64-format=decode-rows into tests when
calling mysqlbinlog to avoid non-deterministic results
- Removing resetting of last_table_id in "RESET MASTER",
which appeared to be dangerous.
Implementing -v command line parameter to mysqlbinlog
to decode and print row events.
mysql-test/include/mysqlbinlog_row_engine.inc
mysql-test/r/mysqlbinlog_row.result
mysql-test/r/mysqlbinlog_row_big.result
mysql-test/r/mysqlbinlog_row_innodb.result
mysql-test/r/mysqlbinlog_row_myisam.result
mysql-test/r/mysqlbinlog_row_trans.result
mysql-test/t/mysqlbinlog_row.test
mysql-test/t/mysqlbinlog_row_big.test
mysql-test/t/mysqlbinlog_row_innodb.test
mysql-test/t/mysqlbinlog_row_myisam.test
mysql-test/t/mysqlbinlog_row_trans.test
Adding tests
client/Makefile.am
Adding new files to symlink
client/mysqlbinlog.cc
Adding -v option
sql/log_event.cc
Impelentations of the new methods
sql/log_event.h
Declaration of the new methods and member
sql/mysql_priv.h
Adding new function prototype
sql/rpl_tblmap.cc
Adding pre-processor conditions
sql/rpl_tblmap.h
Adding pre-processor conditions
sql/rpl_utility.h
Adding pre-processor conditions
sql/sql_base.cc
Adding reset_table_id_sequence() function.
sql/sql_repl.cc
Resetting table_id on "RESET MASTER"
.bzrignore
Ignoring new symlinked files
into zippy.cornsilk.net:/home/cmiller/work/mysql/bug26294/my51-bug26294
client/mysqlbinlog.cc:
Auto merged
include/my_time.h:
Auto merged
sql-common/my_time.c:
Auto merged
sql/init.cc:
Auto merged
sql/tztime.cc:
Auto merged
When linking with some external programs, "multiple definition
of `init_time'"
Rename init_time() to my_init_time() to avoid collision with other
libraries (particularly libmng).
client/mysqlbinlog.cc:
Rename init_time() to my_init_time().
include/my_time.h:
Rename init_time() to my_init_time().
sql-common/my_time.c:
Rename init_time() to my_init_time().
sql/init.cc:
Rename init_time() to my_init_time().
sql/tztime.cc:
Rename init_time() to my_init_time().
Problem: mysqlbinlog does not free memory if an error happens.
Fix: binlog-processing functions do not call exit() anymore. Instead, they
print an error and return an error code. Error codes are propagated all
the way back to main, and all allocated memory is freed on the way.
client/mysqlbinlog.cc:
- New error handling policy: functions processing binlogs don't just
exit() anymore. Instead, they print a message and return an error
status.
- New policy for the global `mysql' and `glob_description_event': these
are not passed as parameters anymore. The global pointer is used
instead.
- More error situations are detected and reported.
- Better error messages: the program never terminates with exit status 1
without explanation any more. Fixed spelling errors. Use consistent
format of messages (a single line beginning with "ERROR: " or
"WARNING: " and ending with "." is printed to stderr.)
- New memory handling: memory is always freed on program termination.
- Better comments: more functions are explained, doxygen is used, and
more precise formulations in some existing comments.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
Result file updated since output format of mysqlbinlog changed while the
test was disabled.
mysql-test/suite/binlog/t/binlog_killed.test:
Mysqlbinlog now works as described when the binlog is open. Hence, the
--force-if-open flag must be passed
mysql-test/suite/binlog/t/binlog_killed_simulate.test:
Mysqlbinlog now works as described when the binlog is open. Hence, the
--force-if-open flag must be passed
mysql-test/suite/binlog/t/disabled.def:
Now that mysqlbinlog cleans up after itself on abnormal termination, we
can enable this test again.
