Resolved all conflicts, bad merges and fixed a few minor bugs in the code.
Commented out the queries from multi_update, view, subselect_sj, func_str,
derived_view, view_grant that failed either with crashes in ps-protocol or
with wrong results.
The failures are clear indications of some bugs in the code and these bugs
are to be fixed.
Only wait for a single debug signal at a time as the signal state
is global. Also, do not activate the query cache debug sync points
if the thread has no associated THD session.
mysql-test/t/query_cache_debug.test:
Only wait for a single debug signal at a time as the signal state
is global.
sql/sql_cache.cc:
Do not execute the debug sync point if the thread has no associated
THD session. This scenario happens for federated threads.
The problem was that threads waiting on the query cache lock
are not easily seen due to the lack of a state indicating that
the thread is waiting on the said lock. This made it difficult
for users to quickly spot (for example, via SHOW PROCESSLIST)
a query cache contention problem.
The solution is to update the thread state when the query cache
lock needs to be acquired. Whenever the lock is to be acquired,
the thread state is updated to "Waiting for query cache lock"
and is reset once the lock is granted or the wait is interrupted.
The intention is to make query cache related hangs more evident.
To further investigate query cache related locking problems, one
may use PERFORMANCE_SCHEMA to track the overhead associated with
the locking bits and determine which particular lock is being a
contention point.
sql/sql_cache.cc:
Set and reset the thread state whenever a attempt to lock the
query cache is made.
Use DEBUG_SYNC instead of the now unnecessary wait_for_kill hack.
thd->in_sub_stmt
In a precursor patch for Bug#52044
(revid:bzr/kostja@stripped), a
number of reorganizations of code was made. In addition some
assertions were added to ensure the correct transactional state.
The reorganization had a small glitch so statements that was
active in the query cache was not followed by a
statement commit/rollback (this code was removed). A section
in the trans_commit_stmt/trans_rollback_stmt code is to
clear the thd->transaction.stmt list of affected storage
engines. When a new statement is initiated, an assert
introduced by the 523044 patch checks if this list is cleared.
When the query cache is accessed, this list may be populated,
and since it's not committed it will not be cleared.
This fix adds explicit statement commit or rollback for
statements that is contained in the query cache.
Fix some bugs where we stored values other than 0 or 1 in my_bool
Fixed some compiler warnings
client/mysql.cc:
Changed interrupted_query from my_bool to int, as we stored 2 in it.
client/mysqladmin.cc:
Changed return variable type to same type as function value type
client/mysqltest.cc:
Changed 'found' to int as we store other values than 0 or 1 into it
Changed type for parameter of set_reconnect() to match usage.
extra/libevent/evbuffer.c:
Added __attribute__((unused))
extra/libevent/event.c:
Added __attribute__((unused))
extra/libevent/signal.c:
Added __attribute__((unused))
sql/event_data_objects.h:
my_bool -> bool
sql/event_db_repository.cc:
my_bool -> bool
sql/event_db_repository.h:
my_bool -> bool
sql/event_parse_data.h:
my_bool -> bool
sql/events.cc:
my_bool -> bool
sql/events.h:
my_bool -> bool
sql/field.cc:
my_bool -> bool
sql/field.h:
my_bool -> bool
sql/hash_filo.h:
my_bool -> bool
sql/item.cc:
my_bool -> bool
sql/item.h:
my_bool -> bool
sql/item_cmpfunc.h:
my_bool -> bool
Changed result_for_null_param from my_bool to int as we stored -1 in it.
sql/item_func.cc:
my_bool -> bool
Modified udf wrapper functions so that the UDF functions would continue to use my_bool. (To keep compatibility with UDF:s)
sql/item_func.h:
my_bool -> bool
sql/item_subselect.h:
my_bool -> bool
sql/item_sum.cc:
Modified udf wrapper functions so that the UDF functions would continue to use my_bool. (To keep compatibility with UDF:s)
sql/parse_file.h:
my_bool -> bool
sql/rpl_mi.h:
my_bool -> bool
sql/sp_rcontext.h:
my_bool -> bool
sql/sql_analyse.h:
my_bool -> bool
sql/sql_base.cc:
Change some assignments so that we don't initialize bool variables with int's.
sql/sql_bitmap.h:
my_bool -> bool
sql/sql_cache.cc:
my_bool -> bool
sql/sql_cache.h:
my_bool -> bool
sql/sql_class.h:
my_bool -> bool
sql/sql_insert.cc:
Change some assignments so that we don't initialize bool variables with int's.
sql/sql_prepare.cc:
my_bool -> bool
sql/table.h:
my_bool -> bool
storage/maria/ma_check.c:
Removed duplicate assignment
strings/decimal.c:
Fixed wrong variable usage.
Don't do complex arithmetic on bool when simple works.
compression protocol.
The loss of connection was caused by a malformed packet
sent by the server in case when query cache was in use.
When storing data in the query cache, the query cache
memory allocation algorithm had a tendency to reduce
the amount of memory block necessary to store a result
set, up to finally storing the entire result set in a single
block. With a significant result set, this memory block
could turn out to be quite large - 30, 40 MB and on.
When such a result set was sent to the client, the entire
memory block was compressed and written to network as a
single network packet. However, the length of the
network packet is limited by 0xFFFFFF (16MB), since
the packet format only allows 3 bytes for packet length.
As a result, a malformed, overly large packet
with truncated length would be sent to the client
and break the client/server protocol.
The solution is, when sending result sets from the query
cache, to ensure that the data is chopped into
network packets of size <= 16MB, so that there
is no corruption of packet length. This solution,
however, has a shortcoming: since the result set
is still stored in the query cache as a single block,
at the time of sending, we've lost boundaries of individual
logical packets (one logical packet = one row of the result
set) and thus can end up sending a truncated logical
packet in a compressed network packet.
As a result, on the client we may require more memory than
max_allowed_packet to keep, both, the truncated
last logical packet, and the compressed next packet.
This never (or in practice never) happens without compression,
since without compression it's very unlikely that
a) a truncated logical packet would remain on the client
when it's time to read the next packet
b) a subsequent logical packet that is being read would be
so large that size-of-new-packet + size-of-old-packet-tail >
max_allowed_packet.
To remedy this issue, we send data in 1MB sized packets,
that's below the current client default of 16MB for
max_allowed_packet, but large enough to ensure there is no
unnecessary overhead from too many syscalls per result set.
sql/net_serv.cc:
net_realloc() modified: consider already used memory
when compare packet buffer length
sql/sql_cache.cc:
modified Query_cache::send_result_to_client: send result to client
in chunks limited by 1 megabyte.
- Changed to still use bcmp() in certain cases becasue
- Faster for short unaligneed strings than memcmp()
- Bettern when using valgrind
- Changed to use my_sprintf() instead of sprintf() to get higher portability for old systems
- Changed code to use MariaDB version of select->skip_record()
- Removed -%::SCCS/s.% from Makefile.am:s to remove automake warnings
to write into a closed socket
sql/protocol.cc:
Protocol::flush modified: set thd->main_da.can_overwrite_status= TRUE
before call to net_flush() in order to prevent crash on assert in case
of socket write failure, reset it to FALSE when net_flush() returned;
Protocol::send_fields modified: return from method with error if call to
my_net_write(), proto.write() or write_eof_packet() failed.
sql/sql_cache.cc:
Query_cache::send_result_to_client modified: call to
thd->main_da.disable_status() only if write to socket
was successful.
sql/sql_cursor.cc:
Materialized_cursor::fetch modified: leave method if call to
result->send_data() failed.
sql/sql_prepare.cc:
send_prep_stmt() modified: call to thd->main_da.disable_status()
only if thd->protocol_text.send_fields() completed successfully.
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
client/mysqldump.c:
Pass my_free directly as its signature is compatible with the
callback type -- which wasn't the case for free_table_ent.
Adding my_global.h first in all files using
NO_EMBEDDED_ACCESS_CHECKS.
Correcting a merge problem resulting from a changed definition
of check_some_access compared to the original patches.
This patch:
- Moves all definitions from the mysql_priv.h file into
header files for the component where the variable is
defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h
Fixed bug in Yassle to get correct error messages in case of errors
Provide better error messages in case of ssl connect failure
Updated out-of-date ssl certificates to fix failing mysql-test-system (certificates now active for 10 years)
Fixed bug in query_cache that could cause asserts and hangs in DEBUG builds.
Fixed bug where one connection did not see changes done by another connection.
configure.in:
Changed version number from RC to stable
extra/yassl/src/yassl_error.cpp:
Fixed bug in Yassle to get correct error messages in case of errors
- 'error' is an enum that hold more error numbers than the enum was defined for
include/violite.h:
Added error output string for sslaccept() and sslconnect() to get reason for connect failure
mysql-test/mysql-test-run.pl:
Write failed test cases if mysql-test-run fails because of too many errors
mysql-test/r/grant.result:
Update results to reflect new certificates
mysql-test/r/openssl_1.result:
Update results to reflect new certificates
mysql-test/std_data/cacert.pem:
Update ssl certificate
mysql-test/std_data/client-cert.pem:
Update ssl certificate
mysql-test/std_data/client-key.pem:
Update ssl certificate
mysql-test/std_data/server-cert.pem:
Update ssl certificate
mysql-test/std_data/server-key.pem:
Update ssl certificate
mysql-test/t/grant.test:
Update test to reflect new certificates
mysql-test/t/openssl_1.test:
Update test to reflect new certificates
mysql-test/t/query_cache_debug.test:
Remove 'big_test' as test is now fast
sql-common/client.c:
Give a better error message if ssl connect fails
sql/net_serv.cc:
Fixed compiler warnings
sql/slave.cc:
Give a better error message in logs if ssl connect fails
sql/sql_cache.cc:
debug_wait_for_kill() now removes the set watch variable after kill signal
This is needed as invalidate_table() may be called twice for one query.
Ensure that net->query_cache_query is reset after query. This fixes assert in
query_cache_end_of_result() if query_cache_query holds results from previous query.
Removed DBUG_ASSERT(0), as this code can be run by query_cache_debug.test
sql/sql_connect.cc:
Give a better error message if ssl connect fails
sql/sql_parse.cc:
Fixed bug where one connection did not see changes done by another connection.
For statements that changes tables, close_thread_tables() MUST be called before
sending OK as a table handler may not make the changes available for other connections
before unlock_tables().
vio/viossl.c:
Give a better error message if ssl connect fails
- Marked a couple of tests with --big
- Fixed xtradb/handler/ha_innodb.cc to call explain_filename()
storage/xtradb/handler/ha_innodb.cc:
Call explain_filename() to get proper names for partitioned tables
Bug#16565 mysqld --help --verbose does not order variablesBug#20413 sql_slave_skip_counter is not shown in show variables
Bug#20415 Output of mysqld --help --verbose is incomplete
Bug#25430 variable not found in SELECT @@global.ft_max_word_len;
Bug#32902 plugin variables don't know their names
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#34829 No default value for variable and setting default does not raise error
Bug#34834 ? Is accepted as a valid sql mode
Bug#34878 Few variables have default value according to documentation but error occurs
Bug#34883 ft_boolean_syntax cant be assigned from user variable to global var.
Bug#37187 `INFORMATION_SCHEMA`.`GLOBAL_VARIABLES`: inconsistent status
Bug#40988 log_output_basic.test succeeded though syntactically false.
Bug#41010 enum-style command-line options are not honoured (maria.maria-recover fails)
Bug#42103 Setting key_buffer_size to a negative value may lead to very large allocations
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#44797 plugins w/o command-line options have no disabling option in --help
Bug#46314 string system variables don't support expressions
Bug#46470 sys_vars.max_binlog_cache_size_basic_32 is broken
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#47212 Crash in DBUG_PRINT in mysqltest.cc when trying to print octal number
Bug#48758 mysqltest crashes on sys_vars.collation_server_basic in gcov builds
Bug#49417 some complaints about mysqld --help --verbose output
Bug#49540 DEFAULT value of binlog_format isn't the default value
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
Bug#49646 mysql --show-warnings crashes when server dies
CMakeLists.txt:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
client/mysql.cc:
don't crash with --show-warnings when mysqld dies
config/ac-macros/plugins.m4:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
include/my_getopt.h:
comments
include/my_pthread.h:
fix double #define
mysql-test/mysql-test-run.pl:
run sys_vars suite by default
properly recognize envirinment variables (e.g. MTR_MAX_SAVE_CORE) set to 0
escape gdb command line arguments
mysql-test/suite/sys_vars/r/rpl_init_slave_func.result:
init_slave+utf8 bug
mysql-test/suite/sys_vars/t/rpl_init_slave_func.test:
init_slave+utf8 bug
mysys/my_getopt.c:
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
mysys/typelib.c:
support for flagset
sql/ha_ndbcluster.cc:
backport from telco tree
sql/item_func.cc:
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
sql/sql_builtin.cc.in:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
sql/sql_plugin.cc:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#32902 plugin variables don't know their names
Bug#44797 plugins w/o command-line options have no disabling option in --help
sql/sys_vars.cc:
all server variables are defined here
storage/myisam/ft_parser.c:
remove unnecessary updates of param->quot
storage/myisam/ha_myisam.cc:
myisam_* variables belong here
strings/my_vsnprintf.c:
%o and %llx
unittest/mysys/my_vsnprintf-t.c:
%o and %llx tests
vio/viosocket.c:
bugfix: fix @@wait_timeout to work with socket timeouts (vs. alarm thread)
revno: 2617.22.4
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: mysql-6.0-runtime
timestamp: Mon 2009-01-26 15:19:14 -0200
message:
Move checks for OPTION_NOT_AUTOCOMMIT | OPTION_BEGIN to a separate
helper function.
sql/ha_ndbcluster.cc:
Use helper method to check transaction mode.
sql/log.cc:
Use helper method to check transaction mode.
sql/sql_cache.cc:
Use helper method to check transaction mode.
sql/sql_class.cc:
Use helper method to check transaction mode.
sql/sql_class.h:
Add helper method to check whether session is in a multi-statement
transaction.
sql/sql_parse.cc:
Use helper method to check transaction mode.
sql/transaction.cc:
Use helper method to check transaction mode.
----------------------------------------------------------
revno: 2630.4.26
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-prelocked_mode-to-push
timestamp: Fri 2008-06-06 23:19:04 +0400
message:
WL#3726: work on review comments.
Remove thd->locked_tables. Always store MYSQL_LOCK instances in
thd->lock.
Rename thd->prelocked_mode to thd->locked_tables_mode.
Use thd->locked_tables_mode to determine if we
are under LOCK TABLES. Update the code to not assume that
if thd->lock is set, LOCK TABLES mode is off.
Review comments.
sql/ha_ndbcluster_binlog.cc:
Don't unlock the lock under LOCK TABLES (safety).
sql/handler.cc:
There is no thd->locked_tables any more.
Update comments.
sql/lock.cc:
There is no thd->locked_tables any more.
sql/log.cc:
Rename thd->prelocked_mode to thd->locked_tables_mode.
sql/set_var.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
sql/sp_head.cc:
Rename thd->prelocked_mode to thd->locked_tables_mode.
sql/sql_base.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
Remove thd->locked_tables.
sql/sql_cache.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
sql/sql_class.cc:
Avoid code duplication.
Do not release the table locks prematurely if we're under LOCK TABLES.
Use thd->locked_tables_mode instead of thd->locked_tables.
sql/sql_class.h:
Remove thd->locked_tables.
Make prelocked mode a kind of LOCK TABLES mode.
Update comments.
sql/sql_cursor.cc:
Update comments.
sql/sql_insert.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
Rename thd->prelocked_mode to thd->locked_tables_mode.
sql/sql_load.cc:
Rename thd->prelocked_mode to thd->locked_tables_mode.
sql/sql_parse.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
Remove thd->locked_tables.
sql/sql_partition.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
sql/sql_rename.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
sql/sql_select.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
sql/sql_table.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
sql/sql_trigger.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
sql/sql_update.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
sql/sql_view.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
storage/myisam/ha_myisam.cc:
Use thd->locked_tables_mode to determine if we are under LOCK TABLES.
This patch introduce a limit on the time the query cache can
block with a lock on SELECTs.
Other operations which causes a change in the table
data will still be blocked.
sql/sql_cache.cc:
* Introduced a timeout value for the qc lock when entering send_result_to_client()
and store_query() methods.
sql/sql_cache.h:
* New signature for Query_cache::try_lock()
- Moved some code from innodb_plugin to xtradb, to ensure that all tests runs
- Did changes in pbxt and maria storage engines becasue of changes in thd->query
- Reverted wrong code in sql_table.cc for how ROW_FORMAT is used.
This is a re-commit of Monty's merge to eliminate an extra commit from
MySQL-5.1.42 that was accidentally included in the merge.
This is a merge of the MySQL 5.1.41 clone-off (clone-5.1.41-build). In
case there are any extra changes done before final MySQL 5.1.41
release, these will need to be merged later before MariaDB 5.1.41
release.
When the query cache is disabled, the server shouldn't attempt to take the
query cache mutex.
By using the command line option --query_cache_type=0, the user can disable
(backport from mysql-pe)
mysql-test/t/query_cache_disabled-master.opt:
* added test case for bug38551
mysql-test/t/query_cache_disabled.test:
* added test case for bug38551
sql/set_var.cc:
* Added before-trigger to verify that query_cache_type wasn't turned off or on during
runtime.
sql/set_var.h:
* Changed order on how the enumeration is processed. By first projecting the
character representation of the variable to a temporary integer we can have
one function instead of two to check if the value is valid.
sql/share/errmsg-utf8.txt:
* Added error message for query cache disabled state
sql/sql_cache.cc:
* If the query cache is disabled at start up, shorten the execution path and avoid
grabbing the query cache mutex each time the invalidate interface methods are called.
sql/sql_cache.h:
* Added new methods to set the query cache into a disabled state.
