CRASHES ON EVERY START ATTEMPT
Description:
------------
push_warning_printf function is used to print the warning message
to the client. So this function should not invoke while recovering
the server. Moreover current_thd is NULL while starting the server.
Solution:
---------
- Avoiding the warning to be printed while recovery.
This patch already pushed in mysql-5.6.
Problem was that repair() did lock and unlock tables, which leaved already locked tables in wrong state
include/my_check_opt.h:
Added option T_NO_LOCKS to disable locking during repair()
Fixed duplicated bit T_NO_CREATE_RENAME_LSN
mysql-test/suite/rpl/r/myisam_external_lock.result:
Test case for MDEV-6871
mysql-test/suite/rpl/t/myisam_external_lock-slave.opt:
Test case for MDEV-6871
mysql-test/suite/rpl/t/myisam_external_lock.test:
Test case for MDEV-6871
storage/maria/ha_maria.cc:
Don't lock tables during enable_indexes()
Removed some calls to current_thd
storage/myisam/ha_myisam.cc:
Don't lock tables during enable_indexes()
Removed some calls to current_thd
modified:
storage/connect/rcmsg.c
- Avoid Memory copying when reading an ODBC table when the entire table
is already in the result set.
modified:
storage/connect/odbconn.cpp
storage/connect/odbconn.h
storage/connect/tabodbc.cpp
storage/connect/tabodbc.h
On PPC64 high-loaded server may crash due to assertion failure in InnoDB
rwlocks code.
This happened because load order between "recursive" and "writer_thread"
wasn't properly enforced.
modified:
storage/connect/value.cpp
- These files were commited even not modified (?)
modified:
storage/connect/ha_connect.cc
storage/connect/odbconn.cpp
storage/connect/odbconn.h
storage/connect/tabodbc.cpp
after Operating system error number 36 in a file operation.
Analysis: os_file_get_status did not handle error ENAMETOOLONG
correctly.
Fix: Add correct handling for error ENAMETOOLONG. Note that on InnoDB
case the error is not passed all the way up to server. That would
be bigger rewamp.
Use traditional statistics estimation by default (innodb-stats-traditional=true).
There could be performance regression for customers if there is a lot of
open table operations.
modified:
storage/connect/odbconn.cpp
storage/connect/odbconn.h
storage/connect/tabodbc.cpp
storage/connect/tabodbc.h
- Moving the calls to VerifyConnect and GetConnectInfo into the try block
in ODBConn::Open (potential crash in case of throw)
modified:
storage/connect/odbconn.cpp
- Handling all ODBC data date types (91, 92, 93)
modified:
storage/connect/ha_connect.cc
storage/connect/odbconn.cpp
- Not assuming string results from ODBC catalog functions are zero terminated
modified:
storage/connect/odbconn.cpp
innodb_stats_sample_pages
Analysis: If you set the number of analyzed pages
to very low number compared to actual pages on
that table/index it randomly pics those pages
(default 8 pages), this leads to fact that query
after analyze table returns different results. If
the index tree is small, smaller than 10 *
n_sample_pages + total_external_size, then the
estimate is ok. For bigger index trees it is
common that we do not see any borders between
key values in the few pages we pick. But still
there may be n_sample_pages different key values,
or even more. And it just tries to
approximate to n_sample_pages (8).
Fix: (1) Introduced new dynamic configuration variable
innodb_stats_sample_traditional that retains
the current design. Default false.
(2) If traditional sample is not used we use
n_sample_pages = max(min(srv_stats_sample_pages,
index->stat_index_size),
log2(index->stat_index_size)*
srv_stats_sample_pages);
(3) Introduced new dynamic configuration variable
stat_modified_counter (default = 0) if set
sets lower bound for row updates when statistics is re-estimated.
If user has provided upper bound for how many rows needs to be updated
before we calculate new statistics we use minimum of provided value
and 1/16 of table every 16th round. If no upper bound is provided
(srv_stats_modified_counter = 0, default) then calculate new statistics
if 1 / 16 of table has been modified
since the last time a statistics batch was run.
We calculate statistics at most every 16th round, since we may have
a counter table which is very small and updated very often.
@param t table
@return true if the table has changed too much and stats need to be
recalculated
*/
#define DICT_TABLE_CHANGED_TOO_MUCH(t) \
((ib_int64_t) (t)->stat_modified_counter > (srv_stats_modified_counter ? \
ut_min(srv_stats_modified_counter, (16 + (t)->stat_n_rows / 16)) : \
16 + (t)->stat_n_rows / 16))
The bug was that full memory barrier was missing in the code that ensures that
a waiter on an InnoDB mutex will not go to sleep unless it is guaranteed to be
woken up again by another thread currently holding the mutex. This made
possible a race where a thread could get stuck waiting for a mutex that is in
fact no longer locked. If that thread was also holding other critical locks,
this could stall the entire server. There is an error monitor thread than can
break the stall, it runs about once per second. But if the error monitor
thread itself got stuck or was not running, then the entire server could hang
infinitely.
This was introduced on i386/amd64 platforms in 5.5.40 and 10.0.13 by an
incorrect patch that tried to fix the similar problem for PowerPC.
This commit reverts the incorrect PowerPC patch, and instead implements a fix
for PowerPC that does not change i386/amd64 behaviour, making PowerPC work
similarly to i386/amd64.
* use the same HAVE_C/CXX_ variables for compiler flag tests as the rest of
the server and tokudb - to use cached results
* plugin's name should be "mroonga" not "ha_mroonga"
* don't use set_property(TARGET plugin_name ...), it aborts cmake when a plugin
id disabled, because the target doesn't exists in that case
result: mroonga can now be disabled from cmake command line
ALTER TABLE: don't fill default values per row, do it once.
And do it in two places - for copy_data_between_tables() and for online ALTER.
Also, run function_defaults test both for MyISAM and for InnoDB.
They are still experimental and should not be used in production.
added:
storage/connect/encas.h
storage/connect/english.msg
storage/connect/enids.h
storage/connect/frcas.h
storage/connect/french.msg
storage/connect/frids.h
storage/connect/frmsg.h
modified:
storage/connect/connect.cc
storage/connect/engmsg.h
storage/connect/filamdbf.cpp
storage/connect/global.h
storage/connect/ha_connect.cc
storage/connect/msgid.h
storage/connect/plgdbutl.cpp
storage/connect/plugutil.c
storage/connect/rcmsg.c
storage/connect/resource.h
storage/connect/tabfmt.h
- Fix global variable not being properly initialized (MDEV-6690, MDEV-7094)
modified:
storage/connect/ha_connect.cc
storage/connect/plugutil.c
storage/connect/user_connect.cc
storage/connect/xindex.cpp
- Implement Rewind for ODBC tables (MDEV-7097)
modified:
storage/connect/odbconn.cpp
storage/connect/odbconn.h
storage/connect/tabmysql.cpp
storage/connect/tabodbc.cpp
- Reset N when reopening MYSQL tables
modified:
storage/connect/tabmysql.cpp
buildbot on work-amd64-valgrind
Fixed issue by finding out first the current used priority
for both treads and using that seeing did we really change
the priority or not.
setting of innodb_io_capacity_max
(a) Changed the behaviour so that if you set innodb_io_capacity to a
value > innodb_io_capacity_max that the value is accepted AND
that innodb_io_capacity_max = innodb_io_capacity * 2.
(b) If someone wants to reduce innodb_io_capacity_max and
reduce it below innodb_io_capacity then innodb_io_capacity
should be reduced to the same level as innodb_io_capacity_max.
In both cases give a warning to user.
Analysis: InnoDB error monitor is responsible to call every second
sync_arr_wake_threads_if_sema_free() to wake up possible hanging
threads if they are missed in mutex_signal_object. This is not
possible if error monitor itself is on mutex/semaphore wait. We
should avoid all unnecessary mutex/semaphore waits on error monitor.
Currently error monitor calls function buf_flush_stat_update()
that calls log_get_lsn() function and there we will try to get
log_sys mutex. Better, solution for error monitor is that in
buf_flush_stat_update() we will try to get lsn with
mutex_enter_nowait() and if we did not get mutex do not update
the stats.
Fix: Use log_get_lsn_nowait() function on buf_flush_stat_update()
function. If returned lsn is 0, we do not update flush stats.
log_get_lsn_nowait() will use mutex_enter_nowait() and if
we get mutex we return a correct lsn if not we return 0.
on work-amd64-valgrind.
Fixed issue by finding out first the current used priority
for both treads and using that seeing did we really change
the priority or not.
COMMUNICATION PACKETS; FEDERATED TABLE
Description:- Execution of FLUSH TABLES on a federated
table which has been idle for wait_timeout (on the remote
server) + tcp_keepalive_time, fails with an error,
"ERROR 1160 (08S01): Got an error writing communication
packets."
Analysis:- During FLUSH TABLE execution the federated
table is closed which will inturn close the federated
connection. While closing the connection, federated server
tries to communincate with the remote server. Since the
connection was idle for wait_timeout(on the remote server)+
tcp_keepalive_time, the socket gets closed. So this
communication fails because of broken pipe and the error is
thrown. But federated connections are expected to reconnect
silently. And also it cannot reconnect because the
"auto_reconnect" variable is set to 0 in "mysql_close()".
Fix:- Before closing the federated connection, in
"ha_federated_close()", a check is added which will verify
wheather the connection is alive or not. If the connection
is not alive, then "mysql->net.error" is set to 2 which
will indicate that the connetion is broken. Also the
setting of "auto_reconnect" variable to 0 is delayed and is
done after "COM_QUIT" command.
NOTE:- For reproducing this issue, "tcp_keepalive_time" has
to be set to a smaller value. This value is set in the
"/proc/sys/net/ipv4/tcp_keepalive_time" file in Unix
systems. So we need root permission for changing it, which
can't be done through mtr test. So submitting the patch
without mtr test.
default ending. Also takes care of files having mixed line endings.
This is done by never using text mode for streams and handle the line
endings in reading and writing. (MDEV-7030)
modified:
storage/connect/filamtxt.cpp
Analysis: InnoDB error monitor is responsible to call every second
sync_arr_wake_threads_if_sema_free() to wake up possible hanging
threads if they are missed in mutex_signal_object. This is not
possible if error monitor itself is on mutex/semaphore wait. We
should avoid all unnecessary mutex/semaphore waits on error monitor.
Currently error monitor calls function buf_flush_stat_update()
that calls log_get_lsn() function and there we will try to get
log_sys mutex. Better, solution for error monitor is that in
buf_flush_stat_update() we will try to get lsn with
mutex_enter_nowait() and if we did not get mutex do not update
the stats.
Fix: Use log_get_lsn_nowait() function on buf_flush_stat_update()
function. If returned lsn is 0, we do not update flush stats.
log_get_lsn_nowait() will use mutex_enter_nowait() and if
we get mutex we return a correct lsn if not we return 0.
modified:
storage/connect/filamap.cpp
- ftell error: indicate in the error msg that is can be due to wrong ENDING value.
filamtxt.cpp (MDEV-7030)
modified:
storage/connect/filamtxt.cpp
- Change STRING according to Alexander Barkov remarks.
Suppress the wrong Strz function. The unconditional function strz is no more
used for s->db and s->table_name because they are zero terminated.
modified:
storage/connect/ha_connect.cc
storage/connect/xobject.cpp
- Change version number
modified:
storage/connect/filamap.cpp
- Change PATH_MAX to FN_REFLEN (MDEV-7036)
modified:
storage/connect/os.h
storage/connect/tabmul.cpp
- Fix bug by adding a void argument for OP_NOT in Makefilter.
modified:
storage/connect/filter.cpp
- Begin implementing XMSG style
Two new system variables are defined:
msg_lang ENUM session
errmsg_dir_path STR global readonly
This is a work in progress.
modified:
storage/connect/ha_connect.cc
storage/connect/plgdbutl.cpp
storage/connect/plugutil.c
storage/connect/rcmsg.c
storage/connect/resource.h
Description:
Using correct length when moving to next field in cmp_ref. The store
length already includes the length bytes of blobs, which is already considered
earlier for blob types.
Approved by Mattias, Jimmy [rb-7088]
modified:
storage/connect/connect.cc
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/mycat.cc
storage/connect/plgdbsem.h
- Get good message when calling ColDB
modified:
storage/connect/connect.cc
- Fix buffer preparation for BIN files
modified:
storage/connect/filamfix.cpp
fix error while updating (force fseek)
modified:
storage/connect/filamfix.cpp
fix error of XCOL column when filtered (typo)
modified:
storage/connect/tabdos.cpp
storage/connect/tabxcl.cpp
fix error when indexing on special column
modified:
storage/connect/tabdos.cpp
The debug configuration parameter innodb_optimistic_insert_debug
which was introduced for testing corner cases in B-tree handling
had a bug in it. The value 1 would trigger an infinite sequence
of page splits.
Fix: When the value 1 is specified, disable this debug feature.
Approved by Yasufumi Kinoshita
The debug configuration parameter innodb_optimistic_insert_debug
which was introduced for testing corner cases in B-tree handling
had a bug in it. The value 1 would trigger an infinite sequence
of page splits.
Fix: When the value 1 is specified, disable this debug feature.
Approved by Yasufumi Kinoshita
Problem:
In the function dict_foreign_remove_from_cache(), the rb tree was updated
without actually verifying whether the given foreign key object is there in the
rb tree or not. There can be an existing foreign key object with the same id
in the rb tree, which must not be removed. Such a scenario comes when an
attempt is made to add a foreign key object with a duplicate identifier.
Solution:
When the foreign key object is removed from the dictionary cache, ensure
that the foreign key object removed from the rbt is the correct one.
rb#7168 approved by Jimmy and Marko.
Problem:
In the function dict_foreign_remove_from_cache(), the rb tree was updated
without actually verifying whether the given foreign key object is there in the
rb tree or not. There can be an existing foreign key object with the same id
in the rb tree, which must not be removed. Such a scenario comes when an
attempt is made to add a foreign key object with a duplicate identifier.
Solution:
When the foreign key object is removed from the dictionary cache, ensure
that the foreign key object removed from the rbt is the correct one.
rb#7168 approved by Jimmy and Marko.
dict_set_corrupted(): Use the canonical way of searching for
less-than-equal (PAGE_CUR_LE) and then checking low_match.
The code that was introduced in MySQL 5.5.17 in
Bug#11830883 SUPPORT "CORRUPTED" BIT FOR INNODB TABLES AND INDEXES
could position the cursor on the page supremum, and then attempt
to overwrite non-existing 7th field of the 1-field supremum record.
Approved by Jimmy Yang
dict_set_corrupted(): Use the canonical way of searching for
less-than-equal (PAGE_CUR_LE) and then checking low_match.
The code that was introduced in MySQL 5.5.17 in
Bug#11830883 SUPPORT "CORRUPTED" BIT FOR INNODB TABLES AND INDEXES
could position the cursor on the page supremum, and then attempt
to overwrite non-existing 7th field of the 1-field supremum record.
Approved by Jimmy Yang
Merged Facebook commit ec1aac68c74f3c1e558d057c4c9fcfe6edbbea93
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6
In C++11, "" is not parsed as before. So "A""B" is not the same as "AB".
Instead, whitespace is required, like: "A" "B"
Merged Facebook commit dd2d11be7aaf3be270e740fb95cbc4eacb52f4d7
authored by Rongrong Zhong from https://github.com/facebook/mysql-5.6
This fixes MySQL Bug #68220 innodb_rows_updated is misleading on slave
http://bugs.mysql.com/bug.php?id=68220
Added innodb_system_rows_read/inserted/updated/deleted counters
that are the equivalent of innodb_rows_* but that only account for
changes made to system databases (mysql, information_schame and
preformance_schema). These counters will be used on slaves to
differentiated the updates made on system databases from those made on
user databases.
innodb_rows_* status counters are not updated when innodb_system_rows_*
are updated.
dd2d11be7a
Merged Facebook commit ecff018632c6db49bad73d9233c3cdc9f41430e9
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6
This change is to fix: http://bugs.mysql.com/62534
This makes innodb_max_dirty_pages_pct a double with min,default,max values
0.001, 75, 99.999.
This also makes innodb_max_dirty_pages_pct_lwm and adaptive_flushing_lwm
doubles, as these sysvars are inter-dependent.
Added more to the BUFFER POOL AND MEMORY section of SHOW INNODB STATUS:
Percent pages dirty: X.X
This is all n_dirty_pages / used_pages
Percent all pages dirty: X.X
This is all n_dirty_pages / all-pages
Max dirty pages percent: X.X
This is innodb_max_dirty_pages_pct
Also changed all of buf from 2 to 3 digits of precision (%.2f -> %.3f).
Merge Facebook commit f981a51a47519b0ba527917887f8adc6df9ae147
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6.
This just moves some structure definitions from inside a
single .cc file to a shared .h file, with a few tweaks to
allow these structures to be shared.
On its own, it should have no actual effect. This is needed later.
Merge Facebook commit 25295d003cb0c17aa8fb756523923c77250b3294
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6
This adds a pointer to the trx to each mtr.
This allows the trx to be accessed in parts of the code
where it was otherwise not available. This is needed later.
compressed pages
buf_flush_LRU() returns the number of pages processed. There are
two types of processing that can happen. A page can get evicted or
a page can get flushed. These two numbers are quite distinct and
should not be mixed.
was not reset causing rows to be lost.
modified:
storage/connect/connect.cc
storage/connect/ha_connect.cc
storage/connect/tabxcl.cpp
- Typo
modified:
storage/connect/connect.cc
storage/connect/ha_connect.cc
Merge Facebook commit 926a077b14b73c14094de7fc7aa913241b801b4d
authored by Inaam Rana from https://github.com/facebook/mysql-5.6.
This is fix for upstream bugs
http://bugs.mysql.com/bug.php?id=71988http://bugs.mysql.com/bug.php?id=70500
page_cleaner should work whether or not there is server activity.
Its iterations become a noop when there is no work to do but we
should not tie it to the server activity.
The page_cleaner thread does spurious background flushing
because of conditional sleep between iterations. The solution
is not to make sleep dependent on server activity etc.
Merged Facebooks commit 6e06bbfa315ffb97d713dd6e672d6054036ddc21
authored by Inaam Rana from https://github.com/facebook/mysql-5.6.
Fixes MySQL bug http://bugs.mysql.com/bug.php?id=72123
lock_timeout thread works in a tight loop waking up every second
and checking for lock_wait_timeout. In addition, when a mysql
thread is forced to wait on a lock, it signals the lock_timeout thread
as well. This call is not required. In a heavily contended workload
each thread going to wait will signal the lock_timeout thread making
it work all the time. As lock_timeout thread scans the array of
waiting threads under lock_sys::wait_mutex which is already very
hot in contneded loads, these extra scans can cause significanct
performance regression.
Also, in various codepaths lock_timeout thread is signalled where
actual intention was to signal the innodb monitor thread.
Merged Facebook commit bdab302a7e3c37da21a1bffe1550cdbe6c906695
by Inaam Rana from https://github.com/facebook/mysql-5.6.
In os_event_wait_time_low() the logic to calculate abs_time
for wait is broken. The bug has been present at least since
5.5. It gets acutely sensitized when sub-second wait intervals
are passed. It is particularly relevant to us because the
page_cleaner thread will mostly request sub-second wait
intervals. This can potentially lead to a near tight loop
behaviour of page_cleaner with much less sleep then what we'd
actually expect.
the beginning. Defining the STRING class and begining to use it (MYSQL)
2) Change the xtrace, use_tempfile and exact_info connect variables from
GLOBAL to SESSION. Remaining GLOBAL variables have been made readonly.
3) Take care of LEX_STRING variables. The .str should not be regarded as
allways being 0 terminated. This is handled by the Strz functions that
make sure to return 0 terminated strings.
Bug fix:
- When inserting in MYSQL table with special column(s) a query such as:
insert into t2 values(0,4,'new04'),(0,5,'new05');
failed saying: column id (the special column) not found in t2.
It is now accepted but must be counted in values (these 0 are ignored)
- ROWID was returning row numbers based 0. Now it is from base 1.
modified:
storage/connect/array.cpp
storage/connect/blkfil.cpp
storage/connect/colblk.cpp
storage/connect/connect.cc
storage/connect/filamap.cpp
storage/connect/filamdbf.cpp
storage/connect/filamfix.cpp
storage/connect/filamtxt.cpp
storage/connect/filamvct.cpp
storage/connect/filamzip.cpp
storage/connect/filamzip.h
storage/connect/filter.cpp
storage/connect/global.h
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/libdoc.cpp
storage/connect/mycat.cc
storage/connect/myconn.cpp
storage/connect/odbconn.cpp
storage/connect/plgdbutl.cpp
storage/connect/plugutil.c
storage/connect/reldef.cpp
storage/connect/tabcol.cpp
storage/connect/tabdos.cpp
storage/connect/tabfix.cpp
storage/connect/tabfmt.cpp
storage/connect/table.cpp
storage/connect/tabmul.cpp
storage/connect/tabmysql.cpp
storage/connect/tabmysql.h
storage/connect/taboccur.cpp
storage/connect/tabodbc.cpp
storage/connect/tabpivot.cpp
storage/connect/tabsys.cpp
storage/connect/tabtbl.cpp
storage/connect/tabutil.cpp
storage/connect/tabvct.cpp
storage/connect/tabwmi.cpp
storage/connect/tabwmi.h
storage/connect/tabxcl.cpp
storage/connect/tabxml.cpp
storage/connect/user_connect.cc
storage/connect/valblk.cpp
storage/connect/value.cpp
storage/connect/value.h
storage/connect/xindex.cpp
storage/connect/xobject.cpp
storage/connect/xobject.h
storage/connect/xtable.h
When UNIV_INTERN is missing in built-in XtraDB, this causes the
innodb_plugin to call the XtraDB version of the function instead
of its own (seen in --embedded-server test failure in Buildbot).
This in turn causes bad things to happen in case of difference
between XtranDB and innodb_plugin.
