- Removed not used variables
- Added __attribute__()
- Added static to some local functions
(gcc 5.4 gives a warning for external functions without an external definition)
fil_extend_space_to_desired_size(): Use a proper type cast when
computing start_offset for the posix_fallocate() call on 32-bit systems
(where sizeof(ulint) < sizeof(os_offset_t)). This could affect 32-bit
systems when extending files that are at least 4 MiB long.
This bug existed in MariaDB 10.0 before MDEV-11520. In MariaDB 10.1
it had been fixed in MDEV-11556.
a large memory buffer on Windows
fil_extend_space_to_desired_size(), os_file_set_size(): Use calloc()
for memory allocation, and handle failures. Properly check the return
status of posix_fallocate(), and pass the correct arguments to
posix_fallocate().
On Windows, instead of extending the file by at most 1 megabyte at a time,
write a zero-filled page at the end of the file.
According to the Microsoft blog post
https://blogs.msdn.microsoft.com/oldnewthing/20110922-00/?p=9573
this will physically extend the file by writing zero bytes.
(InnoDB never uses DeviceIoControl() to set the file sparse.)
I tested that the file extension works properly with a multi-file
system tablespace, both with --innodb-use-fallocate and
--skip-innodb-use-fallocate (the default):
./mtr \
--mysqld=--innodb-use-fallocate \
--mysqld=--innodb-autoextend-increment=1 \
--mysqld=--innodb-data-file-path='ibdata1:5M;ibdata2:5M:autoextend' \
--parallel=auto --force --retry=0 --suite=innodb &
ls -lsh mysql-test/var/*/mysqld.1/data/ibdata2
(several samples while running the test)
The function trx_purge_stop() was calling os_event_reset(purge_sys->event)
before calling rw_lock_x_lock(&purge_sys->latch). The os_event_set()
call in srv_purge_coordinator_suspend() is protected by that X-latch.
It would seem a good idea to consistently protect both os_event_set()
and os_event_reset() calls with a common mutex or rw-lock in those
cases where os_event_set() and os_event_reset() are used
like condition variables, tied to changes of shared state.
For each os_event_t, we try to document the mutex or rw-lock that is
being used. For some events, frequent calls to os_event_set() seem to
try to avoid hangs. Some events are never waited for infinitely, only
timed waits, and os_event_set() is used for early termination of these
waits.
os_aio_simulated_put_read_threads_to_sleep(): Define as a null macro
on other systems than Windows. TODO: remove this altogether and disable
innodb_use_native_aio on Windows.
os_aio_segment_wait_events[]: Initialize only if innodb_use_native_aio=0.
recv_writer_thread(): Do not assign recv_writer_thread_active=true
in order to avoid a race condition with
recv_recovery_from_checkpoint_finish().
recv_init_crash_recovery(): Assign recv_writer_thread_active=true
before creating recv_writer_thread.
Remove the debug parameter innodb_force_recovery_crash that was
introduced into MySQL 5.6 by me in WL#6494 which allowed InnoDB
to resize the redo log on startup.
Let innodb.log_file_size actually start up the server, but ensure
that the InnoDB storage engine refuses to start up in each of the
scenarios.
If InnoDB is started in innodb_read_only mode such that
recovered incomplete transactions exist at startup
(but the redo logs are clean), an assertion will fail at shutdown,
because there would exist some non-prepared transactions.
logs_empty_and_mark_files_at_shutdown(): Do not wait for incomplete
transactions to finish if innodb_read_only or innodb_force_recovery>=3.
Wait for purge to finish in only one place.
trx_sys_close(): Relax the assertion that would fail first.
trx_free_prepared(): Also free recovered TRX_STATE_ACTIVE transactions
if innodb_read_only or innodb_force_recovery>=3.
srv_release_threads(): Actually wait for the threads to resume
from suspension. On CentOS 5 and possibly other platforms,
os_event_set() may be lost.
srv_resume_thread(): A counterpart of srv_suspend_thread().
Optionally wait for the event to be set, optionally with a timeout,
and then release the thread from suspension.
srv_free_slot(): Unconditionally suspend the thread. It is always
in resumed state when this function is entered.
srv_active_wake_master_thread_low(): Only call os_event_set().
srv_purge_coordinator_suspend(): Use srv_resume_thread() instead
of the complicated logic.
crashes server
This bug is the result of merging the Oracle MySQL follow-up fix
BUG#22963169 MYSQL CRASHES ON CREATE FULLTEXT INDEX
without merging the base bug fix:
Bug#79475 Insert a token of 84 4-bytes chars into fts index causes
server crash.
Unlike the above mentioned fixes in MySQL, our fix will not change
the storage format of fulltext indexes in InnoDB or XtraDB
when a character encoding with mbmaxlen=2 or mbmaxlen=3
and the length of a word is between 128 and 84*mbmaxlen bytes.
The Oracle fix would allocate 2 length bytes for these cases.
Compatibility with other MySQL and MariaDB releases is ensured by
persisting the used maximum length in the SYS_COLUMNS table in the
InnoDB data dictionary.
This fix also removes some unnecessary strcmp() calls when checking
for the legacy default collation my_charset_latin1
(my_charset_latin1.name=="latin1_swedish_ci").
fts_create_one_index_table(): Store the actual length in bytes.
This metadata will be written to the SYS_COLUMNS table.
fts_zip_initialize(): Initialize only the first byte of the buffer.
Actually the code should not even care about this first byte, because
the length is set as 0.
FTX_MAX_WORD_LEN: Define as HA_FT_MAXCHARLEN * 4 aka 336 bytes,
not as 254 bytes.
row_merge_create_fts_sort_index(): Set the actual maximum length of the
column in bytes, similar to fts_create_one_index_table().
row_merge_fts_doc_tokenize(): Remove the redundant parameter word_dtype.
Use the actual maximum length of the column. Calculate the extra_size
in the same way as row_merge_buf_encode() does.
InnoDB would refuse to start up if there is a mismatch on
the size of the system tablespace files. However, before this
check is conducted, the system tablespace may already have been
heavily modified.
InnoDB should perform the size check as early as possible.
recv_recovery_from_checkpoint_finish():
Move the recv_apply_hashed_log_recs() call to
innobase_start_or_create_for_mysql().
innobase_start_or_create_for_mysql(): Test the mutex functionality
before doing anything else. Use a compile_time_assert() for a
sizeof() constraint. Check the size of the system tablespace as
early as possible.
recv_scan_log_recs(): Remember if redo log apply is needed,
even if starting up in innodb_read_only mode.
recv_recovery_from_checkpoint_start_func(): Refuse
innodb_read_only startup if redo log apply is needed.
buf_flush_init_flush_rbt() was called too early in MariaDB server 10.0,
10.1, MySQL 5.5 and MySQL 5.6. The memory leak has been fixed in
the XtraDB storage engine and in MySQL 5.7.
As a result, when the server is started to initialize new data files,
the buf_pool->flush_rbt will be created unnecessarily and then leaked.
This memory leak was noticed in MariaDB server 10.1 when running the
test encryption.innodb_first_page.
The problem in MariaDB is introduced by this merge commit:
c33db2cdc0
The merge comes from mysql and the original author comes from this
commit from MySQL:
------------------------------------------------
commit 160b823d146288d66638e4a740d6d2da72f9a689
Author: Marc Alff <marc.alff@oracle.com>
Date: Tue Aug 30 12:14:07 2016 +0200
Bug#22551677 SIGNAL 11 IN LF_PINBOX_PUT_PINS
Backport to 5.6
------------------------------------------------
The breaking change is in start_socket_wait_v1 where instead of using
m_thread_owner, we make use of my_pthread_getspecific_ptr to fetch a
thread local storage value. Unfortunately this invalidates the
"m_thread_owner" member when a socket is created. The internals of the
socket structure have m_thread_owner set to NULL, but when checking for
ownership we actually look at the current thread's key store.
This seems incorrect however it is not immediately apparent why.
To not diverge from MySQL's reasoning as it is not described what the
actual problem was that this commit is trying to fix, I have adjusted the
unittest to account for this new behaviour. We destroy the current
thread in the unit test, such that the newly created socket actually has
no thread owner. The m_thread_owner is untouched in all this.
* Update mysqld_safe script to remove duplicated parameter --crash-script
* Make --core-file-size accept underscores as well as dashes correctly.
* Add mysqld_safe_helper to Debian and Ubuntu files.
* Update innodb minor version to 35
Memory was leaked when ALTER TABLE is attempted on a table
that contains corrupted indexes.
The memory leak was reported by AddressSanitizer for the test
innodb.innodb_corrupt_bit. The leak was introduced into
MariaDB Server 10.0.26, 10.1.15, 10.2.1 by the following:
commit c081c978a2
Merge: 1d21b22155a482e76e65
Author: Sergei Golubchik <serg@mariadb.org>
Date: Tue Jun 21 14:11:02 2016 +0200
Merge branch '5.5' into bb-10.0
MariaDB Server 10.0.28 and 10.1.19 merged code from Percona XtraDB
that introduced support for compressed columns. Much but not all
of this code was disabled by placing #ifdef HAVE_PERCONA_COMPRESSED_COLUMNS
around it.
Among the unused but not disabled code is code to access
some new system tables related to compressed columns.
The creation of these system tables SYS_ZIP_DICT and SYS_ZIP_DICT_COLS
would cause a crash in --innodb-read-only mode when upgrading
from an earlier version to 10.0.28 or 10.1.19.
Let us remove all the dead code related to compressed columns.
Users who already upgraded to 10.0.28 and 10.1.19 will have the two
above mentioned empty tables in their InnoDB system tablespace.
Subsequent versions of MariaDB Server will completely ignore those tables.
- in DOMNODELIST::DropItem
if (Listp == NULL || Listp->length <= n)
return true;
is wrong, should be:
if (Listp == NULL || Listp->length < n)
return true;
- Crash in discovery with libxml2 in XMLColumns because:
if (!tdp->Usedom) // nl was destroyed
vp->nl = vp->pn->GetChildElements(g);
is executed with vp->pn uninitialized. Fixed by adding:
vp->pn = node;
line 264.
-In discovery with libxml2 some columns are not found.
Because list was not recovered properly, nodes being modified and not reallocated.
Fixed lines 214 and 277.
modified: storage/connect/domdoc.cpp
modified: storage/connect/tabxml.cpp
Add support for zipped table files
modified: storage/connect/domdoc.cpp
modified: storage/connect/domdoc.h
modified: storage/connect/filamap.cpp
modified: storage/connect/filamap.h
modified: storage/connect/filamzip.cpp
modified: storage/connect/filamzip.h
modified: storage/connect/ha_connect.cc
modified: storage/connect/libdoc.cpp
modified: storage/connect/plgdbutl.cpp
modified: storage/connect/plgxml.cpp
modified: storage/connect/plgxml.h
modified: storage/connect/tabdos.cpp
modified: storage/connect/tabdos.h
modified: storage/connect/tabfmt.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabxml.cpp
Essentially revert MDEV-6759, which addressed a double free of memory
by removing the freeing altogether, introducing the memory leaks.
No double free was observed when running the test suite -DWITH_ASAN.
Replace some mem_heap_free(foreign->heap) with dict_foreign_free(foreign)
so that the calls can be located and instrumented more easily when needed.
A first experimental and limited implementation.
modified: storage/connect/CMakeLists.txt
modified: storage/connect/filamap.cpp
new file: storage/connect/filamzip.cpp
new file: storage/connect/filamzip.h
modified: storage/connect/ha_connect.cc
new file: storage/connect/ioapi.c
new file: storage/connect/ioapi.h
modified: storage/connect/mycat.cc
modified: storage/connect/plgdbsem.h
modified: storage/connect/plgdbutl.cpp
modified: storage/connect/tabdos.cpp
modified: storage/connect/tabdos.h
modified: storage/connect/tabfmt.cpp
modified: storage/connect/tabfmt.h
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabjson.h
new file: storage/connect/tabzip.cpp
new file: storage/connect/tabzip.h
new file: storage/connect/unzip.c
new file: storage/connect/unzip.h
new file: storage/connect/zip.c
be consistent and don't include the table name into the error message,
no other CREATE TABLE error does it.
(the crash happened, because thd->lex->query_tables was NULL)
Fix includes launchpad fix plus more to cover writing BIN tables.
modified: storage/connect/tabfix.cpp
modified: storage/connect/value.cpp
modified: storage/connect/value.h
- Typo: Change the name of filamzip to filamgz to prepare future ZIP tables.
modified: storage/connect/CMakeLists.txt
added: storage/connect/filamgz.cpp
added: storage/connect/filamgz.h
deleted: storage/connect/filamzip.cpp
deleted: storage/connect/filamzip.h
modified: storage/connect/plgdbsem.h
modified: storage/connect/reldef.cpp
modified: storage/connect/tabdos.cpp
modified: storage/connect/tabdos.h
modified: storage/connect/tabfix.cpp
modified: storage/connect/tabfmt.cpp
modified: storage/connect/tabjson.cpp
By setting the context class loader.
modified: storage/connect/JavaWrappers.jar
modified: storage/connect/JdbcInterface.java
modified: storage/connect/mysql-test/connect/std_data/JdbcMariaDB.jar
This is not a fix, this is instrumentation to find out is MySQL frm dictionary
and InnoDB data dictionary really out-of-sync when this assertion is fired,
or is there some other reason (bug).
