Commit graph

1950 commits

Author SHA1 Message Date
Jan Lindström
ea83226872 MDEV-7088: Query stats for compression based on TRIM size
Analysis: Status variables were missing from innodb_status_variables
array.

Fix: Add missing status variables to the array.
2014-11-12 15:37:52 +02:00
Jan Lindström
0f32299437 MDEV-7035: Remove innodb_io_capacity setting depending on
setting of innodb_io_capacity_max

(a) Changed the behaviour so that if you set innodb_io_capacity to a 
value > innodb_io_capacity_max that the value is accepted AND 
that innodb_io_capacity_max = innodb_io_capacity * 2.

(b) If someone wants to reduce innodb_io_capacity_max and 
reduce it below innodb_io_capacity then innodb_io_capacity 
should be reduced to the same level as innodb_io_capacity_max.

In both cases give a warning to user.
2014-11-13 13:24:26 +02:00
Jan Lindström
bff2d46bf7 MDEV-7100: InnoDB error monitor might unnecessary wait log_sys mutex
Analysis: InnoDB error monitor is responsible to call every second
sync_arr_wake_threads_if_sema_free() to wake up possible hanging
threads if they are missed in mutex_signal_object. This is not
possible if error monitor itself is on mutex/semaphore wait. We
should avoid all unnecessary mutex/semaphore waits on error monitor.
Currently error monitor calls function buf_flush_stat_update()
that calls log_get_lsn() function and there we will try to get
log_sys mutex. Better, solution for error monitor is that in
buf_flush_stat_update() we will try to get lsn with
mutex_enter_nowait() and if we did not get mutex do not update
the stats.

Fix: Use log_get_lsn_nowait() function on buf_flush_stat_update()
function. If returned lsn is 0, we do not update flush stats.
log_get_lsn_nowait() will use mutex_enter_nowait() and if
we get mutex we return a correct lsn if not we return 0.
2014-11-13 12:00:57 +02:00
Elena Stepanova
b99328bbf8 Re-enabling tests disabled due to MDEV-5266 and MySQL:65225 (fixed now) 2014-11-17 20:28:18 +04:00
Jan Lindström
8c7ef99bb2 MDEV-7100: InnoDB error monitor might unnecessary wait log_sys mutex
Analysis: InnoDB error monitor is responsible to call every second
sync_arr_wake_threads_if_sema_free() to wake up possible hanging 
threads if they are missed in mutex_signal_object. This is not 
possible if error monitor itself is on mutex/semaphore wait. We 
should avoid all unnecessary mutex/semaphore waits on error monitor.
Currently error monitor calls function buf_flush_stat_update() 
that calls log_get_lsn() function and there we will try to get 
log_sys mutex. Better, solution for error monitor is that in 
buf_flush_stat_update() we will try to get lsn with 
mutex_enter_nowait() and if we did not get mutex do not update 
the stats.

Fix: Use log_get_lsn_nowait() function on buf_flush_stat_update()
function. If returned lsn is 0, we do not update flush stats. 
log_get_lsn_nowait() will use mutex_enter_nowait() and if
we get mutex we return a correct lsn if not we return 0.
2014-11-13 11:24:19 +02:00
Jan Lindström
a03dd94be8 MDEV-6936: Buffer pool list scan optimization
Merged Facebook commit 617aef9f911d825e9053f3d611d0389e02031225
authored by Inaam Rana to InnoDB storage engine (not XtraDB)
from https://github.com/facebook/mysql-5.6

WL#7047 - Optimize buffer pool list scans and related batch processing

Reduce excessive scanning of pages when doing flush list batches. The
fix is to introduce the concept of "Hazard Pointer", this reduces the
time complexity of the scan from O(n*n) to O.

The concept of hazard pointer is reversed in this work. Academically
hazard pointer is a pointer that the thread working on it will declar
such and as long as that thread is not done no other thread is allowe
do anything with it.

In this WL we declare the pointer as a hazard pointer and then if any
thread attempts to work on it, it is allowed to do so but it has to a
the hazard pointer to the next valid value. We use hazard pointer sol
reverse traversal of lists within a buffer pool instance.

Add an event to control the background flush thread. The background f
thread wait has been converted to an os event timed wait so that it c
signalled by threads that want to kick start a background flush when
buffer pool is running low on free/dirty pages.
2014-11-06 13:17:11 +02:00
Jan Lindström
84de277099 Fix error message output if posix_fallocate (trim) is not successfull. 2014-11-05 09:18:47 +02:00
Jan Lindström
8b1b62dd8f Fix compiler failure on Windows. 2014-11-04 15:41:39 +02:00
Jan Lindström
251fa7ffc5 Fix error on trim operation alligment. Furthermore, make sure that
we do not return simulated out of file space on read operation,
that would cause crash.
2014-11-04 12:26:48 +02:00
Alexander Barkov
43f185e171 MDEV-5528 Command line variable to choose MariaDB-5.3 vs MySQL-5.6 temporal data formats 2014-11-03 21:45:06 +04:00
Jan Lindström
2da6f7ceba MDEV-7017: Add function to print semaphore waits
Add function to print to stderr all current semaphore 
waits. This function should be able to executed 
inside a gdb/ddd.
2014-11-03 15:43:44 +02:00
Jan Lindström
cb37c55768 MDEV-6929: Port Facebook Prefix Index Queries Optimization
Merge Facebook commit 154c579b828a60722a7d9477fc61868c07453d08
and e8f0052f9b112dc786bf9b957ed5b16a5749f7fd authored
by Steaphan Greene from https://github.com/facebook/mysql-5.6

Optimize prefix index queries to skip cluster index lookup when possible.

Currently InnoDB will always fetch the clustered index (primary key
index) for all prefix columns in an index, even when the value of a
particular record is smaller than the prefix length. This change
optimizes that case to use the record from the secondary index and avoid
the extra lookup.

Also adds two status vars that track how effective this is:

innodb_secondary_index_triggered_cluster_reads:
Times secondary index lookup triggered cluster lookup.

innodb_secondary_index_triggered_cluster_reads_avoided:
Times prefix optimization avoided triggering cluster lookup.
2014-11-03 11:18:52 +02:00
Jan Lindström
2bf3e416fe MDEV-6932: Enable Lazy Flushing
Merge Facebook commit 4f3e0343fd2ac3fc7311d0ec9739a8f668274f0d
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6

Adds innodb_idle_flush_pct to enable tuning of the page flushing rate
when the system is relatively idle. We care about this, since doing
extra unnecessary flash writes shortens the lifespan of the flash.
2014-10-29 13:49:12 +02:00
Jan Lindström
58888e2c08 MDEV-6935: Change the default value for innodb_log_compressed_pages to false
Merge Facebook commit ca40b4417fd224a68de6636b58c92f133703fc68
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6
Change the default value for innodb_log_compressed_pages to false

Logging these pages is a waste. We don't want this to be enabled.

One caution here: If the zlib version used by innodb is changed, but
the running version is still the previous version, and the running
version crashes, it is possible crash recovery could fail.

When crash recovery uses a zlib version at all different than the
version used by the crashed instance, it is possible that a redone
compression could fail, where the original did not, because the new
zlib version compresses the same data to a slightly larger size.

Because of the nature of compression, this is even possible when
upgrading to a version of zlib which actually peforms overall better
compression than the previous version.

If this happens, mysql will fail to recover, since a page split can
not be safely triggered during crash recovery.

So, either the exact zlib version must be controlled between builds,
or these rare recovery failures must be accepted. The cost of
logging these pages is quite high, so we consider this limitation to
be worthwhile.

This failure scenario can not happen if there was a clean shutdown.
This is only relevant to restarting crashed instances, or starting an
instance built via a hot backup too (XtraBackup).
2014-10-29 11:07:38 +02:00
Jan Lindström
2d2d11f02b MDEV-6968: CREATE TABLE crashes with InnoDB plugin
Analysis: fil_extend_space_to_desired_size() does not provide file
node to os_aio(). This failed on Windows only because on Windows
we do not use posix_fallocate() to extend file space.

Fix: Add file node to os_aio() function call and make sure that
we do not use NULL pointer at os_aio_array_reserve_slot(). Additionally,
make sure that we do not use 0 as file_block_size (512 is the minimum).
2014-10-29 11:07:37 +02:00
Jan Lindström
b96697d286 MDEV-6648: InnoDB: Add support for 4K sector size if supported
New generation hard drives, SSDs and NVM devices support 4K
sector size. Supported sector size can be found using fstatvfs()
or GetDiskFreeSpace() functions.
2014-10-29 11:07:11 +02:00
Jan Lindström
8777e80151 MDEV-6759: innodb valgrind failures
Fix failure seen on dict_foreign_remove_partial().
2014-10-27 16:58:16 +02:00
Jan Lindström
9646f946e9 MDEV-6759: innodb valgrind failures
Fix failure seen on dict_foreign_remove_partial.
2014-10-27 16:34:53 +02:00
Jan Lindström
85c21dbc47 MDEV-6925: Remove bad "" operators.
Merged Facebook commit ec1aac68c74f3c1e558d057c4c9fcfe6edbbea93
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6

In C++11, "" is not parsed as before.  So "A""B" is not the same as "AB".
Instead, whitespace is required, like: "A" "B"
2014-10-26 07:29:37 +02:00
Jan Lindström
caeffc7a7d MDEV-6926: innodb_rows_updated is misleading on slav
Merged Facebook commit dd2d11be7aaf3be270e740fb95cbc4eacb52f4d7
authored by Rongrong Zhong from https://github.com/facebook/mysql-5.6

This fixes MySQL Bug #68220 innodb_rows_updated is misleading on slave
http://bugs.mysql.com/bug.php?id=68220

Added innodb_system_rows_read/inserted/updated/deleted counters
that are the equivalent of innodb_rows_* but that only account for
changes made to system databases (mysql, information_schame and
preformance_schema). These counters will be used on slaves to
differentiated the updates made on system databases from those made on
user databases.

innodb_rows_* status counters are not updated when innodb_system_rows_*
are updated.

dd2d11be7a
2014-10-26 07:22:51 +02:00
Jan Lindström
60e995cfec MDEV-6930: Make innodb_max_dirty_pages_pct my.cnf variable a double
Merged Facebook commit ecff018632c6db49bad73d9233c3cdc9f41430e9
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6

This change is to fix: http://bugs.mysql.com/62534

This makes innodb_max_dirty_pages_pct a double with min,default,max values
0.001, 75, 99.999.

This also makes innodb_max_dirty_pages_pct_lwm and adaptive_flushing_lwm
doubles, as these sysvars are inter-dependent.

Added more to the BUFFER POOL AND MEMORY section of SHOW INNODB STATUS:
Percent pages dirty: X.X
This is all n_dirty_pages / used_pages
Percent all pages dirty: X.X
This is all n_dirty_pages / all-pages
Max dirty pages percent: X.X
This is innodb_max_dirty_pages_pct

Also changed all of buf from 2 to 3 digits of precision (%.2f -> %.3f).
2014-10-25 09:24:39 +03:00
Jan Lindström
cff0012d28 MDEV-6927: Share more structures
Merge Facebook commit f981a51a47519b0ba527917887f8adc6df9ae147
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6.

This just moves some structure definitions from inside a
single .cc file to a shared .h file, with a few tweaks to
allow these structures to be shared.

On its own, it should have no actual effect. This is needed later.
2014-10-25 08:21:52 +03:00
Jan Lindström
3486135bb5 MDEV-6928: Add trx pointer to struct mtr_t
Merge Facebook commit 25295d003cb0c17aa8fb756523923c77250b3294
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6

This adds a pointer to the trx to each mtr.
This allows the trx to be accessed in parts of the code
where it was otherwise not available. This is needed later.
2014-10-24 22:26:31 +03:00
Jan Lindström
ca0bdc4d5c MDEV-6937: buf_flush_LRU() does not return correct number in case of
compressed pages

buf_flush_LRU() returns the number of pages processed. There are
two types of processing that can happen. A page can get evicted or
a page can get flushed. These two numbers are quite distinct and
should not be mixed.
2014-10-24 22:02:54 +03:00
Jan Lindström
78e31f0a2e MDEV-6931: Page cleaner should do LRU flushing regardless of server activity
Merge Facebook commit 926a077b14b73c14094de7fc7aa913241b801b4d
authored by Inaam Rana from https://github.com/facebook/mysql-5.6.

This is fix for upstream bugs
http://bugs.mysql.com/bug.php?id=71988
http://bugs.mysql.com/bug.php?id=70500

page_cleaner should work whether or not there is server activity.
Its iterations become a noop when there is no work to do but we
should not tie it to the server activity.

The page_cleaner thread does spurious background flushing
because of conditional sleep between iterations. The solution
is not to make sleep dependent on server activity etc.
2014-10-24 18:58:04 +03:00
Jan Lindström
7e71dfa9f5 MDEV-6933: Spurious lock_wait_timeout_thread wakeup in lock_wait_suspend_thread()
Merged Facebooks commit 6e06bbfa315ffb97d713dd6e672d6054036ddc21
authored by Inaam Rana from https://github.com/facebook/mysql-5.6.

Fixes MySQL bug http://bugs.mysql.com/bug.php?id=72123

lock_timeout thread works in a tight loop waking up every second
and checking for lock_wait_timeout. In addition, when a mysql
thread is forced to wait on a lock, it signals the lock_timeout thread
as well. This call is not required. In a heavily contended workload
each thread going to wait will signal the lock_timeout thread making
it work all the time. As lock_timeout thread scans the array of
waiting threads under lock_sys::wait_mutex which is already very
hot in contneded loads, these extra scans can cause significanct
performance regression.

Also, in various codepaths lock_timeout thread is signalled where
actual intention was to signal the innodb monitor thread.
2014-10-24 17:56:04 +03:00
Jan Lindström
f0debf2feb MDEV-6934: os_event_wait_time_low(): wait time calculation is messed up
Merged Facebook commit bdab302a7e3c37da21a1bffe1550cdbe6c906695
by Inaam Rana from https://github.com/facebook/mysql-5.6.

In os_event_wait_time_low() the logic to calculate abs_time
for wait is broken. The bug has been present at least since
5.5. It gets acutely sensitized when sub-second wait intervals
are passed. It is particularly relevant to us because the
page_cleaner thread will mostly request sub-second wait
intervals. This can potentially lead to a near tight loop
behaviour of page_cleaner with much less sleep then what we'd
actually expect.
2014-10-24 17:11:09 +03:00
Jan Lindström
cc656e4d85 Enhance row-merge sort progress logging. 2014-10-22 16:14:25 +03:00
Jan Lindström
c1f5f61c9b Make sure that information schema knows also page compressed page
type and that we decompress fist page if it is page compressed
before really accessing it.
2014-10-20 15:53:07 +03:00
Kristian Nielsen
4c07b93bdb Fix missing UNIV_INTERN on dict_table_check_foreign_keys().
When UNIV_INTERN is missing in built-in XtraDB, this causes the
innodb_plugin to call the XtraDB version of the function instead
of its own (seen in --embedded-server test failure in Buildbot).
This in turn causes bad things to happen in case of difference
between XtranDB and innodb_plugin.
2014-10-20 10:50:10 +02:00
Sergei Golubchik
42f359f982 after-merge fixes 2014-10-15 22:42:08 +02:00
Sergei Golubchik
f62c12b405 Merge 10.0.14 into 10.1 2014-10-15 12:59:13 +02:00
Elena Stepanova
5c33632b50 engine_condition_pushdown has been deprecated 2014-10-12 14:26:27 +04:00
Sergei Golubchik
7f5e51b940 MDEV-34 delete storage/ndb and sql/*ndb* (and collateral changes)
remove:
* NDB from everywhere
* IM from mtr-v1
* packaging/rpm-oel and packaging/rpm-uln
* few unused spec files
* plug.in file
* .bzrignore
2014-10-11 18:53:06 +02:00
Sergei Golubchik
1b75bed00f 5.5.40+ merge 2014-10-09 10:30:11 +02:00
Monty
4af97bc0e3 Removed files that had been accidentally committed
Removed compiler warnings
2014-10-07 11:39:42 +03:00
Sergei Golubchik
1ddfce4840 mysql-5.5.40 2014-10-06 19:53:55 +02:00
Sergei Golubchik
22f9353bfd xtradb/innodb: fix to compile with VS 2008
and compiler warnings
2014-10-04 15:26:04 +02:00
Nirbhay Choubey
b197066ca0 MDEV-6807: InnoDB: Assertion failure in file lock0lock.cc (lock != ctx->wait_lock)
Merged patch for lp:1364840.
2014-10-03 21:22:41 -04:00
Jan Lindström
7474e7baa0 MDEV-6807: InnoDB: Assertion failure in file lock0lock.cc
(lock != ctx->wait_lock)

References: lp:1364840 lp:1280896 - reverted a part of fix for
lp:1280896 (updating a unique key can cause parallel applying to hang )
in revision #4105. This "BF (brute force) lock skipping" caused
regression which surfaced in randgen test for bug lp:1364840
2014-10-02 21:17:57 +03:00
Jan Lindström
251239af67 Fix Windows compiler error 'log2f': identifier not found 2014-10-02 15:09:17 +03:00
Sergei Golubchik
7aabc2ded2 fixing embedded: WaaS. Wsrep as a Service. 2014-10-01 23:48:34 +02:00
Sergei Golubchik
c6b95222c3 use MD5 service in innodb/xtradb 2014-10-01 23:38:28 +02:00
Sergei Golubchik
3620910eea cleanup: galera merge, simple changes 2014-10-01 23:38:27 +02:00
Sergey Vojtovich
5e1fad8512 MDEV-6533 - MySQL Bug#72718 - CACHE_LINE_SIZE in innodb should be 128 on POWER
This patch is suggested by Stewart Smith.
Define CACHE_LINE_SIZE to 128 on POWER.
2014-10-03 17:38:46 +04:00
Michael Widenius
70823e1d91 MDEV-5120 Test suite test maria-no-logging fails
The reason for the failure was a bug in an include file on debian that causes 'struct stat'
to have different sized depending on the environment.

This patch fixes so that we always include my_global.h or my_config.h before we include any other files.

Other things:
- Removed #include <my_global.h> in some include files; Better to always do this at the top level to have as few
  "always-include-this-file-first' files as possible.
- Removed usage of some include files that where already included by my_global.h or by other files.


client/mysql_plugin.c:
  Use my_global.h first
client/mysqlslap.c:
  Remove duplicated include files
extra/comp_err.c:
  Remove duplicated include files
include/m_string.h:
  Remove duplicated include files
include/maria.h:
  Remove duplicated include files
libmysqld/emb_qcache.cc:
  Use my_global.h first
plugin/semisync/semisync.h:
  Use my_pthread.h first
sql/datadict.cc:
  Use my_global.h first
sql/debug_sync.cc:
  Use my_global.h first
sql/derror.cc:
  Use my_global.h first
sql/des_key_file.cc:
  Use my_global.h first
sql/discover.cc:
  Use my_global.h first
sql/event_data_objects.cc:
  Use my_global.h first
sql/event_db_repository.cc:
  Use my_global.h first
sql/event_parse_data.cc:
  Use my_global.h first
sql/event_queue.cc:
  Use my_global.h first
sql/event_scheduler.cc:
  Use my_global.h first
sql/events.cc:
  Use my_global.h first
sql/field.cc:
  Use my_global.h first
  Remove duplicated include files
sql/field_conv.cc:
  Use my_global.h first
sql/filesort.cc:
  Use my_global.h first
  Remove duplicated include files
sql/gstream.cc:
  Use my_global.h first
sql/ha_ndbcluster.cc:
  Use my_global.h first
sql/ha_ndbcluster_binlog.cc:
  Use my_global.h first
sql/ha_ndbcluster_cond.cc:
  Use my_global.h first
sql/ha_partition.cc:
  Use my_global.h first
sql/handler.cc:
  Use my_global.h first
sql/hash_filo.cc:
  Use my_global.h first
sql/hostname.cc:
  Use my_global.h first
sql/init.cc:
  Use my_global.h first
sql/item.cc:
  Use my_global.h first
sql/item_buff.cc:
  Use my_global.h first
sql/item_cmpfunc.cc:
  Use my_global.h first
sql/item_create.cc:
  Use my_global.h first
sql/item_geofunc.cc:
  Use my_global.h first
sql/item_inetfunc.cc:
  Use my_global.h first
sql/item_row.cc:
  Use my_global.h first
sql/item_strfunc.cc:
  Use my_global.h first
sql/item_subselect.cc:
  Use my_global.h first
sql/item_sum.cc:
  Use my_global.h first
sql/item_timefunc.cc:
  Use my_global.h first
sql/item_xmlfunc.cc:
  Use my_global.h first
sql/key.cc:
  Use my_global.h first
sql/lock.cc:
  Use my_global.h first
sql/log.cc:
  Use my_global.h first
sql/log_event.cc:
  Use my_global.h first
sql/log_event_old.cc:
  Use my_global.h first
sql/mf_iocache.cc:
  Use my_global.h first
sql/mysql_install_db.cc:
  Remove duplicated include files
sql/mysqld.cc:
  Remove duplicated include files
sql/net_serv.cc:
  Remove duplicated include files
sql/opt_range.cc:
  Use my_global.h first
sql/opt_subselect.cc:
  Use my_global.h first
sql/opt_sum.cc:
  Use my_global.h first
sql/parse_file.cc:
  Use my_global.h first
sql/partition_info.cc:
  Use my_global.h first
sql/procedure.cc:
  Use my_global.h first
sql/protocol.cc:
  Use my_global.h first
sql/records.cc:
  Use my_global.h first
sql/records.h:
  Don't include my_global.h
  Better to do this at the upper level
sql/repl_failsafe.cc:
  Use my_global.h first
sql/rpl_filter.cc:
  Use my_global.h first
sql/rpl_gtid.cc:
  Use my_global.h first
sql/rpl_handler.cc:
  Use my_global.h first
sql/rpl_injector.cc:
  Use my_global.h first
sql/rpl_record.cc:
  Use my_global.h first
sql/rpl_record_old.cc:
  Use my_global.h first
sql/rpl_reporting.cc:
  Use my_global.h first
sql/rpl_rli.cc:
  Use my_global.h first
sql/rpl_tblmap.cc:
  Use my_global.h first
sql/rpl_utility.cc:
  Use my_global.h first
sql/set_var.cc:
  Added comment
sql/slave.cc:
  Use my_global.h first
sql/sp.cc:
  Use my_global.h first
sql/sp_cache.cc:
  Use my_global.h first
sql/sp_head.cc:
  Use my_global.h first
sql/sp_pcontext.cc:
  Use my_global.h first
sql/sp_rcontext.cc:
  Use my_global.h first
sql/spatial.cc:
  Use my_global.h first
sql/sql_acl.cc:
  Use my_global.h first
sql/sql_admin.cc:
  Use my_global.h first
sql/sql_analyse.cc:
  Use my_global.h first
sql/sql_audit.cc:
  Use my_global.h first
sql/sql_base.cc:
  Use my_global.h first
sql/sql_binlog.cc:
  Use my_global.h first
sql/sql_bootstrap.cc:
  Use my_global.h first
  Use my_global.h first
sql/sql_cache.cc:
  Use my_global.h first
sql/sql_class.cc:
  Use my_global.h first
sql/sql_client.cc:
  Use my_global.h first
sql/sql_connect.cc:
  Use my_global.h first
sql/sql_crypt.cc:
  Use my_global.h first
sql/sql_cursor.cc:
  Use my_global.h first
sql/sql_db.cc:
  Use my_global.h first
sql/sql_delete.cc:
  Use my_global.h first
sql/sql_derived.cc:
  Use my_global.h first
sql/sql_do.cc:
  Use my_global.h first
sql/sql_error.cc:
  Use my_global.h first
sql/sql_explain.cc:
  Use my_global.h first
sql/sql_expression_cache.cc:
  Use my_global.h first
sql/sql_handler.cc:
  Use my_global.h first
sql/sql_help.cc:
  Use my_global.h first
sql/sql_insert.cc:
  Use my_global.h first
sql/sql_lex.cc:
  Use my_global.h first
sql/sql_load.cc:
  Use my_global.h first
sql/sql_locale.cc:
  Use my_global.h first
sql/sql_manager.cc:
  Use my_global.h first
sql/sql_parse.cc:
  Use my_global.h first
sql/sql_partition.cc:
  Use my_global.h first
sql/sql_plugin.cc:
  Added comment
sql/sql_prepare.cc:
  Use my_global.h first
sql/sql_priv.h:
  Added error if we use this before including my_global.h
  This check is here becasue so many files includes sql_priv.h first.
sql/sql_profile.cc:
  Use my_global.h first
sql/sql_reload.cc:
  Use my_global.h first
sql/sql_rename.cc:
  Use my_global.h first
sql/sql_repl.cc:
  Use my_global.h first
sql/sql_select.cc:
  Use my_global.h first
sql/sql_servers.cc:
  Use my_global.h first
sql/sql_show.cc:
  Added comment
sql/sql_signal.cc:
  Use my_global.h first
sql/sql_statistics.cc:
  Use my_global.h first
sql/sql_table.cc:
  Use my_global.h first
sql/sql_tablespace.cc:
  Use my_global.h first
sql/sql_test.cc:
  Use my_global.h first
sql/sql_time.cc:
  Use my_global.h first
sql/sql_trigger.cc:
  Use my_global.h first
sql/sql_udf.cc:
  Use my_global.h first
sql/sql_union.cc:
  Use my_global.h first
sql/sql_update.cc:
  Use my_global.h first
sql/sql_view.cc:
  Use my_global.h first
sql/sys_vars.cc:
  Added comment
sql/table.cc:
  Use my_global.h first
sql/thr_malloc.cc:
  Use my_global.h first
sql/transaction.cc:
  Use my_global.h first
sql/uniques.cc:
  Use my_global.h first
sql/unireg.cc:
  Use my_global.h first
sql/unireg.h:
  Removed inclusion of my_global.h
storage/archive/ha_archive.cc:
  Added comment
storage/blackhole/ha_blackhole.cc:
  Use my_global.h first
storage/csv/ha_tina.cc:
  Use my_global.h first
storage/csv/transparent_file.cc:
  Use my_global.h first
storage/federated/ha_federated.cc:
  Use my_global.h first
storage/federatedx/federatedx_io.cc:
  Use my_global.h first
storage/federatedx/federatedx_io_mysql.cc:
  Use my_global.h first
storage/federatedx/federatedx_io_null.cc:
  Use my_global.h first
storage/federatedx/federatedx_txn.cc:
  Use my_global.h first
storage/heap/ha_heap.cc:
  Use my_global.h first
storage/innobase/handler/handler0alter.cc:
  Use my_global.h first
storage/maria/ha_maria.cc:
  Use my_global.h first
storage/maria/unittest/ma_maria_log_cleanup.c:
  Remove duplicated include files
storage/maria/unittest/test_file.c:
  Added comment
storage/myisam/ha_myisam.cc:
  Move sql_plugin.h first as this includes my_global.h
storage/myisammrg/ha_myisammrg.cc:
  Use my_global.h first
storage/oqgraph/oqgraph_thunk.cc:
  Use my_config.h and my_global.h first
  One could not include my_global.h before oqgraph_thunk.h (don't know why)
storage/spider/ha_spider.cc:
  Use my_global.h first
storage/spider/hs_client/config.cpp:
  Use my_global.h first
storage/spider/hs_client/escape.cpp:
  Use my_global.h first
storage/spider/hs_client/fatal.cpp:
  Use my_global.h first
storage/spider/hs_client/hstcpcli.cpp:
  Use my_global.h first
storage/spider/hs_client/socket.cpp:
  Use my_global.h first
storage/spider/hs_client/string_util.cpp:
  Use my_global.h first
storage/spider/spd_conn.cc:
  Use my_global.h first
storage/spider/spd_copy_tables.cc:
  Use my_global.h first
storage/spider/spd_db_conn.cc:
  Use my_global.h first
storage/spider/spd_db_handlersocket.cc:
  Use my_global.h first
storage/spider/spd_db_mysql.cc:
  Use my_global.h first
storage/spider/spd_db_oracle.cc:
  Use my_global.h first
storage/spider/spd_direct_sql.cc:
  Use my_global.h first
storage/spider/spd_i_s.cc:
  Use my_global.h first
storage/spider/spd_malloc.cc:
  Use my_global.h first
storage/spider/spd_param.cc:
  Use my_global.h first
storage/spider/spd_ping_table.cc:
  Use my_global.h first
storage/spider/spd_sys_table.cc:
  Use my_global.h first
storage/spider/spd_table.cc:
  Use my_global.h first
storage/spider/spd_trx.cc:
  Use my_global.h first
storage/xtradb/handler/handler0alter.cc:
  Use my_global.h first
storage/xtradb/handler/i_s.cc:
  Use my_global.h first
2014-09-30 20:31:14 +03:00
Jan Lindström
fc2df3c637 MDEV-6812: Merge Kakao: Add global status variables which tell
you the progress of inplace alter table and row log buffer usage

-    (x 100%, it's 4-digit. 10000 means 100.00%)
-    Innodb_onlineddl_rowlog_rows
       Shows how many rows are stored in row log buffer.
-    Innodb_onlineddl_rowlog_pct_used
       Shows row log buffer usage in percent ( *100%, it's 4-digit. 10000 means 100.00% ).
-    Innodb_onlineddl_pct_progress
       Shows the progress of inplace alter table. It might
       be not so accurate because inplace alter is highly
       depend on disk and buffer pool status.
       But still it is useful and better than nothing.

-    Add some log for inplace alter table
       XtraDB/InnoDB will print some message before and
       after doing some task.
2014-09-30 14:50:34 +03:00
Nirbhay Choubey
c916085e27 bzr merge -rtag:mariadb-10.0.14 maria/10.0/ 2014-09-28 20:43:56 -04:00
Sergei Golubchik
7e29c1b539 5.5 merge 2014-09-16 14:03:17 +02:00
Elena Stepanova
d2ae40ac2e Fixes in storage_engine test suite
- after a change in 5.6.20, tests with big blobs require bigger
    innodb_log_file_size value;
  - innobase relay was running with the default InnoDB (XtraDB)
2014-09-12 03:21:54 +04:00
Sergei Golubchik
75796d9ecb InnoDB 5.6.20 2014-09-11 16:42:54 +02:00
Jan Lindström
c30a8442e8 MDEV-6729: InnoDB: Failing assertion: state == TRX_STATE_NOT_STARTED in
file trx0trx.ic line 60 

Problem was that trx might have not been started when we enter release
savepoint, this can happen when trx with savepoint is already aborted 
or when we try to release non-existing savepoint.
2014-09-11 15:41:30 +03:00
Jan Lindström
7e2bc140e3 Remove unnecessary debug output causing unnecessary warnings. 2014-09-10 20:19:41 +03:00
Sergei Golubchik
8deb9066e2 fix compilation on windows - wrong include file 2014-09-08 17:10:48 +02:00
Murthy Narkedimilli
3139aa87b4 Adding patch for security bug 19471516 2014-09-08 11:33:55 +02:00
Jan Lindström
d3ceb934f1 MDEV-6701: InnoDB tests fail in 10.1
Fixed test failures seen on defragment tests, innodb.innodb-wl5522-debug-zip
and innodb.innodb_bug12902967.
2014-09-08 09:34:03 +03:00
Sergey Vojtovich
c01c819209 Backport from 10.0:
MDEV-6483 - Deadlock around rw_lock_debug_mutex on PPC64

This problem affects only debug builds on PPC64.

There are at least two race conditions around
rw_lock_debug_mutex_enter and rw_lock_debug_mutex_exit:

- rw_lock_debug_waiters was loaded/stored without setting
  appropriate locks/memory barriers.
- there is a gap between calls to os_event_reset() and
  os_event_wait() and in such case we're supposed to pass
  return value of the former to the latter.

Fixed by replacing self-cooked spinlocks with system mutexes.
These days system mutexes offer much better performance. OTOH
performance is not that critical for debug builds.
2014-08-29 16:14:11 +04:00
Sergey Vojtovich
40497577ff Backport from 10.0:
MDEV-6450 - MariaDB crash on Power8 when built with advance tool
            chain

InnoDB mutex_exit() function calls __sync_test_and_set() to release
the lock. According to manual this function is supposed to create
"acquire" memory barrier whereas in fact we need "release" memory
barrier at mutex_exit().

The problem isn't repeatable with gcc because it creates
"acquire-release" memory barrier for __sync_test_and_set().
ATC creates just "acquire" barrier.

Fixed by creating proper barrier at mutex_exit() by using
__sync_lock_release() instead of __sync_test_and_set().
2014-08-29 16:02:46 +04:00
Jan Lindström
422b99ed87 Fix incorrect merge. 2014-08-27 20:00:13 +03:00
Jan Lindström
6907da2341 Fix small error on LZMA compression error printout. 2014-08-27 15:35:49 +03:00
Jan Lindström
4fb45aa219 Fix compiler error when WITH_WSREP is not used. 2014-08-27 15:28:43 +03:00
Jan Lindström
a60ea193ba Fix compiler error when WITH_WSREP is not used. 2014-08-27 15:19:45 +03:00
Jan Lindström
ab150128ce MDEV-6247: Merge 10.0-galera to 10.1.
Merged lp:maria/maria-10.0-galera up to revision 3880.

    Added a new functions to handler API to forcefully abort_transaction,
    producing fake_trx_id, get_checkpoint and set_checkpoint for XA. These
    were added for future possiblity to add more storage engines that
    could use galera replication.
2014-08-27 13:15:37 +03:00
Jan Lindström
df4dd593f2 MDEV-6247: Merge 10.0-galera to 10.1.
Merged lp:maria/maria-10.0-galera up to revision 3879.

Added a new functions to handler API to forcefully abort_transaction,
producing fake_trx_id, get_checkpoint and set_checkpoint for XA. These
were added for future possiblity to add more storage engines that
could use galera replication.
2014-08-26 15:43:46 +03:00
Jan Lindström
18b307a7d2 MDEV-6590: InnoDB recover ends in signal 11
Analysis: When database is migrated from 5.5 or earlier and
database needs crash recovery, there is possibility that
SYS_DATAFILES system table does not exists, but
crash recovery in function dict_check_tablespaces_and_store_max_id()
assumes that SYS_DATAFILES exists.

Fix: If SYS_DATAFILES does not exists, create it before
we end up to function dict_check_tablespaces_and_store_max_id()
on crash recovery.
2014-08-25 13:35:33 +03:00
Jan Lindström
9b506d4bcb MDEV-6602: rpl.rpl_mdev6020 fails sporadically with SIGABRT
Analysis: Problem is that we execute galera code when we are actually
executing asyncronoush replication.

Fix: Do not execute galera code if wsrep provider is not set.
2014-08-25 09:13:15 +03:00
Michael Widenius
5569132ffe MDEV-6450 - MariaDB crash on Power8 when built with advance tool chain
Part of this work is based on Stewart Smitch's memory barrier and lower priori
patches for power8.

- Added memory syncronization for innodb & xtradb for power8.
- Added HAVE_WINDOWS_MM_FENCE to CMakeList.txt
- Added os_isync to fix a syncronization problem on power
- Added log_get_lsn_nowait which is now used srv_error_monitor_thread to ensur
  if log mutex is locked.

All changes done both for InnoDB and Xtradb
2014-08-19 19:28:35 +03:00
Jan Lindström
8984bef2e4 MDEV-6172: Monitor progress of ALTER TABLE ... ADD INDEX,
ALGORITHM=INPLACE for InnoDB

Added thd_progress calls to row_merge_sort to give
some hint how merge sort progresses.
2014-08-18 10:52:59 +03:00
Jan Lindström
2062c9a65a MDEV-6567: Raw debug output in the error log.
Removed raw debug output when no error on OS operation has not happened.
2014-08-13 09:03:28 +03:00
Nirbhay Choubey
857abf1481 bzr merge -r4104..4120 codership/5.6/ 2014-08-12 14:05:44 -04:00
Nirbhay Choubey
8358dd53b7 bzr merge -r4346 maria/10.0 (maria-10.0.13) 2014-08-11 23:55:41 -04:00
Monty
3bde13932e Minor cleanups, fix compiler warnings 2014-08-09 13:22:01 +03:00
Sergei Golubchik
6fb17a0601 5.5.39 merge 2014-08-07 18:06:56 +02:00
Jan Lindström
d87ffeb491 MDEV-6548: Incorrect compression on LZMA.
Analysis: Provided incorrect parameter to output buffer size
and incorrectly determined actual payload size after compression.
2014-08-07 13:44:00 +03:00
Jan Lindström
50777e26f0 Fix Windows compiler error by disabling for now the nullptr class
implementation.
2014-08-07 13:41:46 +03:00
Sergei Golubchik
2023fac281 innodb-5.6.19 2014-08-06 20:05:10 +02:00
Jan Lindström
6dad23f04a MDEV-5834: Merge Kakao Defragmentation implementation to MariaDB 10.1
Merge https://github.com/kakao/mariadb-10.0 that contains Facebook's
    implementation for defragmentation

    facebook/mysql-5.6@a2d3a74
    facebook/mysql-5.6@def96c8
    facebook/mysql-5.6@9c67c5d
    facebook/mysql-5.6@921a81b
    facebook/mysql-5.6@aa519bd
    facebook/mysql-5.6@fea7d13
    facebook/mysql-5.6@09b29d3
    facebook/mysql-5.6@9284abb
    facebook/mysql-5.6@dbd623d
    facebook/mysql-5.6@aed55dc
    facebook/mysql-5.6@aad5c82

    This version does not add new SQL-syntax and new handler API function.
    Instead optimize table is mapped to defragment table if
    innodb_defragment=ON, by default the feature is off.

    Contains changes authored by Sunguck Lee (Kakao).
2014-08-06 15:28:58 +03:00
Sergei Golubchik
be253829d7 compilation failure on labrador 2014-08-06 10:11:08 +02:00
unknown
07668e6751 Remove unstable test case innodb_bug18942294, approved by Jimmy over IM. 2014-08-06 09:51:20 +08:00
Sergey Vojtovich
82ce2a2503 MDEV-6450 - MariaDB crash on Power8 when built with advance
tool chain

Reverted addition to the original patch:
InterlockedExchangeAcquire and InterlockedAndRelease are
supported only on Itanium-based systems.
2014-08-04 11:55:38 +04:00
Sergei Golubchik
1c6ad62a26 mysql-5.5.39 merge
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
2014-08-02 21:26:16 +02:00
Nirbhay Choubey
30b5a4de5d MDEV-6492: MariaDB Galera Cluster cant use rsync sst
A donor node does a flush tables and then tries to
freeze innodb writes before proceeding with SST.
However, innodb_disallow_writes was missing in xtradb.
Merged 'InnodbFreeze' patch from maria-5.5-galera's to
xtradb. Also, merged some changes missing in innobase's
os0file.cc.

Added a basic test case for innodb_disallow_writes system
variable.
2014-07-31 16:40:32 -04:00
Sergey Vojtovich
53360fd45f MDEV-6450 - MariaDB crash on Power8 when built with advance
tool chain

This is an addition to the original patch. On Windows
InterlockedExchange implies full memory barrier, whereas
only acquire/release barriers required.
2014-07-31 14:31:05 +04:00
Jan Lindström
c39a501cbe Try to fix compiler error seen on Labrador. 2014-07-31 14:43:35 +03:00
Jan Lindström
e974b56438 MDEV-6512: InnoDB: Assertion failure in thread 4537024512 in file
buf0buf.cc line 2642.

Analysis: innodb_compression_algorithm is a global variable and
can change while we are building page compressed page. This could
lead page corruption.

Fix: Cache innodb_compression_algorithm on local variable before
doing any compression or page formating to avoid concurrent
change. Improved page verification on debug builds.
2014-07-31 11:31:39 +03:00
Sergei Golubchik
3e9d4541d5 MDEV-6340 Mariadb 10.0.12 fatal "Lost connection" error w/ GCC 4.9 'Release' build; workaround ~ CFLAGS="-fno-delete-null-pointer-checks"
don't use attribute nonnull for arguments that can be null
2014-07-31 09:51:05 +02:00
Michael Widenius
a1c1700b89 Fixed some compiler warnings 2014-07-30 10:05:01 +03:00
Sergei Golubchik
6ef139780d MDEV-6497 InnoDB deadlocks on UNINSTALL PLUGIN
Free the trx of the current thd (if any) in innobase_end()
2014-07-29 09:09:52 +02:00
Jan Lindström
476feba941 Fix unnecessary printout of same writer thread more than once. Fixed
also a compiler warning.
2014-07-26 20:17:59 +03:00
Jan Lindström
56c4b016ad Fiix random test failures on fil_decompress_page_2 function.
Analysis: InnoDB writes also files that do not contain FIL-header.
This could lead incorrect analysis on os_fil_read_func function
when it tries to see is page page compressed based on FIL_PAGE_TYPE
field on FIL-header. With bad luck uncompressed page that does
not contain FIL-headed, the byte on FIL_PAGE_TYPE position could
indicate that page is page comrpessed.

Fix: Upper layer must indicate is file space page compressed
or not. If this is not yet known, we need to read the FIL-header
and find it out. Files that we know that are not page compressed
we can always just provide FALSE.
2014-07-25 14:37:10 +03:00
Jan Lindström
a3acd72570 Merge InnoDB fixes from 5.5 revisions 4229, 4230, 4233, 4237 and 4238 i.e.
4229: MDEV-5670: Assertion failure in file buf0lru.c line 2355
      Add more status information if repeatable.

4230: MDEV-5673: Crash while parallel dropping multiple tables under heavy load
      Improve long semaphore wait output to include all semaphore waits
      and try to find out if there is a sequence of waiters.

4233: Fix compiler errors on product build.

4237: Fix too agressive long semaphore wait output and add guard against introducing
      compression failures on insert buffer.

4238: Fix test failure caused by simulated compression failure on
      IBUF_DUMMY table.
2014-07-25 10:30:16 +03:00
Sergey Vojtovich
6192f0bffa MDEV-6483 - Deadlock around rw_lock_debug_mutex on PPC64
This problem affects only debug builds on PPC64.

There are at least two race conditions around
rw_lock_debug_mutex_enter and rw_lock_debug_mutex_exit:

- rw_lock_debug_waiters was loaded/stored without setting
  appropriate locks/memory barriers.
- there is a gap between calls to os_event_reset() and
  os_event_wait() and in such case we're supposed to pass
  return value of the former to the latter.

Fixed by replacing self-cooked spinlocks with system mutexes.
These days system mutexes offer much better performance. OTOH
performance is not that critical for debug builds.
2014-07-24 18:12:32 +04:00
Jan Lindström
c104965746 Fix test failure caused by simulated compression failure on
IBUF_DUMMY table.
2014-07-25 09:34:05 +03:00
Jan Lindström
7bf45bec54 Fix too agressive long semaphore wait output and add guard against introducing
compression failures on insert buffer.
2014-07-24 14:35:09 +03:00
Jan Lindström
be00265557 Fix compiler errors on product build. 2014-07-23 13:52:17 +03:00
Jan Lindström
47daf3b422 Fix default value for innodb_compression_algorithm to 0 (uncompressed)
to avoid test failures.
2014-07-23 11:55:34 +03:00
Jan Lindström
a1e41e3258 MDEV-6470: Restrict number of error messages about persistent statictic tables not found
If mysql.innodb_table_stats or mysql.innodb_index_stats is not found or has
unexpected structure output that error only once and no other error for
every table trying to use them. If they do exists, then print fetch or
recalculation errors only once / table or index.
2014-07-22 19:31:45 +03:00
Jan Lindström
ef67c3a239 MDEV-6443: Server crashed with assertaion failure in file ha_innodb.cc
line 8473

In case InnoDB index is not found, print the MySQL and InnoDB index
name we were trying to find and all MySQL and InnoDB index names there
is for this table.
2014-07-22 13:17:16 +03:00
Jan Lindström
fe3859cc9d MDEV-6443: Server crashed with assertaion failure in file
ha_innodb.cc line 8473

If index is not found from InnoDB make sure we print what we
were trying to find and all mysql and InnoDB index names there
is for this table.
2014-07-22 13:08:32 +03:00
Jan Lindström
0bb0230e3a MDEV-6426: Maria DB crashes randomly on creating indexes
Improve OS error messages on Windows.
2014-07-22 10:10:56 +03:00