Commit graph

175385 commits

Author SHA1 Message Date
Oleksandr Byelkin
098dff10ba MDEV-11359 Implement IGNORE for bulk operation 2016-11-29 08:29:46 +01:00
Otto Kekäläinen
d9c03c4339 Deb: Streamline package listing order to make comparisons downstream easier 2016-11-27 22:25:17 +01:00
Otto Kekäläinen
ca3df8f68d Deb: wrap-and-sort for easier comparison to downstream in future 2016-11-27 22:25:17 +01:00
Otto Kekäläinen
f977709b65 Deb: Make libmariadb3 to provide the libmysqlclient.so.XX links
For backwards compatibility reasons.

E.g. innotop builds against libmysqlclient.so and expects to find
a libmysqlclient.so.18 library and does not use libmariadb.so.3 even
when built from our own sources.

This commit mimics to dcfe6cb834
2016-11-27 22:25:17 +01:00
Otto Kekäläinen
1965f03c80 Deb: Rename libmariadbclient-dev to libmariadb-dev
This change was accidentally omitted from commit 2bcc16c17b
2016-11-27 22:25:17 +01:00
Otto Kekäläinen
2949282ccf Deb: Fix libmariadbclient.so.18 link path to point to libmariadb.so.3
As libmariadb.so.3 does not reside in a multi-arch directory (yet),
use the plain path.
2016-11-27 22:25:16 +01:00
Alexander Barkov
9bfde897c3 MDEV-11360 Dynamic SQL: DEFAULT as a bind parameter
This patch adds DEFAULT as a possible dynamic SQL parameter, e.g.:
  EXECUTE IMMEDIATE 'INSERT INTO t1 (column) VALUES(?)' USING DEFAULT;
  EXECUTE IMMEDIATE 'UPDATE t1 SET column=?' USING DEFAULT;
and for similar PREPARE..EXECUTE queries.

This is done for symmetry with the STMT_INDICATOR_DEFAULT indicator in
the client-server PS protocol.

The changes include:
- Allowing DEFAULT as a possible option in execute USING clause (sql_yacc.yy)
- Adding "virtual bool Item::save_in_param(THD *thd, Item_param *param)",
  because "normal" items (that have real values) and Item_default_value
  have now different actions when assigning itself as an Item_param value.
- Fixing switch() statements in a few Item_param methods not to have "default",
  because it was easy to forget to add a new "case" when adding a new XXX_VALUE
  value into the enum Item_param::enum_item_param_state.
  This is important, as we'll be adding new values soon, e.g. for MDEV-11359.
  Removing "default" helped to find and report bugs MDEV-11361 and MDEV-11362,
  because DECIMAL_VALUE is obviously not properly handled in some cases.
2016-11-27 18:21:18 +04:00
Igor Babaev
1d0f17415a Fixed bug mdev-11313.
The fix for bug 11072 was not complete though it also fixed
the bug mdev-10800.
This patch resolves the problems of all three bugs.
2016-11-26 21:23:39 -08:00
Otto Kekäläinen
b5b68b6bb8 MDEV-10126: replace deprecated iproute dependency with iproute2
Fall back to old iproute on Debian Wheezy and Ubuntu Precise that
don't have iproute2 available.
2016-11-26 22:25:40 +01:00
Otto Kekäläinen
3a6e781ea3 MDEV-9165: Run chown much faster on the datadir during install/update 2016-11-26 19:13:26 +01:00
Marko Mäkelä
618edd4057 Merge branch 'kevgs-10.2_warnings' into 10.2 2016-11-25 14:30:47 +02:00
Marko Mäkelä
cc3aba26ff Merge branch '10.2_warnings' of https://github.com/kevgs/server into kevgs-10.2_warnings
Revert the XtraDB changes, because 10.2 does not currently build with XtraDB.

Also omit some changes that need further investigation.

Ensure that all callers of partition_info::get_clone() are passing this!=NULL.
2016-11-25 14:28:31 +02:00
Marko Mäkelä
d247d64988 MDEV-11349 (2/2) Fix some bogus-looking Valgrind warnings
buf_block_init(): Initialize buf_page_t::flush_type.
For some reason, Valgrind 3.12.0 would seem to flag some
bits in adjacent bitfields as uninitialized, even though only
the two bits of flush_type were left uninitialized. Initialize
the field to get rid of many warnings.

buf_page_init_low(): Initialize buf_page_t::old.
For some reason, Valgrind 3.12.0 would seem to flag all 32
bits uninitialized when buf_page_init_for_read() invokes
buf_LRU_add_block(bpage, TRUE). This would trigger bogus warnings
for buf_page_t::freed_page_clock being uninitialized.
(The V-bits would later claim that only "old" is initialized
in the 32-bit word.) Perhaps recent compilers
(GCC 6.2.1 and clang 4.0.0) generate more optimized x86_64 code
for bitfield operations, confusing Valgrind?

mach_write_to_1(), mach_write_to_2(), mach_write_to_3():
Rewrite the assertions that ensure that the most significant
bits are zero. Apparently, clang 4.0.0 would optimize expressions
of the form ((n | 0xFF) <= 0x100) to (n <= 0x100). The redundant
0xFF was added in the first place in order to suppress a
Valgrind warning. (Valgrind would warn about comparing uninitialized
values even in the case when the uninitialized bits do not affect
the result of the comparison.)
2016-11-25 12:43:43 +02:00
Marko Mäkelä
cdaa1d768e MDEV-11349 (1/2) Fix some clang 4.0 warnings
In functions that declare pointer parameters as nonnull,
remove nullness checks, because GCC would optimize them away anyway.

Use #ifdef instead of #if when checking for a configuration flag.

Clang says that left shifts of negative values are undefined.
So, use ~0U instead of ~0 in a number of macros.

Some functions that were defined as UNIV_INLINE were declared as
UNIV_INTERN. Consistently use the same type of linkage.

ibuf_merge_or_delete_for_page() could pass bitmap_page=NULL to
buf_page_print(), conflicting with the __attribute__((nonnull)).
2016-11-25 12:43:34 +02:00
Sergey Vojtovich
1a1749e38c MDEV-11296 - InnoDB stalls under OLTP RW on P8
Threads may fall asleep forever while acquiring InnoDB rw-lock on Power8. This
regression was introduced along with InnoDB atomic operations fixes.

The problem was that proper memory order wasn't enforced between "writers"
store and "lock_word" load:

  my_atomic_store32((int32*) &lock->waiters, 1);
  ...
  local_lock_word = lock->lock_word;

Locking protocol is such that store to "writers" must be completed before load
of "lock_word". my_atomic_store32() was expected to enforce memory order because
it issues strongest MY_MEMORY_ORDER_SEQ_CST memory barrier.

According to C11:
- any operation with this memory order is both an acquire operation and a
  release operation
- for atomic store order must be one of memory_order_relaxed
  memory_order_release or memory_order_seq_cst. Otherwise the behavior is
  undefined.

That is it doesn't say explicitly that this expectation is wrong, but there are
indications that acquire (which is actually supposed to guarantee memory order
in this case) may not be issued along with MY_MEMORY_ORDER_SEQ_CST.

A good fix for this is to encode waiters into lock_word, but it is a bit too
intrusive. Instead we change atomic store to atomic fetch-and-store, which
does memory load and is guaranteed to issue acquire.
2016-11-25 12:41:35 +04:00
Sergey Vojtovich
fb7caad72b MDEV-11296 - InnoDB stalls under OLTP RW on P8
Simplify away recursive flag: it is not necessary for rw-locks to operate
properly. Now writer_thread != 0 means recursive.

As we only need correct value of writer_thread only in writer_thread itself
it is rather safe to load and update it non-atomically.

This patch also fixes potential reorder of "writer_thread" and "recursive"
loads (aka MDEV-7148), which was reopened along with InnoDB thread fences
simplification. Previous versions are unaffected, because they have os_rmb
in rw_lock_lock_word_decr(). It wasn't observed at the moment of writing
though.
2016-11-25 12:41:35 +04:00
Sergey Vojtovich
68a8537327 MDEV-11296 - InnoDB stalls under OLTP RW on P8
Clean-up INNODB_RW_LOCKS_USE_ATOMICS: it is always set.
2016-11-25 12:41:35 +04:00
Sergey Vojtovich
8d010c44ef MDEV-11296 - InnoDB stalls under OLTP RW on P8
Simplified away rw_lock_get_waiters(), rw_lock_set_waiter_flag(),
rw_lock_reset_waiter_flag(). Let waiters have predictable data type.
2016-11-25 12:41:35 +04:00
Sergey Vojtovich
bb7e84b79a MDEV-11296 - InnoDB stalls under OLTP RW on P8
Simplified away rw_lock_lock_word_incr().
2016-11-25 12:41:35 +04:00
Otto Kekäläinen
e06e455eea Deb: make server core package breaks/replaces earlier client packages
This is required, as the innochecksum binary has moved package.
Without this change the following error would be emitted:

  Unpacking mariadb-server-core-10.2 (10.2.3+maria~jessie) over (10.2.2+maria-1~jessie) ...
  dpkg: error processing archive /var/cache/apt/archives/mariadb-server-core-10.2_10.2.3+maria~jessie_amd64.deb (--unpack):
    trying to overwrite '/usr/bin/innochecksum', which is also in package mariadb-client-10.2 10.2.2+maria-1~jessie
2016-11-25 08:50:58 +01:00
Otto Kekäläinen
89236a8ad9 Deb: skip invoke-rc.d mysql stop if no mysql process is running at all
Quite often in upgrades on systemd systems dpkg emitted an error like:

  Failed to stop mysql.service: Unit mysql.service not loaded.
  invoke-rc.d: initscript mysql, action "stop" failed.
  invoke-rc.d returned 5
  There is a MySQL server running, but we failed in our attempts to stop it.
  Stop it yourself and try again!
  dpkg: error processing archive /var/cache/apt/archives/mariadb-server-10.2

This is because the mariadb/mysql.service file is not loaded during
the upgrade/unpack stage of dpkg in certain situations. With this simple
check we can easily skip the shutdown step when it is really not needed,
which is for sure the case if no mysqld process at all is running on the
entire system.
2016-11-25 08:50:57 +01:00
Lawrin Novitsky
e726ae6b04 Made all capability bit constants to be ULL. All new MariaDB
capabilities(in upper 32bits) were sent as 0 on 32bit Windows. They were
reset because CLIENT_SSL, CLIENT_COMPRESS and CLIENT_SSL_VERIFY_SERVER_CERT were defined as UL
Plus MARIADB_CLIENT_STMT_BULK_OPERATIONS was defined as 1UL << 34, and that is undefined operation.
2016-11-24 17:39:12 +01:00
kevg
780db8e252 fix build and some warnings 2016-11-24 17:36:02 +03:00
Igor Babaev
f4d6f26a4f Fixed bug mdev-11315.
There were no implementations for the virtual functions
exclusive_dependence_on_table_processor and
exclusive_dependence_on_table_processor. As a result
the procedure pushdown_cond_for_derived erroneously
detected some conditions with outer references as pushable
into materialized view / derived table.
2016-11-22 17:19:29 -08:00
Oleksandr Byelkin
779d416a99 MDEV-10724 Assertion `vcol_table == 0 || vcol_table == table' failed in fill_record(THD*, TABLE*, List<Item>&, List<Item>&, bool, bool)
Attempt to insert in several tables now checked just by table map.
2016-11-22 13:53:17 +01:00
Nirbhay Choubey
4a27ab23cb MDEV-10792: Assertion `thd->mdl_context.is_lock_owner ..
.. (MDL_key::TABLE, table->db, table->table_name, MDL_SHARED)'
failed in mysql_rm_table_no_locks

Reset error flag after temporary table has been
successfully dropped.
2016-11-21 17:14:14 -05:00
Nirbhay Choubey
ebe061900b MDEV-10442: "Address already in use" on restart
SST processes should inherit mysqld's process group.
2016-11-21 16:20:10 -05:00
Nirbhay Choubey
44ccb8f29e MDEV-10432: Post-fix after merging PR#205 2016-11-21 15:38:33 -05:00
Bernard Spil
cf1b0c1ace Implement native/base process checks for FreeBSD
- Make rsync process checks OS-dependent
  - Use (BSD) netstat on FreeBSD (not lsof)
2016-11-21 15:38:33 -05:00
Bernard Spil
f16ead51fb POSIX-ify wsrep scripts
- Replace #!/bin/bash with #!/bin/sh
  - Split username:password using POSIX compat %% and ##
  - Don't use array for FILTERS
  - Replace == tests with POSIX-compat =
2016-11-21 15:38:33 -05:00
Igor Babaev
665045f985 Fixed bug mdev-11081.
The idea of this fix was taken from the patch by Roy Lyseng
for mysql-5.6 bug iBug#14740889: "Wrong result for aggregate
functions when executing query through cursor".

Here's Roy's comment for his patch:
"
The problem was that a grouped query did not behave properly when
executed using a cursor. On further inspection, the query used one
intermediate temporary table for the grouping.
Then, Select_materialize::send_result_set_metadata created a temporary
table for storing the query result. Notice that get_unit_column_types()
is used to retrieve column meta-data for the query. The items contained
in this list are later modified so that their result_field points to
the row buffer of the materialized temporary table for the cursor.
But prior to this, these result_field objects have been prepared for
use in the grouping operation, by JOIN::make_tmp_tables_info(), hence
the grouping operation operates on wrong column buffers.

The problem is solved by using the list JOIN::fields when copying data
to the materialized table. This list is set by JOIN::make_tmp_tables_info()
and points to the columns of the last intermediate temporary table of
the executed query. For a UNION, it points to the temporary table
that is the result of the UNION query.

Notice that we have to assign a value to ::fields early in JOIN::optimize()
in case the optimization shortcuts due to a const plan detection.

A more optimal solution might be to avoid creating the final temporary
table when the query result is already stored in a temporary table.
"
The patch does not contain a test case, but the description of the
problem corresponds exactly what could be observed in the test
case for mdev-11081.
2016-11-21 10:33:39 -08:00
Kristian Nielsen
022aeda4c0 Attempt to fix strange rpm dependency issue following prior patch 2016-11-21 08:24:10 +01:00
Sergei Petrunia
58532f36d0 Update the testcase for MDEV-10330
Instead of restart_mysqld.inc, truncate the P_S tables that are used
by the test. We don't want to query some irrelevant data.
2016-11-21 02:32:48 +03:00
Kristian Nielsen
f77bd5faf5 Fix use of require in mysql-test-run.
The motivation for this is that Perl is moving towards not having
current directory ./ in @INC by default. This is causing
mysql-test-run.pl to fail in latest Debian Unstable:

  https://lists.debian.org/debian-devel-announce/2016/08/msg00013.html

However, we have `use "lib"`, there is no need for current directory
in @INC, except for a gross hack. In mtr_cases.pm, there is a
`require "mtr_misc.pl"`, which hides mtr_misc.pl away in mtr_cases
namespace. And things only work because mysql-test-run.pl loads it
with a different name, `require "lib/mtr_misc.pl"`! (Perl will
`require` only once for each unique filename).

Fix this by only using `require` in main program, and referencing
functions with :: scope from other namespaces. For multi-use in
different namespaces, proper `use` modules should be used.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2016-11-20 14:52:49 +02:00
Otto Kekäläinen
d49cffa124 Deb: provide the libmysqlclient shim packages that exist in Debian/Ubuntu
This way all 3rd party apps configured to build against any MySQL or
MariaDB version will pick up the libmariadb3-dev package instead.
2016-11-20 10:51:39 +02:00
Otto Kekäläinen
dcfe6cb834 Deb: make libmariadb3 to provide the libmariadbclient.so.18 link
In Debian many existing applications in Debian/Ubuntu have been built
while libmariadbclient-dev or libmariadbclient-dev-compat was installed.

To satisfy installation dependencies, the package name libmariadbclient18
must be provided, and at runtime a shared library must by the name
libmariadbclient.so.18.

Provide these to remain backwards compatible.
2016-11-20 10:51:14 +02:00
Otto Kekäläinen
2bcc16c17b Deb: rename client library packages to reflect its contents
The new library name libmariadb3 matches the libmariadb.so.3 filename.
Packages that want to build against MariaDB Connector C have as
build-dependency libmariadb-dev and as run-time dependency libmariadb3.

Make the package provide libmariadbclient18 for backwards compatibility,
though compatibility is not always assured. There library did change
to a whole new generation after all, even though ABI compatibility has
been a design goal.

Also do the equivalent change for the -dev package. Packages that
explicitly want to use the MariaDB Connector C should mark it as
their build-depends.

Also provide an empty libmariadbclient18 metapackage to facilitate
upgrades from old MariaDB installs, just like there is an empty
libmysqlclient18 package. Create more of these in the future as needed.
2016-11-20 10:49:12 +02:00
Otto Kekäläinen
54bd67a4ed Deb: provide the default-mysql-* packages that exist in Debian/Ubuntu
Since Debian 9 (Stretch) and Ubuntu 16.10 (Yakkety) the following
packages have existed:
* virtual-mysql-client
* virtual-mysql-client-core
* virtual-mysql-server
* virtual-mysql-server-core
* virtual-libmysqlclient-dev

They are metapackages that in Debian depend on MariaDB and in
Ubuntu currently on MySQL. We need to provide them and point
them to MariaDB so that systems that have the mariadb.org
repositories enabled automatically get everything MariaDB
and not MySQL.

This change makes the packaging provide the four first ones,
and later commits will fix the client library issues.
2016-11-20 10:49:12 +02:00
Otto Kekäläinen
df8ba7a3a7 Deb: install GSSAPI and Cracklib config files 2016-11-20 10:49:12 +02:00
Otto Kekäläinen
68535b5649 Deb: correct comment about socket auth in Spider/Mroong maintainer scripts 2016-11-20 10:49:11 +02:00
Sergei Petrunia
039bab2290 Undo the unfinished patch for MDEV-8359:
Patch b8b1b928ff was pushed by mistake
2016-11-20 00:26:40 +03:00
Alexander Barkov
e0fc6dc8bd Merge pull request #244 from hholzgra/hartmut-MDEV-726
MDEV-726 - CREATE and ALTER SERVER need to lowercase host name
2016-11-19 12:59:22 +04:00
Otto Kekäläinen
45049864b4 Deb: install libmysqld.so and libmysqld.so.19 in correct packages
The versioned so-file belongs to the run-time package and the unversioned
symlink shall be in the development package.
2016-11-18 22:12:50 +02:00
Sergei Petrunia
a4dc9561ee MDEV-10330: main.show_explain_ps fails sporadically in buildbot
The reason is that selecting from events_waits_history_long creates
a race condition: it can happen that SHOW EXPLAIN thread was kicked
off CPU exactly after posting a SHOW EXPLAIN request and then it wont
need to wait.

It doesn't seem to make sense to add more waits to stabilize the testcase.
Let's instead make a check that SHOW EXPLAIN statement has a
"stage/sql/show explain" stage.
2016-11-18 22:46:54 +03:00
Sergei Petrunia
d66e1119d3 Merge branch '10.2' of github.com:MariaDB/server into 10.2 2016-11-18 22:33:25 +03:00
Oleksandr Byelkin
b162068456 Fixed embedded prepared statements. 2016-11-18 18:29:03 +01:00
Sergey Vojtovich
84fd0bc43c Update libmariadb 2016-11-17 23:38:08 +04:00
Sergey Vojtovich
e1caf318b9 Mark spider tests big
Due to high memory reqirements spider tests fail often on automated testing
VM's due to rather limited resource allocation.

For example with 10.2 spider needs at least 200M * 8 mysqld instances = 1.6Gb
RAM per mtr instance. With --parallel=4 it needs 6.4Gb, while appropriate hosts
have just 3Gb.
2016-11-17 17:36:06 +04:00
Daniel Black
7072ca1f58 MDEV-5725: mysqld embedded libraries into libmariadbd19 Deb package
libmariadbd19 was intended to be added as the package that
included the libmysqld shared library. This was missing
from the debian control file.

The libmariadbd-dev package requires libmariadbd19 to provide
the shared library.

The shared libraries for embedded mysql will go into the libmariadbd18
package rather than the libmariadbd-dev development package.

/usr/bin/mariadb_config is a executable that assists embedded developers
to use the correctly correct header and library files during their
development.

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2016-11-17 12:37:21 +02:00
Daniel Black
2387c91b8a MDEV-5725: Don't install private mysql header files
Previously private/*.h where included in the package. These represent internal
mysqld structures that aren't guarenteed to provide a stable ABI.

There aren't intended to be used by embedded mysqld applications so
they have been removed.

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2016-11-17 12:37:21 +02:00