Analysis:
A query with implicit grouping is one with aggregate functions and
no GROUP BY clause. MariaDB inherits from MySQL an SQL extenstion
that allows mixing aggregate functions with non-aggregate fields.
If a query with such mixed select clause produces an empty result
set, the meaning of aggregate functions is well defined - either
NULL (MIN, MAX, etc.), or 0 (count(*)). However the non-aggregated
fields must also have some value, and the only reasonable value in
the case of empty result is NULL.
The cause of the many wrong results was that if a field is declared
as non-nullable (e.g. because it is a PK or NOT NULL), the semantic
analysis and the optimization phases treat this field as non-nullable,
and generate all related query plan elements based on this assumption.
Later during execution, these incorrectly configured/generated query
plan elements result in a wrong result because the selected fields
are not null due to the not-null assumption during optimization.
Solution:
Detect before the context analysys phase that a query uses implicit
grouping with mixed aggregates/non-aggregates, and set all fields
as nullable. The parser already walks the SELECT clause, and
already sets Item::with_sum_func for Items that reference aggreagate
functions. The patch adds a symmetric Item::with_field so that all
Items that reference an Item_field are marked during their
construction at parse time in the same way as with aggregate function
use.
- Fixed some issues with partitions and connection_string, which also fixed lp:716890 "Pre- and post-recovery crash in Aria"
- Fixed wrong assert in Aria
Now need to merge with latest xtradb before pushing
sql/ha_partition.cc:
Ensure that m_ordered_rec_buffer is not freed before close.
sql/mysqld.cc:
Changed to use opt_stack_trace instead of opt_pstack.
Removed references to pstack
sql/partition_element.h:
Ensure that connect_string is initialized
storage/maria/ma_key_recover.c:
Fixed wrong assert
Open issues:
- A better fix for #57688; Igor is working on this
- Test failure in index_merge_innodb.test ; Igor promised to look at this
- Some Innodb tests fails (need to merge with latest xtradb) ; Kristian promised to look at this.
- Failing tests: innodb_plugin.innodb_bug56143 innodb_plugin.innodb_bug56632 innodb_plugin.innodb_bug56680 innodb_plugin.innodb_bug57255
- Werror is disabled; Should be enabled after merge with xtradb.
Bug#57820 extractvalue crashes
Problem: ExtractValue and Replace crashed in some cases
due to invalid handling of empty and NULL arguments.
Per file comments:
@mysql-test/r/ctype_ujis.result
@mysql-test/r/xml.result
@mysql-test/t/ctype_ujis.test
@mysql-test/t/xml.test
Adding tests
@sql/item_strfunc.cc
Make sure Item_func_replace::val_str safely handles empty strings.
@sql/item_xmlfunc.cc
set null_value if nodeset_func returned NULL,
which is possible when the second argument is an
unset user variable.
Problem: some call of INET_NTOA() function may lead
to a crash due to missing its character set initialization.
Fix: explicitly set the character set.
mysql-test/r/func_misc.result:
Fix for bug#57283: inet_ntoa() crashes
- test result.
mysql-test/t/func_misc.test:
Fix for bug#57283: inet_ntoa() crashes
- test case.
sql/item_strfunc.cc:
Fix for bug#57283: inet_ntoa() crashes
- explicitly set buffer's character set.
Problem: if multibyte and binary string arguments passed to
RPAD(), LPAD() or INSERT() functions, they might return
wrong results or even lead to a server crash due to missed
character set convertion.
Fix: perform the convertion if necessary.
mysql-test/r/ctype_utf8.result:
Fix for bug#57272: crash in rpad() when using utf8
- test result.
mysql-test/t/ctype_utf8.test:
Fix for bug#57272: crash in rpad() when using utf8
- test case.
sql/item_strfunc.cc:
Fix for bug#57272: crash in rpad() when using utf8
- convert multibyte argument's character set to binary in case of
FUNCTION(MULTIBYTE_ARG, .., BINARY_ARG,..) for RPAD(), LPAD() and
INSERT() functions.
Changed to use longlong10_to_str() instead of longlong2str() when base is 10 or -10 as former is much faster than later
Changed my_vsnprintf() to use longlong2str instead of int2str() to get rid of warnings and to get support for long pointers even when long is 32 bit.
client/mysqltest.cc:
longlong2str() -> longlong10_to_str()
include/m_string.h:
Added extra argument to longlong2str() to make it have same prototype is int2str()
mysys/charset.c:
Fixed compiler warning
mysys/mf_soundex.c:
Fixed compiler warning
mysys/my_getopt.c:
longlong2str() -> longlong10_to_str()
sql/create_options.cc:
Fixed compiler warning
sql/item_strfunc.cc:
Added extra argument to longlong2str
sql/opt_range.cc:
longlong2str() -> longlong10_to_str()
sql/partition_info.cc:
longlong2str() -> longlong10_to_str()
sql/slave.cc:
longlong2str() -> longlong10_to_str()
sql/sql_bitmap.h:
Added extra argument to longlong2str
sql/sql_partition.cc:
Added extra argument to longlong2str
sql/sql_select.cc:
longlong2str() -> longlong10_to_str()
sql/sql_show.cc:
Added extra argument to longlong2str
storage/innodb_plugin/handler/ha_innodb.cc:
Update to new parameters for longlong2str()
storage/maria/ma_dbug.c:
longlong2str() -> longlong10_to_str()
storage/maria/maria_chk.c:
Added extra argument to longlong2str
storage/myisam/mi_dbug.c:
longlong2str() -> longlong10_to_str()
storage/myisam/myisamchk.c:
Added extra argument to longlong2str
storage/xtradb/handler/ha_innodb.cc:
Update to new parameters for longlong2str()
strings/longlong2str.c:
Added extra argument to longlong2str() to make it have same prototype is int2str()
strings/my_vsnprintf.c:
Changed my_vsnprintf() to use longlong2str instead of int2str() to get rid of warnings and to get support for long pointers even when long is 32 bit.
Added cast to get rid of compiler warnings
Fixed some bugs introduced in 5.1.47
Disabled some tests until we have merged with latest Xtradb
configure.in:
Added testing if valgrind/memcheck.h exists
storage/pbxt/src/ha_pbxt.cc:
LOCK_plugin is not anymore locked in init
Iterative patch improvement. Previously committed patch
caused wrong result on Windows. The previous patch also
broke secure_file_priv for symlinks since not all file
paths which must be compared against this variable are
normalized using the same norm.
The server variable opt_secure_file_priv wasn't
normalized properly and caused the operations
LOAD DATA INFILE .. INTO TABLE ..
and
SELECT load_file(..)
to do different interpretations of the
--secure-file-priv option.
The patch moves code to the server initialization
routines so that the path always is normalized
once and only once.
It was also intended that setting the option
to an empty string should be equal to
lifting all previously set restrictions. This
is also fixed by this patch.
mysql-test/r/loaddata.result:
* Removed test code which will currently break the much used --mem feature of mtr.
mysql-test/t/loaddata.test:
* Removed test code which will currently break the much used --mem feature of mtr.
sql/item_strfunc.cc:
* Replaced string comparing code on opt_secure_file_priv with an interface which guarantees that both file paths are normalized using the same norm on all platforms.
sql/mysql_priv.h:
* Added signature for is_secure_file_path()
sql/mysqld.cc:
* New function for checking if a path compatible with the secure path restriction.
* Added initialization of the opt_secure_file_priv variable.
sql/sql_class.cc:
* Replaced string comparing code on opt_secure_file_priv with an interface which guarantees that both file paths are normalized using the same norm on all platforms.
sql/sql_load.cc:
* Replaced string comparing code on opt_secure_file_priv with an interface which guarantees that both file paths are normalized using the same norm on all platforms.
Procedure, while DECIMAL works
Selecting of the CONCAT(...<SP variable>...) result into
a user variable may return wrong data.
Item_func_concat::val_str contains a number of memory
allocation-saving tricks. One of them concatenates
strings inplace inserting the value of one string
at the beginning of the other string. However,
this trick didn't care about strings those points
to the same data buffer: this is possible when
a CONCAT() parameter is a stored procedure variable -
Item_sp_variable::val_str() uses the intermediate
Item_sp_variable::str_value field, where it may
store a reference to an external buffer.
The Item_func_concat::val_str function has been
modified to take into account val_str functions
(such as Item_sp_variable::val_str) that return
a pointer to an internal Item member variable
that may reference to a buffer provided.
mysql-test/r/func_concat.result:
Test case for the bug #40625.
mysql-test/t/func_concat.test:
Test case for the bug #40625.
sql/item_strfunc.cc:
Bug #40625: Concat fails on DOUBLE values in a Stored
Procedure, while DECIMAL works
The Item_func_concat::val_str function has been
modified to take into account val_str functions
(such as Item_sp_variable::val_str) that return
a pointer to an internal Item member variable
that may reference to a buffer provided.
Rename method as to not hide a base.
Reorder attributes initialization.
Remove unused variable.
Rework code to silence a warning due to assignment used as truth value.
sql/item_strfunc.cc:
Rename method as to not hide a base.
sql/item_strfunc.h:
Rename method as to not hide a base.
sql/log_event.cc:
Reorder attributes initialization.
sql/rpl_injector.cc:
Rework code to silence a warning due to assignment used as truth value.
sql/rpl_record.cc:
Remove unused variable.
sql/sql_db.cc:
Rework code to silence a warning due to assignment used as truth value.
sql/sql_parse.cc:
Rework code to silence a warning due to assignment used as truth value.
sql/sql_table.cc:
Rework code to silence a warning due to assignment used as truth value.
MySQL's hash functions MD5 and SHA relied on the somewhat slow
sprintf function to convert the digests to hex representations.
This patch replaces the sprintf with a specific and inline hex
conversion function.
Patch contributed by Jan Steemann.
sql/item_strfunc.cc:
Add a hex conversion function.
- Marked a couple of tests with --big
- Fixed xtradb/handler/ha_innodb.cc to call explain_filename()
storage/xtradb/handler/ha_innodb.cc:
Call explain_filename() to get proper names for partitioned tables
Selecting of the CONCAT_WS(...<PS parameter>...) result into
a user variable may return wrong data.
Item_func_concat_ws::val_str contains a number of memory
allocation-saving optimization tricks. After the fix
for bug 46815 the control flow has been changed to a
branch that is commented as "This is quite uncommon!":
one of places where we are trying to concatenate
strings inplace. However, that "uncommon" place
didn't care about PS parameters, that have another
trick in Item_sp_variable::val_str(): they use the
intermediate Item_sp_variable::str_value field,
where they may store a reference to an external
argument's buffer.
The Item_func_concat_ws::val_str function has been
modified to take into account val_str functions
(such as Item_sp_variable::val_str) that return a
pointer to an internal Item member variable that
may reference to a buffer provided.
mysql-test/r/func_concat.result:
Added test case for bug #50096.
mysql-test/t/func_concat.test:
Added test case for bug #50096.
sql/item_strfunc.cc:
Bug #50096: CONCAT_WS inside procedure returning wrong data
The Item_func_concat_ws::val_str function has been
modified to take into account val_str functions
(such as Item_sp_variable::val_str) that return a
pointer to an internal Item member variable that
may reference to a buffer provided.
The problem was that the multiple evaluations of a ENCODE or
DECODE function within a single statement caused the random
generator to be reinitialized at each evaluation, even though
the parameters were constants.
The solution is to initialize the random generator only once
if the password (seed) parameter is constant.
This patch borrows code and ideas from Georgi Kodinov's patch.
mysql-test/r/func_str.result:
Add test case result.
mysql-test/r/ps.result:
Add test case result.
mysql-test/t/func_str.test:
Add test case for Bug#49141
mysql-test/t/ps.test:
Add test case for Bug#49141
sql/item_strfunc.cc:
Move seed generation code to a separate method.
Seed only once if the password (seed) argument
is constant.
Remove duplicated code and use a transform method
to apply encoding or decoding.
sql/item_strfunc.h:
Add parameter to signal whether the PRNG is already seeded.
Introduce transform method.
Combine val_str methods.
sql/sql_crypt.cc:
Remove method.
sql/sql_crypt.h:
Seed is supplied as two long integers.
Problem: Some system functions that could return different values on
master and slave were not marked unsafe. In particular:
GET_LOCK
IS_FREE_LOCK
IS_USED_LOCK
MASTER_POS_WAIT
RELEASE_LOCK
SLEEP
SYSDATE
VERSION
Fix: Mark these functions unsafe.
mysql-test/extra/rpl_tests/rpl_stm_000001.test:
- The test does not work in mixed mode any more, since it tries to
simulate an error in the sql thread in a query that uses get_lock.
Since get_lock now causes the query to be logged in row format,
the error didn't happen. Hence, we now force statement mode.
- Warnings must be disabled when the unsafe query is issued.
- Replaced some save_master_pos+connection slave+sync_with_master
by sync_slave_with_master.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
updated result file
mysql-test/suite/binlog/r/binlog_stm_row.result:
updated result file
mysql-test/suite/binlog/r/binlog_unsafe.result:
updated result file
mysql-test/suite/binlog/t/binlog_killed.test:
binlog_killed only works in statement format now, since
it switches to row mode in mixed mode.
mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test:
suppress warnings for unsafe statements
mysql-test/suite/binlog/t/binlog_stm_row.test:
- Suppress warnings in test that causes warnings.
- The test sets binlog format explicitly, so no need to execute it
twice.
mysql-test/suite/binlog/t/binlog_unsafe.test:
Added test for all unsafe system functions. This test also includes
system functions that were unsafe prior to BUG#47995.
mysql-test/suite/rpl/r/rpl_err_ignoredtable.result:
updated result file
mysql-test/suite/rpl/r/rpl_get_lock.result:
updated result file
mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result:
new result file
mysql-test/suite/rpl/r/rpl_stm_000001.result:
updated result file
mysql-test/suite/rpl/r/rpl_trigger.result:
updated result file
mysql-test/suite/rpl/t/rpl_err_ignoredtable.test:
- suppress warnings for unsafe statement
- replaced save_master_pos+connection slave+sync_with_master
with sync_slave_with_master
mysql-test/suite/rpl/t/rpl_get_lock.test:
update test case that causes new warnings
mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test:
Added new test case for nondeterministic functions.
mysql-test/suite/rpl/t/rpl_trigger.test:
update test case that causes new warnings
sql/item_create.cc:
Marked some system functions unsafe.
sql/item_strfunc.cc:
Clarified comment related to this bug.
sql/sql_yacc.yy:
Marked sysdate unsafe.
- Moved some code from innodb_plugin to xtradb, to ensure that all tests runs
- Did changes in pbxt and maria storage engines becasue of changes in thd->query
- Reverted wrong code in sql_table.cc for how ROW_FORMAT is used.
This is a re-commit of Monty's merge to eliminate an extra commit from
MySQL-5.1.42 that was accidentally included in the merge.
This is a merge of the MySQL 5.1.41 clone-off (clone-5.1.41-build). In
case there are any extra changes done before final MySQL 5.1.41
release, these will need to be merged later before MariaDB 5.1.41
release.
The problem is that argument buffer can be used as result buffer
and it leads to argument value change.
The fix is to use 'old buffer' as result buffer only
if first argument is not constant item.
mysql-test/r/func_str.result:
test result
mysql-test/t/func_str.test:
test case
sql/item_strfunc.cc:
The problem is that argument buffer can be used as result buffer
and it leads to argument value change.
The fix is to use 'old buffer' as result buffer only
if first argument is not constant item.
* Finished Monty and Jani's merge
* Some InnoDB tests still fail (because it's old xtradb code run against
newer testsuite). They are expected to go after mergning with the latest
xtradb.
The assertion in String::copy was added in order to avoid
valgrind errors when the destination was the same as the source.
Eased restriction to allow for the case when str == NULL.
mysql-test/r/func_set.result:
Bug#45168: Test result
mysql-test/t/func_set.test:
Bug#45168: Test case
sql/item_strfunc.cc:
Bug#45168: Code cleanup and grammar correction in comment
sql/sql_string.cc:
Bug#45168: Fix
old_password() functions
The PASSWORD() and OLD_PASSWORD() functions could lead to
memory reads outside of an internal buffer when used with BLOB
arguments.
String::c_ptr() assumes there is at least one extra byte
in the internally allocated buffer when adding the trailing
'\0'. This, however, may not be the case when a String object
was initialized with externally allocated buffer.
The bug was fixed by adding an additional "length" argument to
make_scrambled_password_323() and make_scrambled_password() in
order to avoid String::c_ptr() calls for
PASSWORD()/OLD_PASSWORD().
However, since the make_scrambled_password[_323] functions are
a part of the client library ABI, the functions with the new
interfaces were implemented with the 'my_' prefix in their
names, with the old functions changed to be wrappers around
the new ones to maintain interface compatibility.
mysql-test/r/func_crypt.result:
Added a test case for bug #44767.
mysql-test/t/func_crypt.test:
Added a test case for bug #44767.
sql/item_strfunc.cc:
Use the new my_make_scrambled_password*() to avoid
String::c_ptr().
sql/item_strfunc.h:
Changed Item_func[_old]_password::alloc() interfaces so that
we can use the new my_make_scrambled_password*() functions.
sql/mysql_priv.h:
Added declarations for the new my_make_scrambled_password*()
functions.
sql/password.c:
Added new my_make_scrambled_password*() functions with an
additional "length" argument. Changed ones to be wrappers
around the new ones to maintain interface compatibility.
sql/sql_yacc.yy:
Utilize the new password hashing functions with additional length
argument.
- Version number.
- Valgrind false alarms in libz.
- New variant of suppression for Valgrind warning in dlclose().
- Fix double free() in plugin init error case.
configure.in:
Fix version number. We should reset the maria variant back to `1' when the MySQL version
number increases.
include/my_sys.h:
Fix false alarms in Valgrind for zlib.
Apply same fix as for archive storage handler also to the cases of compression in the
client protocol, and to the compression SQL function.
mysql-test/valgrind.supp:
A new variant of the dlclose() suppression is needed now.
mysys/my_compress.c:
Fix false alarms in Valgrind for zlib.
Apply same fix as for archive storage handler also to the cases of compression in the
client protocol, and to the compression SQL function.
sql/handler.cc:
Fix a double free() in error case for plugin initialisation.
sql/item_strfunc.cc:
Fix false alarms in Valgrind for zlib.
Apply same fix as for archive storage handler also to the cases of compression in the
client protocol, and to the compression SQL function.
bug#44766: valgrind error when using convert() in a subquery
Problem: input and output buffers may be the same
converting a string to some charset.
That may lead to wrong results/valgrind warnings.
Fix: use different buffers.
mysql-test/r/cast.result:
Fix for bug#44743: Join in combination with concat does not always work
bug#44766: valgrind error when using convert() in a subquery
- test result.
mysql-test/r/func_concat.result:
Fix for bug#44743: Join in combination with concat does not always work
bug#44766: valgrind error when using convert() in a subquery
- test result.
mysql-test/t/cast.test:
Fix for bug#44743: Join in combination with concat does not always work
bug#44766: valgrind error when using convert() in a subquery
- test case.
mysql-test/t/func_concat.test:
Fix for bug#44743: Join in combination with concat does not always work
bug#44766: valgrind error when using convert() in a subquery
- test case.
sql/item.cc:
Fix for bug#44743: Join in combination with concat does not always work
bug#44766: valgrind error when using convert() in a subquery
- comment added.
sql/item_strfunc.cc:
Fix for bug#44743: Join in combination with concat does not always work
bug#44766: valgrind error when using convert() in a subquery
- '&args[0]->str_value' used as a parameter of args[0]->val_str(),
as 'str' may be equal to 'str_value' which we use as the output buffer
converting strings.
sql/sql_string.cc:
Fix for bug#44743: Join in combination with concat does not always work
bug#44766: valgrind error when using convert() in a subquery
- input and output buffers must NOT be the same.
warnings after uncompressed_length
UNCOMPRESSED_LENGTH() did not validate its argument. In
particular, if the argument length was less than 4 bytes,
an uninitialized memory value was returned as a result.
Since the result of COMPRESS() is either an empty string or
a 4-byte length prefix followed by compressed data, the bug was
fixed by ensuring that the argument of UNCOMPRESSED_LENGTH() is
either an empty string or contains at least 5 bytes (as done in
UNCOMPRESS()). This is the best we can do to validate input
without decompressing.
mysql-test/r/func_compress.result:
Added a test case for bug #44796.
mysql-test/t/func_compress.test:
Added a test case for bug #44796.
sql/item_strfunc.cc:
Make sure that the argument of UNCOMPRESSED_LENGTH() contains
at least 5 bytes (as done in UNCOMPRESS()).
Problem: using LOAD_FILE() in some cases we pass a file name string
without a trailing '\0' to fn_format() which relies on that however.
That may lead to valgrind warnings.
Fix: add a trailing '\0' to the file name passed to fn_format().
mysql-test/r/func_str.result:
Fix for bug#44774: load_file function produces valgrind warnings
- test result.
mysql-test/t/func_str.test:
Fix for bug#44774: load_file function produces valgrind warnings
- test case.
sql/item_strfunc.cc:
Fix for bug#44774: load_file function produces valgrind warnings
- passing a file name to fn_format(), file_name->c_ptr() replaced
with file_name->c_ptr_safe() to ensure we have a trailing '\0'.
replaced String->c_ptr() with String->c_ptr_safe()
mysql-test/r/func_encrypt.result:
test result
mysql-test/t/func_encrypt.test:
test case
sql/item_strfunc.cc:
replaced String->c_ptr() with String->c_ptr_safe()
The warning happens because string argument is not zero ended.
The fix is to add new parameter 'length' to SQL_CRYPT() and
use ptr() instead of c_ptr().
mysql-test/r/func_str.result:
test result
mysql-test/t/func_str.test:
test case
sql/item_strfunc.cc:
Added new parameter 'length' to SQL_CRYPT
sql/sql_crypt.cc:
Added new parameter 'length' to SQL_CRYPT
sql/sql_crypt.h:
Added new parameter 'length' to SQL_CRYPT
to wrong results
3 problems found with DES_ENCRYPT/DES_DECRYPT :
1. The max length was not calculated properly. Fixed in fix_length_and_dec()
2. DES_ENCRYPT had a side effect of sometimes reallocating and changing
the value of its argument. Fixed by explicitly pre-allocating the necessary
space to pad the argument with trailing '*' (stars) when calculating the
DES digest.
3. in DES_ENCRYPT the string buffer for the result value was not
reallocated to the correct size and only string length was assigned to it.
Fixed by making sure there's enough space to hold the result.
Took the Xfree implementation (based on the same rewrite as the NDB one)
and added it instead of the current implementation.
Added a macro to make the calls to MD5 more streamlined.
client/mysqlmanager-pwgen.c:
Bug #42434: changed to call the macro
include/my_md5.h:
Bug #42434: use the Xfree implementation
mysys/md5.c:
Bug #42434: use the Xfree implementation
sql/item_strfunc.cc:
Bug #42434: changed to call the macro
sql/table.cc:
Bug #42434: changed to call the macro
tools/mysqlmanager.c:
Bug #42434: changed to call the macro
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
There were so many changes into mtr (this is the new mtr coming) that I rather
copied mtr from 6.0-main here (at least this one knows how to run Maria tests).
I also fixed suite/maria tests to be accepted by the new mtr.
mysys/thr_mutex.c:
adding DBUG_PRINT here, so that we can locate where the warning is issued.
