Commit graph

1798 commits

Author SHA1 Message Date
Marko Mäkelä
9c7a456a92 Merge 10.4 into 10.5 2021-06-01 10:38:09 +03:00
Marko Mäkelä
77d8da57d7 Merge 10.3 into 10.4 2021-06-01 09:14:59 +03:00
Marko Mäkelä
950a220060 Merge 10.2 into 10.3 2021-06-01 08:40:59 +03:00
Sergei Golubchik
d06205ba37 CONNECT: use my_snprintf 2021-05-27 17:36:32 +02:00
Marko Mäkelä
365cd08345 Merge 10.4 into 10.5 2021-05-26 09:47:28 +03:00
Marko Mäkelä
1dea7f7977 Merge 10.3 into 10.4 2021-05-25 15:38:57 +03:00
Olivier Bertrand
be7e41db96 - Mongo defined columns
modified:   storage/connect/cmgoconn.cpp
    modified:   storage/connect/cmgoconn.h

- Change default array LIMIT from 10 to 50
    modified:   storage/connect/ha_connect.cc
    modified:   storage/connect/tabbson.cpp
    modified:   storage/connect/tabjson.cpp
    modified:   storage/connect/tabxml.cpp
2021-05-24 16:56:12 +02:00
Marko Mäkelä
1864a8ea93 Merge 10.2 into 10.3 2021-05-24 09:38:49 +03:00
Monty
30c9089095 Fixed compiler warnings from CONNECT 2021-05-23 19:41:49 +03:00
Sergei Golubchik
d7321893d8 CONNECT: move jar files to /usr/share and include them in DEBs 2021-05-22 21:56:51 +02:00
Olivier Bertrand
e0edfc277f - Fix MDEV-25715
modified:   storage/connect/bsonudf.cpp
    modified:   storage/connect/jsonudf.cpp
2021-05-18 16:42:19 +02:00
Olivier Bertrand
17533c1ffc - Put all jar files in the SHARE directory (was PLUGIN)
modified:   sql/mysqld.h
    modified:   storage/connect/CMakeLists.txt
    modified:   storage/connect/javaconn.cpp
    modified:   storage/connect/mycat.cc
    modified:   storage/connect/mycat.h

- Get a handled on a not pooled client
  This to avoid a .50 delay when closing
    modified:   storage/connect/cmgoconn.cpp
    modified:   storage/connect/cmgoconn.h

- Allow FIELD_FORMAT options for DECIMAL type
    modified:   storage/connect/tabdos.cpp

- Update tests and result to reflect last changes
  Also because Oracle password has changed
    modified:   storage/connect/mysql-test/connect/r/jdbc_oracle.result
    modified:   storage/connect/mysql-test/connect/r/json_java_2.result
    modified:   storage/connect/mysql-test/connect/r/json_java_3.result
    modified:   storage/connect/mysql-test/connect/r/json_mongo_c.result
    modified:   storage/connect/mysql-test/connect/r/mongo_c.result
    modified:   storage/connect/mysql-test/connect/r/odbc_oracle.result
    modified:   storage/connect/mysql-test/connect/t/jdbc_oracle.test
    modified:   storage/connect/mysql-test/connect/t/odbc_oracle.test

- Typo
    modified:   storage/connect/reldef.cpp
2021-05-17 19:17:31 +02:00
Marko Mäkelä
0e1437e147 Merge 10.4 into 10.5 2021-05-10 10:01:15 +03:00
Marko Mäkelä
8c73fab7f7 Merge 10.3 into 10.4 2021-05-10 09:52:01 +03:00
Marko Mäkelä
98e6159892 Merge 10.2 into 10.3 2021-05-10 09:09:50 +03:00
Sergei Golubchik
ad4b51948f Revert "Connect: remove Mongo dependencies"
This reverts commit 72fa9dabad

but doesn't recover deleted jars - they still exist in
mysql-test/connect/std_data, no need to have them twice.

Also it removes a redundant copy on JavaWrappers.jar
2021-05-08 18:02:11 +02:00
Marko Mäkelä
76c2b5106e Fix clang++-11 -Wsometimes-uninitialized 2021-05-07 14:52:46 +03:00
Nikita Malyavin
3f55c56951 Merge branch bb-10.4-release into bb-10.5-release 2021-05-05 23:57:11 +03:00
Nikita Malyavin
509e4990af Merge branch bb-10.3-release into bb-10.4-release 2021-05-05 23:03:01 +03:00
Olivier Bertrand
5ae67c6d63 - All this concern Json or Mongo tables based on MongoDB collections.
- Limit decimals of doubles printed from MongoDB
  Done in function Mini for Mongo C Driver and Java Driver
  Done in function SerializeValue for Java tables using the J Driver
        modified:   storage/connect/cmgoconn.cpp
        modified:   storage/connect/json.cpp
        modified:   storage/connect/json.h
        modified:   storage/connect/tabjmg.cpp

- Fix crash when using BSON_TYPE_DECIMAL128
        modified:   storage/connect/cmgoconn.cpp

- Collection name default to table name
  Fix it when creating tables via discovery
        modified:   storage/connect/ha_connect.cc
        modified:   storage/connect/tabbson.cpp
        modified:   storage/connect/tabjson.cpp
2021-05-05 00:31:21 +02:00
Nikita Malyavin
a8a925dd22 Merge branch bb-10.2-release into bb-10.3-release 2021-05-04 14:49:31 +03:00
Nikita Malyavin
72fa9dabad Connect: remove Mongo dependencies 2021-05-03 19:02:05 +03:00
Olivier Bertrand
3b5dabeb96 Typo 2021-05-02 11:59:54 +02:00
Olivier Bertrand
2294f9de8d Fix compile warning as error 2021-05-02 11:03:21 +02:00
Olivier Bertrand
ef0829ef40 - Major update of the json/bson/mongo table types programs.
Fix several bugs, chiefly concerning CURL operations.
        modified:   storage/connect/bson.cpp
        modified:   storage/connect/cmgfam.cpp
        modified:   storage/connect/cmgoconn.cpp
        modified:   storage/connect/cmgoconn.h
        modified:   storage/connect/colblk.h
        modified:   storage/connect/ha_connect.cc
        modified:   storage/connect/jmgfam.cpp
        modified:   storage/connect/jmgoconn.cpp
        modified:   storage/connect/jmgoconn.h
        modified:   storage/connect/json.cpp
        modified:   storage/connect/json.h
        modified:   storage/connect/mysql-test/connect/r/bson_mongo_c.result
        modified:   storage/connect/mysql-test/connect/r/json_mongo_c.result
        modified:   storage/connect/mysql-test/connect/r/mongo_c.result
        modified:   storage/connect/mysql-test/connect/r/mongo_java_2.result
        modified:   storage/connect/mysql-test/connect/r/mongo_java_3.result
        modified:   storage/connect/mysql-test/connect/std_data/Mongo2.jar
        modified:   storage/connect/mysql-test/connect/std_data/Mongo3.jar
        modified:   storage/connect/tabbson.cpp
        modified:   storage/connect/tabbson.h
        modified:   storage/connect/tabcmg.cpp
        modified:   storage/connect/tabcmg.h
        modified:   storage/connect/tabjmg.cpp
        modified:   storage/connect/tabjmg.h
        modified:   storage/connect/tabjson.cpp
        modified:   storage/connect/tabjson.h
2021-05-01 22:29:38 +02:00
Marko Mäkelä
80ed136e6d Merge 10.4 into 10.5 2021-04-21 09:01:01 +03:00
Monty
031f11717d Fix all warnings given by UBSAN
The easiest way to compile and test the server with UBSAN is to run:
./BUILD/compile-pentium64-ubsan
and then run mysql-test-run.
After this commit, one should be able to run this without any UBSAN
warnings. There is still a few compiler warnings that should be fixed
at some point, but these do not expose any real bugs.

The 'special' cases where we disable, suppress or circumvent UBSAN are:
- ref10 source (as here we intentionally do some shifts that UBSAN
  complains about.
- x86 version of optimized int#korr() methods. UBSAN do not like unaligned
  memory access of integers.  Fixed by using byte_order_generic.h when
  compiling with UBSAN
- We use smaller thread stack with ASAN and UBSAN, which forced me to
  disable a few tests that prints the thread stack size.
- Verifying class types does not work for shared libraries. I added
  suppression in mysql-test-run.pl for this case.
- Added '#ifdef WITH_UBSAN' when using integer arithmetic where it is
  safe to have overflows (two cases, in item_func.cc).

Things fixed:
- Don't left shift signed values
  (byte_order_generic.h, mysqltest.c, item_sum.cc and many more)
- Don't assign not non existing values to enum variables.
- Ensure that bool and enum values are properly initialized in
  constructors.  This was needed as UBSAN checks that these types has
  correct values when one copies an object.
  (gcalc_tools.h, ha_partition.cc, item_sum.cc, partition_element.h ...)
- Ensure we do not called handler functions on unallocated objects or
  deleted objects.
  (events.cc, sql_acl.cc).
- Fixed bugs in Item_sp::Item_sp() where we did not call constructor
  on Query_arena object.
- Fixed several cast of objects to an incompatible class!
  (Item.cc, Item_buff.cc, item_timefunc.cc, opt_subselect.cc, sql_acl.cc,
   sql_select.cc ...)
- Ensure we do not do integer arithmetic that causes over or underflows.
  This includes also ++ and -- of integers.
  (Item_func.cc, Item_strfunc.cc, item_timefunc.cc, sql_base.cc ...)
- Added JSON_VALUE_UNITIALIZED to json_value_types and ensure that
  value_type is initialized to this instead of to -1, which is not a valid
  enum value for json_value_types.
- Ensure we do not call memcpy() when second argument could be null.
- Fixed that Item_func_str::make_empty_result() creates an empty string
  instead of a null string (safer as it ensures we do not do arithmetic
  on null strings).

Other things:

- Changed struct st_position to an OBJECT and added an initialization
  function to it to ensure that we do not copy or use uninitialized
  members. The change to a class was also motived that we used "struct
  st_position" and POSITION randomly trough the code which was
  confusing.
- Notably big rewrite in sql_acl.cc to avoid using deleted objects.
- Changed in sql_partition to use '^' instead of '-'. This is safe as
  the operator is either 0 or 0x8000000000000000ULL.
- Added check for select_nr < INT_MAX in JOIN::build_explain() to
  avoid bug when get_select() could return NULL.
- Reordered elements in POSITION for better alignment.
- Changed sql_test.cc::print_plan() to use pointers instead of objects.
- Fixed bug in find_set() where could could execute '1 << -1'.
- Added variable have_sanitizer, used by mtr.  (This variable was before
  only in 10.5 and up).  It can now have one of two values:
  ASAN or UBSAN.
- Moved ~Archive_share() from ha_archive.cc to ha_archive.h and marked
  it virtual. This was an effort to get UBSAN to work with loaded storage
  engines. I kept the change as the new place is better.
- Added in CONNECT engine COLBLK::SetName(), to get around a wrong cast
  in tabutil.cpp.
- Added HAVE_REPLICATION around usage of rgi_slave, to get embedded
  server to compile with UBSAN. (Patch from Marko).
- Added #ifdef for powerpc64 to avoid a bug in old gcc versions related
  to integer arithmetic.

Changes that should not be needed but had to be done to suppress warnings
from UBSAN:

- Added static_cast<<uint16_t>> around shift to get rid of a LOT of
  compiler warnings when using UBSAN.
- Had to change some '/' of 2 base integers to shift to get rid of
  some compile time warnings.

Reviewed by:
- Json changes: Alexey Botchkov
- Charset changes in ctype-uca.c: Alexander Barkov
- InnoDB changes & Embedded server: Marko Mäkelä
- sql_acl.cc changes: Vicențiu Ciorbaru
- build_explain() changes: Sergey Petrunia
2021-04-20 12:30:09 +03:00
Olivier Bertrand
4df6952ce3 Typo modified: CMakeLists.txt 2021-04-09 16:10:37 +02:00
Olivier Bertrand
36fd94aa5c Remove add_jar modified: CMakeLists.txt 2021-04-09 15:24:34 +02:00
Olivier Bertrand
46340ad8d1 Fix add_jar modified: CMakeLists.txt 2021-04-09 14:41:50 +02:00
Olivier Bertrand
03f929aea8 Fix install modified: CMakeLists.txt 2021-04-09 11:58:13 +02:00
Olivier Bertrand
bbec4aafda typo tabrest.cpp 2021-04-08 19:05:11 +02:00
Olivier Bertrand
710e1bac5a tabrest.cpp 2021-04-08 19:05:10 +02:00
Olivier Bertrand
757aaa3bcc tabrest.cpp 2021-04-08 19:05:09 +02:00
Olivier Bertrand
e4c23fe2d0 tabrest.cpp 2021-04-08 17:35:21 +02:00
Olivier Bertrand
953c84657b try to fix tabrest.cpp compile error 2021-04-08 17:35:19 +02:00
Olivier Bertrand
26e5ba4b51 - Fix Linux compile errors
modified:   storage/connect/tabrest.cpp
  modified:   storage/connect/CMakeLists.txt

- Fix cmake error
 modified:   libmariadb/cmake/ConnectorName.cmake
2021-04-08 17:35:17 +02:00
Olivier Bertrand
d1b6cad028 Test tabrest.cpp fix 2021-04-08 17:35:16 +02:00
Olivier Bertrand
c030f4c625 Commit last pull from origin 2021-04-06 12:52:44 +02:00
Olivier Bertrand
28b76afcea - Fix(?) Linux compile errors
modified:   storage/connect/tabrest.cpp
  modified:   storage/connect/CMakeLists.txt

-fix MDEV-24794
 modified:   storage/connect/valblk.h
2021-04-06 00:10:07 +02:00
Olivier Bertrand
2aefe0bee1 - Fix crash when not specifying the collection for MongoDB
modified:   storage/connect/cmgoconn.cpp

- Fix(?) Linux compile errors
  modified:   storage/connect/tabrest.cpp
2021-04-05 18:33:37 +02:00
Olivier Bertrand
caff19ada5 - Copy Mongo2.jar and Mongo3.jar in plugin directory
modified:   storage/connect/CMakeLists.txt
  modified:   storage/connect/javaconn.cpp

- Check privileges while creating tables with Discovery
  modified:   storage/connect/ha_connect.cc

- Calculate LRECL for JSON tables created with Discovery
  modified:   storage/connect/tabjson.cpp

- Use CreateProcess (Windows) or fork/exec (linux)
  to retrieve the result from REST queries
  modified:   storage/connect/tabrest.cpp

- Typo
  modified:   storage/connect/jmgoconn.cpp
2021-04-05 17:01:43 +02:00
Otto Kekäläinen
cebf9ee204 Fix various spelling errors still found in code
Reseting -> Resetting
Unknow -> Unknown
capabilites -> capabilities
choosen -> chosen
direcory -> directory
informations -> information
openned -> opened
refered -> referred
to access -> one to access
missmatch -> mismatch
succesfully -> successfully
dont -> don't
2021-03-22 18:10:39 +11:00
Olivier Bertrand
e85006e670 - Fix bug making REST table fail using CURL
This when the HTTP contains & characters
  modified:   storage/connect/tabbson.cpp
  modified:   storage/connect/tabjson.cpp

- Make stringfy option work on only one Json item
  modified:   storage/connect/tabbson.cpp
  modified:   storage/connect/tabbson.h
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h

- Make Json/Bson DATE columns accept JSON date syntax
  modified:   storage/connect/tabbson.cpp
  modified:   storage/connect/tabjson.cpp

- Fix bug making REST table default file not being
  erased when dropping the table
  modified:   storage/connect/tabbson.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabrest.cpp
  modified:   storage/connect/tabxml.cpp

- Suppress CHAR(36) --> VARCHAR(36) when DEVELOPMENT
  This was fixed in MyClient
  modified:   storage/connect/ha_connect.cc
2021-03-16 23:32:58 +01:00
David CARLIER
a6cd4612a9 connect storage, little compile issues fix proposal. 2021-03-11 09:36:23 +01:00
Monty
1799caa3a1 MDEV-24422 Server crashes in ha_connect::GetRealType upon ALTER TABLE
The problem was that the CONNECT engine is trying to open the .frm file
during drop_table(), which the code did not take into account.
Fixed by adding the HA_REUSES_FILE_NAMES table flag to CONNECT.

Other things:
- Fixed a wrong test of HA_REUSE_FILE_NAMES of in mysql_alter_table()
  (Comment was correct, no the code)
- Added a test in the connect engine that if the .frm it tries to use in
  delete is not made for connect, it will generate an error instead of
  crash.
2021-03-10 17:27:10 +02:00
Olivier Bertrand
801a6d500f - Add new JPATH features
modified:   storage/connect/bson.cpp
  modified:   storage/connect/bsonudf.cpp
  modified:   storage/connect/bsonudf.h
  modified:   storage/connect/json.cpp
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/jsonudf.h
  modified:   storage/connect/mysql-test/connect/r/json_udf.result
  modified:   storage/connect/tabbson.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h
2021-02-16 00:31:27 +01:00
Sergei Golubchik
25d9d2e37f Merge branch 'bb-10.4-release' into bb-10.5-release 2021-02-15 16:43:15 +01:00
Sergei Golubchik
00a313ecf3 Merge branch 'bb-10.3-release' into bb-10.4-release
Note, the fix for "MDEV-23328 Server hang due to Galera lock conflict resolution"
was null-merged. 10.4 version of the fix is coming up separately
2021-02-12 17:44:22 +01:00
Oleksandr Byelkin
6f93df1c2e Merge branch '10.2' into 10.3 2021-02-05 20:56:33 +01:00