Commit graph

1733 commits

Author SHA1 Message Date
Olivier Bertrand
3ad6c0ef8a Fix compile error. Modified ha_connect.cc 2020-11-07 16:33:01 +01:00
Olivier Bertrand
46edfd6338 - Getting text of json items now includes all array members
modified:   storage/connect/json.cpp
  modified:   storage/connect/json.h
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/jsonudf.h
  modified:   storage/connect/mysql-test/connect/r/json.result
  modified:   storage/connect/mysql-test/connect/r/json_java_2.result
  modified:   storage/connect/mysql-test/connect/r/json_mongo_c.result
  modified:   storage/connect/mysql-test/connect/r/json_udf.result
  modified:   storage/connect/mysql-test/connect/r/json_udf_bin.result
  modified:   storage/connect/mysql-test/connect/r/zip.result
  modified:   storage/connect/mysql-test/connect/t/zip.test
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h
  modified:   storage/connect/xobject.h
2020-11-07 15:40:46 +01:00
Olivier Bertrand
d3372258b0 Update tests to cope with changes 2020-11-06 15:34:13 +01:00
Olivier Bertrand
a4e999eec5 Try to fix failing tests 2020-11-05 23:04:37 +01:00
Olivier Bertrand
addb28f62d Try to fix failing tests 2020-11-05 22:14:01 +01:00
Olivier Bertrand
ecb00f3cd8 Try to fix failing tests 2020-11-05 19:13:26 +01:00
Olivier Bertrand
a13642a82f Try to fix that F..k gcc operator delete error 2020-11-04 16:33:10 +01:00
Olivier Bertrand
6a94ad98fb Fix crash on Json date columns 2020-11-04 15:46:02 +01:00
Olivier Bertrand
49428c8fa6 Fix compile error on LINUX (no suitable operator delete) 2020-11-04 11:36:29 +01:00
Olivier Bertrand
78ccc605a5 Fix compile error on LINUX (LARGE_INTEGER) 2020-11-03 23:19:22 +01:00
Olivier Bertrand
28af4212b6 - Implementation of the Json BJSON representation.
VAL structures replace VALUE classes in binary trees.
  These parsed binary trees are swapped and saved on file
  Swapping is to replace pointers by offsets to make it portable.
  In restoring, class pointers to functions are realloced on place.
  Making BJSON files is done by the new UDF function jfile_bjson.
  modified:   storage/connect/block.h
  modified:   storage/connect/filamtxt.cpp
  modified:   storage/connect/filamtxt.h
  modified:   storage/connect/global.h
  modified:   storage/connect/json.cpp
  modified:   storage/connect/json.h
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/jsonudf.h
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h
  modified:   storage/connect/value.h

- Make file (record) length and map memory possibly larger than MAX_INT
  modified:   storage/connect/filamap.cpp
  modified:   storage/connect/filamvct.cpp
  modified:   storage/connect/maputil.cpp
  modified:   storage/connect/maputil.h
  modified:   storage/connect/tabdos.cpp
  modified:   storage/connect/xindex.cpp

- Make column length as bytes (not characters)
  This when making column definitions
  modified:   storage/connect/ha_connect.cc

- Change the message when making index fails
  modified:   storage/connect/ha_connect.cc

- Update tests and results to reflect recent changes
  modified:   storage/connect/mysql-test/connect/r/alter_xml.result
  modified:   storage/connect/mysql-test/connect/r/alter_xml2.result
  modified:   storage/connect/mysql-test/connect/r/jdbc_oracle.result
  modified:   storage/connect/mysql-test/connect/r/json.result
  modified:   storage/connect/mysql-test/connect/r/json_java_2.result
  modified:   storage/connect/mysql-test/connect/r/json_java_3.result
  modified:   storage/connect/mysql-test/connect/r/json_mongo_c.result
  modified:   storage/connect/mysql-test/connect/r/mongo_c.result
  modified:   storage/connect/mysql-test/connect/r/mongo_java_2.result
  modified:   storage/connect/mysql-test/connect/r/mongo_java_3.result
  modified:   storage/connect/mysql-test/connect/r/odbc_oracle.result
  modified:   storage/connect/mysql-test/connect/r/xml.result
  modified:   storage/connect/mysql-test/connect/r/xml2.result
  modified:   storage/connect/mysql-test/connect/r/xml2_html.result
  modified:   storage/connect/mysql-test/connect/r/xml2_mult.result
  modified:   storage/connect/mysql-test/connect/r/xml2_zip.result
  modified:   storage/connect/mysql-test/connect/r/xml_html.result
  modified:   storage/connect/mysql-test/connect/r/xml_mult.result
  modified:   storage/connect/mysql-test/connect/r/xml_zip.result
  modified:   storage/connect/mysql-test/connect/t/alter_xml.test
  modified:   storage/connect/mysql-test/connect/t/alter_xml2.test
  modified:   storage/connect/mysql-test/connect/t/jdbc_oracle.test
  modified:   storage/connect/mysql-test/connect/t/json.test
  modified:   storage/connect/mysql-test/connect/t/mongo_test.inc
  modified:   storage/connect/mysql-test/connect/t/odbc_oracle.test
  modified:   storage/connect/mysql-test/connect/t/xml.test
  modified:   storage/connect/mysql-test/connect/t/xml2.test
  modified:   storage/connect/mysql-test/connect/t/xml2_html.test
  modified:   storage/connect/mysql-test/connect/t/xml2_mult.test
  modified:   storage/connect/mysql-test/connect/t/xml2_zip.test
  modified:   storage/connect/mysql-test/connect/t/xml_html.test
  modified:   storage/connect/mysql-test/connect/t/xml_mult.test
  modified:   storage/connect/mysql-test/connect/t/xml_zip.test

- Typo
  modified:   storage/connect/value.cpp
2020-11-03 18:40:28 +01:00
Marko Mäkelä
533a13af06 Merge 10.3 into 10.4 2020-11-03 14:49:17 +02:00
Oleksandr Byelkin
8e1e2856f2 Merge branch '10.4' into 10.5 2020-11-01 14:26:15 +01:00
Oleksandr Byelkin
80c951ce28 Merge branch '10.3' into 10.4 2020-10-31 21:06:49 +01:00
Oleksandr Byelkin
794f665139 Merge branch '10.2' into 10.3 2020-10-30 17:23:53 +01:00
Marko Mäkelä
898521e2dd Merge 10.4 into 10.5 2020-10-30 11:15:30 +02:00
Marko Mäkelä
7b2bb67113 Merge 10.3 into 10.4 2020-10-29 13:38:38 +02:00
Oleksandr Byelkin
65e26bc1ba Merge branch '10.1' into 10.2 2020-10-28 10:56:38 +01:00
Marko Mäkelä
a8de8f261d Merge 10.2 into 10.3 2020-10-28 10:01:50 +02:00
Oleksandr Byelkin
784473b986 Merge remote-tracking branch 'connect/10.2' into 10.2 2020-10-26 09:08:44 +01:00
Monty
21c15cc20b Fixed compiler warning in connect/filemazip.cpp 2020-10-21 03:09:29 +03:00
Olivier Bertrand
dc3a693b70 - Inline MakePtr and MakeOff with OFFSET as size_t
Also add a new member Saved_Size in the Global structure.
  modified:   storage/connect/global.h
  modified:   storage/connect/plugutil.cpp
  modified:   storage/connect/user_connect.cc
  modified:   storage/connect/jsonudf.cpp

- Add session variables json_all_path and default_depth
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/mongo.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabxml.cpp

- ADD column options JPATH and XPATH
  Work as FIELD_FORMAT but are more readable
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h
  modified:   storage/connect/mysql-test/connect/r/json_java_2.result
  modified:   storage/connect/mysql-test/connect/r/json_java_3.result
  modified:   storage/connect/mysql-test/connect/r/json_mongo_c.result

- Handle negative numbes in the option list
  modified:   storage/connect/ha_connect.cc

- Fix Json parse that could crash the server.
  Was because it could use THROW out of the TRY block.
  Also handle all error by THROW.
  It is now done by a new class JSON.
  modified:   storage/connect/json.cpp
  modified:   storage/connect/json.h

- Add a new UDF function jfile_translate.
  It translate a Json file to pretty = 0.
  Fast because it does not a real parse of the file.
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/jsonudf.h

- Add a now options JSIZE and STRINGIFY to Json tables.
  STRINGIFY makes Objects or Arrays to be returned by their
  json representation instead of by their concatenated values.
  JSIZE allows to specify the LRECL (was 256) defaults to 1024.
  Also fix a bug about locating the sub-table by its path.
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h
2020-10-18 17:20:44 +02:00
Olivier Bertrand
5d2ddef26e Fix search for json subtable in tabjson.cpp 2020-10-06 12:50:12 +02:00
Olivier Bertrand
d4138e7eed Fix compile error in tabjson.cpp (ULONG -> ulong) 2020-10-05 15:21:58 +02:00
Olivier Bertrand
307258c8ee - Use BIN type when charset='binary'
modified:   storage/connect/ha_connect.cc

- Allow JSON columns to be "binary"
  By setting their type as VARBINAY(132)
  and their name begin with Jbin_
  modified:   storage/connect/json.h
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/value.cpp
  modified:   storage/connect/value.h

- CHARSET BINARY cannot be used for text columns
  modified:   storage/connect/mysql-test/connect/r/updelx.result
  modified:   storage/connect/mysql-test/connect/t/updelx.test
2020-10-05 12:29:51 +02:00
Olivier Bertrand
c6eb127ca8 - Fix allocating work space larger than 4GB
The variable connect_work_size is now ulong or ulonglong for 64bit machines.
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/user_connect.cc
2020-10-03 19:06:05 +02:00
Olivier Bertrand
99ab562a92 - Make possible to allocate work space larger than 4GB
All variables handling sizes that were uint are now size_t.
  The variable connect_work_size is now ulong (was uint);
  Also make Json functiosn to allocate a larger memory (M=9 was 7)
  modified:   storage/connect/global.h
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/json.cpp
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/plgdbutl.cpp
  modified:   storage/connect/plugutil.cpp
  modified:   storage/connect/user_connect.cc

- Fix uninitialised variable (pretty) in Json_File.
  Make Jbin_file accept the same arguments as Json_File ones.
  modified:   storage/connect/jsonudf.cpp

- Change the Level option to Depth (the word currently used)
  (Level being still accepted)
  modified:   storage/connect/mongo.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabxml.cpp

- Suppress 2nd argument default value for MYSQLtoPLG function
  modified:   storage/connect/myutil.h

- Allow REST tables to be create not specifying a file_name
  modified:   storage/connect/tabrest.cpp
2020-10-01 19:18:26 +02:00
Michael Widenius
58e759a939 Added 'final' to some classes to improve generated code
Final added to:
- All reasonable classes inhereted from Field
- All classes inhereted from Protocol
- Almost all Handler classes
- Some important Item classes

The stripped size of mariadbd is just 4K smaller, but several object files
showed notable improvements in common execution paths.
- Checked field.o and item_sum.o

Other things:
- Added 'override' to a few class functions touched by this patch.
- Removed 'virtual' from a new class functions that had/got 'override'
- Changed Protocol_discard to inherit from Protocol instad of Protocol_text
2020-08-04 17:27:32 +02:00
Oleksandr Byelkin
48b5777ebd Merge branch '10.4' into 10.5 2020-08-04 17:24:15 +02:00
Oleksandr Byelkin
57325e4706 Merge branch '10.3' into 10.4 2020-08-03 14:44:06 +02:00
Oleksandr Byelkin
c32f71af7e Merge branch '10.2' into 10.3 2020-08-03 13:41:29 +02:00
Oleksandr Byelkin
6d8af36bc7 Merge remote-tracking branch 'connect/10.2' into 10.2 2020-08-02 11:14:56 +02:00
Olivier Bertrand
671d9b6c61 - Fix MDEV-22571 and MDEV-22572. Allow multiple ZIP table
and enable using special column in them.
  modified:   storage/connect/tabzip.cpp
  modified:   storage/connect/tabzip.h

- Fix some compiler errors
  modified:   storage/connect/tabcmg.cpp
2020-07-16 16:30:54 +02:00
Olivier Bertrand
ad0d2424dd Fix compile error for KVM. Modified filamdbf.cpp 2020-07-13 20:02:51 +02:00
Olivier Bertrand
1e07df99f3 - Fix MDEV-22561 Unable to access DBF inside a ZIP archive
modified:   storage/connect/filamdbf.cpp
  modified:   storage/connect/filamdbf.h
  modified:   storage/connect/filamzip.cpp
  modified:   storage/connect/filamzip.h
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/plgxml.cpp
  modified:   storage/connect/tabdos.cpp
  modified:   storage/connect/tabdos.h
  modified:   storage/connect/tabfix.h

- Add/Init Level class member
  modified:   storage/connect/mongo.cpp
  modified:   storage/connect/mongo.h
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h

- Typo
  modified:   storage/connect/connect.cc
2020-07-13 16:30:57 +02:00
Monty
5bcb1d6532 MDEV-11412 Ensure that table is truly dropped when using DROP TABLE
The used code is largely based on code from Tencent

The problem is that in some rare cases there may be a conflict between .frm
files and the files in the storage engine. In this case the DROP TABLE
was not able to properly drop the table.

Some MariaDB/MySQL forks has solved this by adding a FORCE option to
DROP TABLE. After some discussion among MariaDB developers, we concluded
that users expects that DROP TABLE should always work, even if the
table would not be consistent. There should not be a need to use a
separate keyword to ensure that the table is really deleted.

The used solution is:
- If a .frm table doesn't exists, try dropping the table from all storage
  engines.
- If the .frm table exists but the table does not exist in the engine
  try dropping the table from all storage engines.
- Update storage engines using many table files (.CVS, MyISAM, Aria) to
  succeed with the drop even if some of the files are missing.
- Add HTON_AUTOMATIC_DELETE_TABLE to handlerton's where delete_table()
  is not needed and always succeed. This is used by ha_delete_table_force()
  to know which handlers to ignore when trying to drop a table without
  a .frm file.

The disadvantage of this solution is that a DROP TABLE on a non existing
table will be a bit slower as we have to ask all active storage engines
if they know anything about the table.

Other things:
- Added a new flag MY_IGNORE_ENOENT to my_delete() to not give an error
  if the file doesn't exist. This simplifies some of the code.
- Don't clear thd->error in ha_delete_table() if there was an active
  error. This is a bug fix.
- handler::delete_table() will not abort if first file doesn't exists.
  This is bug fix to handle the case when a drop table was aborted in
  the middle.
- Cleaned up mysql_rm_table_no_locks() to ensure that if_exists uses
  same code path as when it's not used.
- Use non_existing_Table_error() to detect if table didn't exists.
  Old code used different errors tests in different position.
- Table_triggers_list::drop_all_triggers() now drops trigger file if
  it can't be parsed instead of leaving it hanging around (bug fix)
- InnoDB doesn't anymore print error about .frm file out of sync with
  InnoDB directory if .frm file does not exists. This change was required
  to be able to try to drop an InnoDB file when .frm doesn't exists.
- Fixed bug in mi_delete_table() where the .MYD file would not be dropped
  if the .MYI file didn't exists.
- Fixed memory leak in Mroonga when deleting non existing table
- Fixed memory leak in Connect when deleting non existing table

Bugs fixed introduced by the original version of this commit:
MDEV-22826 Presence of Spider prevents tables from being force-deleted from
           other engines
2020-06-14 19:39:42 +03:00
Marko Mäkelä
701efbb25b Merge 10.4 into 10.5 2020-06-03 09:45:39 +03:00
Marko Mäkelä
8059148154 Merge 10.3 into 10.4 2020-06-03 07:32:09 +03:00
Marko Mäkelä
8300f639a1 Merge 10.2 into 10.3 2020-06-02 10:25:11 +03:00
Alexander Barkov
6a6aa1c089 MDEV-21764 CONNECT table with INET6 field produces warnings upon SELECT 2020-06-01 11:46:30 +04:00
Sergei Golubchik
4832b751ad cmake: quieter 2020-05-31 11:29:43 +02:00
Monty
4ea171ffab Fixed compiler warnings from gcc and clang 5.0.1 2020-05-23 12:29:10 +03:00
Olivier Bertrand
2654ddd2d5 - Fix MDEV-22571 and MDEV-22572. Allow multiple ZIP table
and enable using special column in them.
  modified:   storage/connect/tabzip.cpp
  modified:   storage/connect/tabzip.h

- Fix some compiler errors
  modified:   storage/connect/tabcmg.cpp
2020-05-19 00:05:56 +02:00
Olivier Bertrand
e3163524ea - Fix MDEV-22571 and MDEV-22572. Allow multiple ZIP table
and enable using special column in them.
  modified:   storage/connect/tabzip.cpp
  modified:   storage/connect/tabzip.h
2020-05-18 23:52:33 +02:00
Marko Mäkelä
7bcaa541aa Merge 10.4 into 10.5 2020-05-05 21:16:22 +03:00
Marko Mäkelä
2c3c851d2c Merge 10.3 into 10.4 2020-05-05 20:33:10 +03:00
Oleksandr Byelkin
7fb73ed143 Merge branch '10.2' into 10.3 2020-05-04 16:47:11 +02:00
Oleksandr Byelkin
ca091e6372 Merge branch '10.1' into 10.2 2020-05-02 08:44:17 +02:00
Marko Mäkelä
37c14690fc Merge 10.4 into 10.5 2020-03-30 19:07:25 +03:00
Marko Mäkelä
e2f1f88fa6 Merge 10.3 into 10.4 2020-03-30 14:50:23 +03:00