Commit graph

31 commits

Author SHA1 Message Date
Olivier Bertrand
e6b563f8be Fix some XML table type bugs:
- in DOMNODELIST::DropItem
  if (Listp == NULL || Listp->length <= n)
    return true;
is wrong, should be:
  if (Listp == NULL || Listp->length < n)
    return true;
- Crash in discovery with libxml2 in XMLColumns because:
            if (!tdp->Usedom)    // nl was destroyed
              vp->nl = vp->pn->GetChildElements(g);
is executed with vp->pn uninitialized. Fixed by adding:
          vp->pn = node;
line 264.
-In discovery with libxml2 some columns are not found.
Because list was not recovered properly, nodes being modified and not reallocated.
Fixed lines 214 and 277.
  modified:   storage/connect/domdoc.cpp
  modified:   storage/connect/tabxml.cpp

Add support for zipped table files
  modified:   storage/connect/domdoc.cpp
  modified:   storage/connect/domdoc.h
  modified:   storage/connect/filamap.cpp
  modified:   storage/connect/filamap.h
  modified:   storage/connect/filamzip.cpp
  modified:   storage/connect/filamzip.h
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/libdoc.cpp
  modified:   storage/connect/plgdbutl.cpp
  modified:   storage/connect/plgxml.cpp
  modified:   storage/connect/plgxml.h
  modified:   storage/connect/tabdos.cpp
  modified:   storage/connect/tabdos.h
  modified:   storage/connect/tabfmt.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabxml.cpp
2016-12-23 16:58:32 +01:00
Olivier Bertrand
952306502e - MDEV-11295: developing handling files contained in ZIP file.
Enable using multiple zip files
  modified:   storage/connect/filamzip.cpp
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h
  modified:   storage/connect/mycat.h
  modified:   storage/connect/tabdos.cpp
  modified:   storage/connect/tabdos.h
  modified:   storage/connect/tabfmt.cpp
  modified:   storage/connect/tabjson.cpp
2016-12-14 14:20:23 +01:00
Olivier Bertrand
d44723e621 - MDEV-11295: developing handling files contained in ZIP file.
A first experimental and limited implementation.
  modified:   storage/connect/CMakeLists.txt
  modified:   storage/connect/filamap.cpp
  new file:   storage/connect/filamzip.cpp
  new file:   storage/connect/filamzip.h
  modified:   storage/connect/ha_connect.cc
  new file:   storage/connect/ioapi.c
  new file:   storage/connect/ioapi.h
  modified:   storage/connect/mycat.cc
  modified:   storage/connect/plgdbsem.h
  modified:   storage/connect/plgdbutl.cpp
  modified:   storage/connect/tabdos.cpp
  modified:   storage/connect/tabdos.h
  modified:   storage/connect/tabfmt.cpp
  modified:   storage/connect/tabfmt.h
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h
  new file:   storage/connect/tabzip.cpp
  new file:   storage/connect/tabzip.h
  new file:   storage/connect/unzip.c
  new file:   storage/connect/unzip.h
  new file:   storage/connect/zip.c
2016-12-12 10:57:19 +01:00
Olivier Bertrand
599d8cc2de - MDEV-11366 SIGBUS errors in Connect Storage Engine for ArmHF and MIPS.
Fix includes launchpad fix plus more to cover writing BIN tables.
  modified:   storage/connect/tabfix.cpp
  modified:   storage/connect/value.cpp
  modified:   storage/connect/value.h

- Typo: Change the name of filamzip to filamgz to prepare future ZIP tables.
  modified:   storage/connect/CMakeLists.txt
  added:      storage/connect/filamgz.cpp
  added:      storage/connect/filamgz.h
  deleted:    storage/connect/filamzip.cpp
  deleted:    storage/connect/filamzip.h
  modified:   storage/connect/plgdbsem.h
  modified:   storage/connect/reldef.cpp
  modified:   storage/connect/tabdos.cpp
  modified:   storage/connect/tabdos.h
  modified:   storage/connect/tabfix.cpp
  modified:   storage/connect/tabfmt.cpp
  modified:   storage/connect/tabjson.cpp
2016-12-02 23:03:43 +01:00
Olivier Bertrand
8286b680b2 - Copy error message from G to g when using temporary storage for parsing.
modified:   storage/connect/tabjson.cpp
2015-12-13 18:39:32 +01:00
Olivier Bertrand
8ba013a258 - Serialize: Protect again eventual longjmp's.
Always return NULL on error.
Adding also the file length.
  modified:   storage/connect/json.cpp
  modified:   storage/connect/jsonudf.cpp

- JSONCOL::WriteColumn Add types SHORT and BIGINT as accepted
  modified:   storage/connect/tabjson.cpp

- TDBJSN: Make this type use a separate storage for Json parsing
and retrieve this memory between each rows. This is necessary
to be able to handle big tables. See MDEV-9228.
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h
2015-12-08 16:39:13 +01:00
Olivier Bertrand
27f9d2f929 Commit updating CONNECT from the 10.1 version 2015-12-04 22:38:16 +01:00
Olivier Bertrand
335ec7a363 Prevent wrong update of expanded columns when pretty is not 2.
modified:   storage/connect/tabjson.cpp
2015-08-11 21:15:33 +02:00
Olivier Bertrand
b6a56370d6 Change all preprocessor compiler directives to use __WIN__ as the mean of
specifying Windows or not Windows compile. This is what MariaDB does.
  modified:   storage/connect/array.cpp
  modified:   storage/connect/blkfil.cpp
  modified:   storage/connect/block.h
  modified:   storage/connect/colblk.cpp
  modified:   storage/connect/domdoc.cpp
  modified:   storage/connect/filamap.cpp
  modified:   storage/connect/filamdbf.cpp
  modified:   storage/connect/filamfix.cpp
  modified:   storage/connect/filamtxt.cpp
  modified:   storage/connect/filamvct.cpp
  modified:   storage/connect/filamzip.cpp
  modified:   storage/connect/filter.cpp
  modified:   storage/connect/filter.h
  modified:   storage/connect/fmdlex.c
  modified:   storage/connect/global.h
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/json.cpp
  modified:   storage/connect/macutil.cpp
  modified:   storage/connect/macutil.h
  modified:   storage/connect/maputil.cpp
  modified:   storage/connect/mycat.cc
  modified:   storage/connect/myconn.cpp
  modified:   storage/connect/myconn.h
  modified:   storage/connect/myutil.cpp
  modified:   storage/connect/odbconn.cpp
  modified:   storage/connect/odbconn.h
  modified:   storage/connect/os.h
  modified:   storage/connect/osutil.c
  modified:   storage/connect/plgdbsem.h
  modified:   storage/connect/plgdbutl.cpp
  modified:   storage/connect/plugutil.c
  modified:   storage/connect/rcmsg.c
  modified:   storage/connect/reldef.cpp
  modified:   storage/connect/reldef.h
  modified:   storage/connect/tabdos.cpp
  modified:   storage/connect/tabfix.cpp
  modified:   storage/connect/tabfmt.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabmac.cpp
  modified:   storage/connect/tabmac.h
  modified:   storage/connect/tabmul.cpp
  modified:   storage/connect/tabmul.h
  modified:   storage/connect/tabmysql.cpp
  modified:   storage/connect/taboccur.cpp
  modified:   storage/connect/tabodbc.cpp
  modified:   storage/connect/tabpivot.cpp
  modified:   storage/connect/tabsys.cpp
  modified:   storage/connect/tabtbl.cpp
  modified:   storage/connect/tabutil.cpp
  modified:   storage/connect/tabvct.cpp
  modified:   storage/connect/tabwmi.cpp
  modified:   storage/connect/tabxcl.cpp
  modified:   storage/connect/tabxml.cpp
  modified:   storage/connect/valblk.cpp
  modified:   storage/connect/value.cpp
  modified:   storage/connect/xindex.cpp
  modified:   storage/connect/xindex.h
2015-05-27 16:23:38 +02:00
Olivier Bertrand
fb98632930 JSONColumns and XMLColumns revisited. They can retrieve their parameters directly
from the PTOS argument. For this to work, finding the table options is now split
in HA_CONNECT functions and exported functions available from out of ha_connect.
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/libdoc.cpp
  modified:   storage/connect/mycat.h
  modified:   storage/connect/plgdbsem.h
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h
  modified:   storage/connect/tabxml.cpp
  modified:   storage/connect/tabxml.h

The BIN table formats have been changed to handle the case of floating point values
when used with Big Endian or Little Endian machines.
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/mysql-test/connect/r/bin.result
  modified:   storage/connect/mysql-test/connect/t/bin.test
  modified:   storage/connect/reldef.cpp
  modified:   storage/connect/tabdos.cpp
  modified:   storage/connect/tabdos.h
  modified:   storage/connect/tabfix.cpp
  modified:   storage/connect/tabfix.
h
2015-05-26 01:02:33 +02:00
Olivier Bertrand
37840d5313 Security: EOM modules must now be loaded from the plugin directory.
modified:   storage/connect/mycat.cc
  modified:   storage/connect/reldef.cpp

Json array index (position) always defaults to 0
  modified:   storage/connect/tabjson.cpp
2015-05-20 11:19:44 +02:00
Olivier Bertrand
db33294fdc Json array index (position) was badly set for default array setting
modified:   storage/connect/tabjson.cpp
2015-05-17 19:55:48 +02:00
Olivier Bertrand
a82171c9aa In BIN table date_format now imply by default field_format='C'.
modified:   storage/connect/tabfix.cpp
  modified:   storage/connect/reldef.cpp

Json array index (position) is now 0 based by default. This corresponds
to what all json applications and functions do. Also fix ROWNUM calculation.
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/mysql-test/connect/r/json.result
  modified:   storage/connect/mysql-test/connect/t/json.test
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h
2015-05-17 15:22:42 +02:00
Olivier Bertrand
f5d0c77062 Get rid of GCC warnings about unused parameters
modified:   storage/connect/array.cpp
  modified:   storage/connect/blkfil.cpp
  modified:   storage/connect/block.h
  modified:   storage/connect/catalog.h
  modified:   storage/connect/colblk.cpp
  modified:   storage/connect/colblk.h
  modified:   storage/connect/connect.cc
  modified:   storage/connect/filamap.cpp
  modified:   storage/connect/filamdbf.cpp
  modified:   storage/connect/filamfix.cpp
  modified:   storage/connect/filamtxt.cpp
  modified:   storage/connect/filamtxt.h
  modified:   storage/connect/filamvct.cpp
  modified:   storage/connect/filamzip.cpp
  modified:   storage/connect/filter.h
  modified:   storage/connect/ha_connect.c
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/mycat.h
  modified:   storage/connect/myconn.cpp
  modified:   storage/connect/plgdbutl.cpp
  modified:   storage/connect/reldef.cpp
  modified:   storage/connect/reldef.h
  modified:   storage/connect/tabcol.cpp
  modified:   storage/connect/tabdos.cpp
  modified:   storage/connect/tabdos.h
  modified:   storage/connect/tabfix.cpp
  modified:   storage/connect/tabfmt.cpp
  modified:   storage/connect/tabfmt.h
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h
  modified:   storage/connect/table.cpp
  modified:   storage/connect/tabmul.cpp
  modified:   storage/connect/tabmysql.cpp
  modified:   storage/connect/tabmysql.h
  modified:   storage/connect/taboccur.cpp
  modified:   storage/connect/tabpivot.cpp
  modified:   storage/connect/tabsys.cpp
  modified:   storage/connect/tabtbl.cpp
  modified:   storage/connect/tabtbl.h
  modified:   storage/connect/tabutil.cpp
  modified:   storage/connect/tabutil.h
  modified:   storage/connect/tabvct.cpp
  modified:   storage/connect/tabvir.cpp
  modified:   storage/connect/tabvir.h
  modified:   storage/connect/tabxcl.cpp
  modified:   storage/connect/tabxcl.h
  modified:   storage/connect/tabxml.cpp
  modified:   storage/connect/tabxml.h
  modified:   storage/connect/valblk.cpp
  modified:   storage/connect/valblk.h
  modified:   storage/connect/value.cpp
  modified:   storage/connect/value.h
  modified:   storage/connect/xindex.cpp
  modified:   storage/connect/xindex.h
  modified:   storage/connect/xobject.h
  modified:   storage/connect/xtable.h
2015-05-09 17:30:20 +02:00
Olivier Bertrand
f5b05a11c1 - Fix bug on updating JSON expanded values
modified:
  json.result
  tabjson.cpp
  tabjson.h

- Fix bug on multiple tables (directories must be eliminated from file list)
modified:
  tabmul.cpp

- Update version
modified:
  ha_connect.cc

- Typo
modified:
  global.h
  ha_connect.cc
  tabtbl.cpp
  tabutil.cpp
  value.cpp
2015-05-01 15:59:12 +02:00
Olivier Bertrand
eae8318b19 - Fix Catalog JSON table crash when no Jpath
- Added JSON OBJECT specification for pretty != 2.
- Fix NULL values not recognized for nullable JSON columns
- Issue an error message when a JSON table is created without specifying LRECL if PRETTY != 2.
- Make JSONColumns use a TDBJSON class.
- Make JSON table using MAPFAM
modified:
  filamap.h
  filamtxt.h
  ha_connect.cc
  json.result
  tabjson.cpp
  tabjson.h
  table.cpp

- Implementing Discovery for the XML table type.
modified:
  domdoc.cpp
  domdoc.h
  ha_connect.cc
  libdoc.cpp
  plgxml.cpp
  plgxml.h
  reldef.cpp
  reldef.h
  tabxml.cpp
  tabxml.h

- Providing an error message when creating an ODBC table via discovery returns
  columns of more than one table.
modified:
  ha_connect.cc

- TableOptionStruct declaration moved from ha_connect.h to mycat.h
  To make it easier to use by other classes.
modified:
  ha_connect.cc
  ha_connect.h
  mycat.cc
  mycat.h
  reldef.cpp
  tabmysql.cpp
  taboccur.cpp
  tabpivot.cpp
  tabtbl.cpp
  tabutil.cpp
  tabxcl.cpp
2015-04-17 20:05:41 +02:00
Olivier Bertrand
9cace9932f - Fix a bug that caused a crash when doing delete on a json table with wrong syntax file
- Suppress MYSQL_SUPPORT preprocessor variable
2015-03-22 11:34:29 +01:00
Olivier Bertrand
2bb42803df This commit includes changes done in a previous (deleted) branch plus new ones.
From the previous branch:
commit eda4928ff122a0845baf5ade83b4aa29244a3a89
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Mon Mar 9 22:34:56 2015 +0100

- Add discovery to JSON tables
  When columns are not defined, CONNECT analyses the json file to find column definitions.
  This wors only on table that are an array of objects. Pair keys are used to generate the
  column names and pair values are used for its definition. When the LEVEL option is defined
  as a not null integer, the eventual JPATH is scanned up to the LEVEL value.

From the current one:
- Fix MDEV-7521 when column names are utf8 encoded (not a general multi-charset fix)

- Adds more to JSON discovery processing and UDF's

- Use PlugDup everywhere it replaces PlugSubAlloc + strcpy.
2015-03-18 13:30:14 +01:00
Olivier Bertrand
d862d7c049 - Implement random access to ODBC tables
modified:
  storage/connect/odbconn.cpp
  storage/connect/odbconn.h

- Fix get proper length of ODBC DECIMAL column in discovery
modified:
  storage/connect/ha_connect.cc
  storage/connect/mysql-test/connect/r/odbc_oracle.result

- Implement random access to JSON tables
modified:
  storage/connect/tabjson.cpp
  storage/connect/tabjson.h

- Fix MDEV-7636
modified:
  storage/connect/tabutil.cpp
2015-02-28 23:01:55 +01:00
Olivier Bertrand
aa107ef3ab - FIX assert failure when sorting JSON tables
modified:
  storage/connect/tabjson.cpp
  storage/connect/tabjson.h
2015-02-25 11:59:00 +01:00
Olivier Bertrand
e027f5e8d5 - Fix MDEV-7616 by adding SQLCOM_SET_OPTION to the accepted command list.
modified:
  storage/connect/ha_connect.cc

- Add new JSON UDF functions and JSON functionalities.
modified:
  storage/connect/json.cpp
  storage/connect/json.h
  storage/connect/jsonudf.cpp
  storage/connect/tabjson.cpp
2015-02-24 23:18:04 +01:00
Olivier Bertrand
a736e63f7c - Add new Json UDF's Json_Array_Add, Json_Array_Grp and Json_Object_Grp.
Handle longjmp's raised during json processing.
modified:
  storage/connect/global.h
  storage/connect/ha_connect.cc
  storage/connect/json.cpp
  storage/connect/jsonudf.cpp

- Fix wrong references to the suppressed g->Trace variables.
modified:
  storage/connect/global.h
  storage/connect/plugutil.c
  storage/connect/tabjson.cpp
  storage/connect/tabodbc.cpp
2015-02-22 17:53:02 +01:00
Olivier Bertrand
564d41faf4 - Work on JSON and JSON UDF's
modified:
  storage/connect/json.cpp
  storage/connect/jsonudf.cpp
  storage/connect/tabjson.cpp

- CntReadNext: Enable EvalColumns for longjmp
modified:
  storage/connect/connect.cc
2015-02-19 00:59:02 +01:00
Olivier Bertrand
dcfe068d59 - Adding json udf's. Making the second version of json tables.
added:
  storage/connect/jsonudf.cpp
modified:
  storage/connect/CMakeLists.txt
  storage/connect/json.cpp
  storage/connect/json.h
  storage/connect/tabjson.cpp
  storage/connect/tabjson.h

- Fix utf8 issues with PROXY tables
modified:
  storage/connect/ha_connect.cc
  storage/connect/tabutil.cpp
  storage/connect/tabutil.h
2015-02-11 21:39:41 +01:00
Olivier Bertrand
180b2be63c - Add the possibility to establish an ODBC connection via SQLConnect (the
default being still to use SQLDriverConnect)
modified:
  storage/connect/ha_connect.cc
  storage/connect/odbccat.h
  storage/connect/odbconn.cpp
  storage/connect/odbconn.h
  storage/connect/tabodbc.cpp
  storage/connect/tabodbc.h

- Cardinality testing not possible for TBL tables
modified:
  storage/connect/tabtbl.cpp

- Make all line ending LF instead of CRLF
modified:
  storage/connect/tabjson.cpp
2015-01-31 15:05:43 +01:00
Olivier Bertrand
fd1ca70bb3 - Enhance JSON tables handling.
modified:
  storage/connect/json.cpp
  storage/connect/json.h
  storage/connect/mysql-test/connect/r/json.result
  storage/connect/mysql-test/connect/t/json.test
  storage/connect/tabjson.cpp
  storage/connect/tabjson.h

- Avoid crash when a partition table name pattern is ill formed
  (such as using place holder %i instead of %s)
modified:
  storage/connect/ha_connect.cc
2015-01-30 10:57:00 +01:00
Olivier Bertrand
dbf690b5d1 - Fix correctly MDEV-7498. The problem was not in AllocateValue but in
the constructor of TYPVAL<PSZ> The new strg was widely uninitialised
  just by setting a terminated null char. Now the whole string is zero'ed.
  In Tabjson JSONCOL::ParseJpath, the original code was restored. 
modified:
  storage/connect/tabjson.cpp
  storage/connect/value.cpp
2015-01-24 11:46:18 +01:00
Olivier Bertrand
2cd36ad7d0 - This to fix MDEV-7498. All changes made to AllocateValue to be sure that
the sp and p variable be initialised failed. Not understanding what causes
  this valgrind warning, I finally changed the way Mulval is allocated just
  to avoid it. This is a BAD solution as it does not really fix the problem
  but just hide it. This will have to be re-considered.
modified:
  storage/connect/tabjson.cpp
  storage/connect/value.cpp
2015-01-23 21:54:29 +01:00
Olivier Bertrand
dc091a2935 - Fix MDEV-7489 (in add_field)
modified:
  storage/connect/ha_connect.cc

- Fix MDEV-7494 (adding Insert_quoted in the STRING class)
modified:
  storage/connect/tabmysql.cpp
  storage/connect/xobject.cpp
  storage/connect/xobject.h

- Fix MDEV-7498 in value.cpp (AllocateValue)
modified:
  storage/connect/value.cpp

- Handle backslash in Json serialize + uchar + typo.
modified:
  storage/connect/json.cpp
  storage/connect/tabjson.cpp
2015-01-23 17:54:53 +01:00
Olivier Bertrand
8fc6956430 - Fix compile errors and warnings of LINUX G++
added:
  storage/connect/json.cpp
  storage/connect/json.h
  storage/connect/mysql-test/connect/r/json.result
  storage/connect/mysql-test/connect/std_data/biblio.jsn
  storage/connect/mysql-test/connect/std_data/expense.jsn
  storage/connect/mysql-test/connect/std_data/mulexp3.jsn
  storage/connect/mysql-test/connect/std_data/mulexp4.jsn
  storage/connect/mysql-test/connect/std_data/mulexp5.jsn
  storage/connect/mysql-test/connect/t/json.test
  storage/connect/tabjson.cpp
  storage/connect/tabjson.h
modified:
  storage/connect/CMakeLists.txt
  storage/connect/engmsg.h
  storage/connect/filamtxt.h
  storage/connect/ha_connect.cc
  storage/connect/msgid.h
  storage/connect/mycat.cc
  storage/connect/plgdbsem.h
  storage/connect/tabdos.cpp
  storage/connect/value.cpp
  storage/connect/value.h
2015-01-20 01:21:56 +01:00
Olivier Bertrand
2d2e110bce - Adding the JSON table type
added:
  storage/connect/json.cpp
  storage/connect/json.h
  storage/connect/mysql-test/connect/r/json.result
  storage/connect/mysql-test/connect/std_data/biblio.jsn
  storage/connect/mysql-test/connect/std_data/expense.jsn
  storage/connect/mysql-test/connect/std_data/mulexp3.jsn
  storage/connect/mysql-test/connect/std_data/mulexp4.jsn
  storage/connect/mysql-test/connect/std_data/mulexp5.jsn
  storage/connect/mysql-test/connect/t/json.test
  storage/connect/tabjson.cpp
  storage/connect/tabjson.h
modified:
  storage/connect/CMakeLists.txt
  storage/connect/engmsg.h
  storage/connect/filamtxt.h
  storage/connect/ha_connect.cc
  storage/connect/msgid.h
  storage/connect/mycat.cc
  storage/connect/plgdbsem.h
  storage/connect/tabdos.cpp
  storage/connect/value.cpp
  storage/connect/value.h
2015-01-19 18:55:25 +01:00