Commit graph

14 commits

Author SHA1 Message Date
Mikhail Chalov
19af1890b5 Use memory safe snprintf() in Connect Engine
This commit replaces sprintf(buf, ...) with
snprintf(buf, sizeof(buf), ...),
specifically in the "easy" cases where buf is allocated with a size
known at compile time.

The changes make sure we are not write outside array/string bounds which
will lead to undefined behaviour. In case the code is trying to write
outside bounds - safe version of functions simply cut the string
messages so we process this gracefully.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the BSD-new
license.  I am contributing on behalf of my employer Amazon Web Services,
Inc.

bsonudf.cpp warnings cleanup by Daniel Black

Reviewer: Daniel Black
2022-07-26 16:28:59 +10:00
Olivier Bertrand
0f18135ec8 - Make user variable prefix recognized by IsArgJson and IsJson
modified:   storage/connect/bsonudf.cpp
  modified:   storage/connect/jsonudf.cpp

- Stringify option is now a ; separated list of columns
  modified:   storage/connect/json.cpp
  modified:   storage/connect/json.h
  modified:   storage/connect/mongo.h
  modified:   storage/connect/tabbson.cpp
  modified:   storage/connect/tabcmg.cpp
  modified:   storage/connect/tabcmg.h
  modified:   storage/connect/tabjmg.cpp
  modified:   storage/connect/tabjmg.h
  modified:   storage/connect/tabjson.cpp

- PrepareColist not a static function anymore (+ typo)
  modified:   storage/connect/taboccur.cpp

- JDVC: Recognize schema (database) from a wrapper server
  modified:   storage/connect/tabjdbc.cpp
2021-07-24 16:28:57 +02:00
Olivier Bertrand
ed70f76cf7 - Make function strz return null when LEX_STRING is null
modified:   storage/connect/ha_connect.cc

- Use NOTE instead of WARNING in connect_assisted_discovery
  This because MariaDB raise an error when doing so
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/tabrest.cpp

- Make MONGO tables recognize STRINGIFY and JsonAllPath
  modified:   storage/connect/mongo.cpp
  modified:   storage/connect/mongo.h
  modified:   storage/connect/tabcmg.h
  modified:   storage/connect/tabjmg.cpp
  modified:   storage/connect/tabcmg.cpp
  modified:   storage/connect/tabjmg.h

- Fix OBJECT option for Pretty != 2 JSN and BSON tables
  Accept all syntaxes  for the OBJECT option
  modified:   storage/connect/tabbson.cpp
  modified:   storage/connect/tabjson.cpp

- Use my_snprintf in function xcurl (by vuvova)
  modified:   storage/connect/tabrest.cpp

- Format dates entered as integer when formatted
  modified:   storage/connect/value.cpp
  modified:   storage/connect/value.h
2021-06-24 00:46:12 +02:00
Olivier Bertrand
5ae67c6d63 - All this concern Json or Mongo tables based on MongoDB collections.
- Limit decimals of doubles printed from MongoDB
  Done in function Mini for Mongo C Driver and Java Driver
  Done in function SerializeValue for Java tables using the J Driver
        modified:   storage/connect/cmgoconn.cpp
        modified:   storage/connect/json.cpp
        modified:   storage/connect/json.h
        modified:   storage/connect/tabjmg.cpp

- Fix crash when using BSON_TYPE_DECIMAL128
        modified:   storage/connect/cmgoconn.cpp

- Collection name default to table name
  Fix it when creating tables via discovery
        modified:   storage/connect/ha_connect.cc
        modified:   storage/connect/tabbson.cpp
        modified:   storage/connect/tabjson.cpp
2021-05-05 00:31:21 +02:00
Olivier Bertrand
ef0829ef40 - Major update of the json/bson/mongo table types programs.
Fix several bugs, chiefly concerning CURL operations.
        modified:   storage/connect/bson.cpp
        modified:   storage/connect/cmgfam.cpp
        modified:   storage/connect/cmgoconn.cpp
        modified:   storage/connect/cmgoconn.h
        modified:   storage/connect/colblk.h
        modified:   storage/connect/ha_connect.cc
        modified:   storage/connect/jmgfam.cpp
        modified:   storage/connect/jmgoconn.cpp
        modified:   storage/connect/jmgoconn.h
        modified:   storage/connect/json.cpp
        modified:   storage/connect/json.h
        modified:   storage/connect/mysql-test/connect/r/bson_mongo_c.result
        modified:   storage/connect/mysql-test/connect/r/json_mongo_c.result
        modified:   storage/connect/mysql-test/connect/r/mongo_c.result
        modified:   storage/connect/mysql-test/connect/r/mongo_java_2.result
        modified:   storage/connect/mysql-test/connect/r/mongo_java_3.result
        modified:   storage/connect/mysql-test/connect/std_data/Mongo2.jar
        modified:   storage/connect/mysql-test/connect/std_data/Mongo3.jar
        modified:   storage/connect/tabbson.cpp
        modified:   storage/connect/tabbson.h
        modified:   storage/connect/tabcmg.cpp
        modified:   storage/connect/tabcmg.h
        modified:   storage/connect/tabjmg.cpp
        modified:   storage/connect/tabjmg.h
        modified:   storage/connect/tabjson.cpp
        modified:   storage/connect/tabjson.h
2021-05-01 22:29:38 +02:00
Olivier Bertrand
9644415fa9 - Fix MDEV-16672 Connect: Warnings with 10.0
filamtxt.cpp: DOSFAM::RenameTempFile: Change sprintf to snprintf.
  filamvct.cpp: VECFAM::RenameTempFile: Change sprintf to snprintf.
  javaconn.cpp:
    Add JAVAConn::GetUTFString function.
    Use it instead of env->GetStringUTFChars.
    Fix wrong identation.
  javaconn.h: Add GetUTFString declaration.
  jdbconn.cpp:
    Use GetUTFString function instead of env->GetStringUTFChars.
  jmgoconn.cpp:
    Use GetUTFString function instead of env->GetStringUTFChars.
    Fix wrong identation.
  jsonudf.cpp: change 139 to BMX line 4631.
  tabjmg.cpp:
    Add ReleaseStringUTF.
    Fix wrong identation.
  tabpivot.cpp: Fix wrong identation.
  tabutil.cpp: TDBPRX::GetSubTable: Change sprintf to snprintf.
  modified:   storage/connect/filamtxt.cpp
  modified:   storage/connect/filamvct.cpp
  modified:   storage/connect/javaconn.cpp
  modified:   storage/connect/javaconn.h
  modified:   storage/connect/jdbconn.cpp
  modified:   storage/connect/jmgoconn.cpp
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/tabjmg.cpp
  modified:   storage/connect/tabpivot.cpp
  modified:   storage/connect/tabutil.cpp

- Fix MDEV-16895 CONNECT engine's get_error_message can cause buffer
                 overflow and server crash with long queries
  ha_connect_cc: Update version.
    get_error_message: Remove charset conversion.
  modified:   storage/connect/ha_connect.cc

- Fix a server crash on inserting bigint to a JDBC table
  JDBConn::SetUUID:
    Suppress check on ctyp that causes a server crash because ctyp
    can be negative and this triggers an DEBUG_ASSERT on return.
  modified:   storage/connect/jdbconn.cpp

- Update jdbc.result
  mysql-test/connect/r/jdbc.result: Recorded to reflect a message change.
  modified:   storage/connect/mysql-test/connect/r/jdbc.result
2018-08-06 19:42:00 +02:00
Olivier Bertrand
c83e2a6345 - Fix a bug in TYPVAL<PSZ> compute causing it sometime not to be executed
This was the cause of the bug in CalculateArray
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/value.cpp

- Avoid some compiler warnings
  modified:   storage/connect/mongo.cpp
  modified:   storage/connect/tabjmg.cpp
  modified:   storage/connect/tabtbl.cpp
  modified:   storage/connect/tabutil.cpp
2017-10-16 15:08:17 +02:00
Olivier Bertrand
bcfb5b2de9 - Update version number
modified:   storage/connect/ha_connect.cc

- Include MONGO in all Java enabled distributions
  Mongo will be enabled only for 10.2 and 10.3
  modified:   storage/connect/CMakeLists.txt

- Change JDBC_SUPPORT to JAVA_SUPPORT which also replaces MONGO_SUPPORT
  MONGO_SUPPORT is now just used to enable the MONGO table type
  modified:   storage/connect/filter.cpp
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h
  modified:   storage/connect/mongo.cpp
  modified:   storage/connect/mycat.cc
  modified:   storage/connect/plgdbutl.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h

- Move MakeSelector function from FILTER to mongo.cpp
  modified:   storage/connect/filter.cpp
  modified:   storage/connect/filter.h
  modified:   storage/connect/cmgoconn.cpp
  modified:   storage/connect/jmgoconn.cpp
  modified:   storage/connect/mongo.cpp

- Do mongo_init only on first use of the MongoDB C Driver
  This will permit to delay load the mongo lib on Windows
  modified:   storage/connect/cmgoconn.cpp
  modified:   storage/connect/cmgoconn.h
  modified:   storage/connect/ha_connect.cc

- Replace NEW_VAR by a test on MYSQL_VERSION_ID
  modified:   storage/connect/ha_connect.cc

- Suppress enable_mongo session variable
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/mycat.cc

- Make some function headers identical in .h and .cc file
  (replacing const char* by PCSZ)
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h

- Change a parameter type from uchar* to const uchar*
  (for ScanRecord and CheckRecord)
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h

- Changes on LIKE and NOT LIKE does not fix a bug yet
  modified:   storage/connect/ha_connect.cc

- Suppress PIVOT_SUPPORT (PIVOT type is unconditionnal)
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/mycat.cc

- Change the strz function from inline to static
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h

- export the JavaConn class and the MgoColumns and IsNum functions
  modified:   storage/connect/javaconn.h
  modified:   storage/connect/json.h
  modified:   storage/connect/mongo.h

- Fix MDEV-13924
  modified:   storage/connect/jdbconn.cpp

- Make a temporary fix for the compiler bug in CalculateArray
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/tabjson.cpp

- Typo
  modified:   storage/connect/jdbccat.h
  modified:   storage/connect/reldef.h
  modified:   storage/connect/tabext.h
  modified:   storage/connect/tabjmg.cpp
  modified:   storage/connect/tabxml.h
  modified:   storage/connect/valblk.h
  modified:   storage/connect/value.h
  modified:   storage/connect/xtable.h

- Fix a bug in MONGO tests by changing 'MONGO' to $TYPE
  modified:   storage/connect/mysql-test/connect/t/mongo_test.inc

- Record test results to reflect all changes
  modified:   storage/connect/mysql-test/connect/r/json_java_2.result
  modified:   storage/connect/mysql-test/connect/r/json_java_3.result
  modified:   storage/connect/mysql-test/connect/r/json_mongo_c.result
  modified:   storage/connect/mysql-test/connect/r/mongo_c.result
  modified:   storage/connect/mysql-test/connect/r/mongo_java_2.result
  modified:   storage/connect/mysql-test/connect/r/mongo_java_3.result
2017-10-15 16:13:23 +02:00
Olivier Bertrand
d8f99f165b - Add FBLOCK when opening ODBC, JSON and MONGO tables.
This to have automatic closing in case of thrown error.
  modified:   storage/connect/cmgoconn.cpp
  modified:   storage/connect/cmgoconn.h
  modified:   storage/connect/filamzip.cpp
  modified:   storage/connect/javaconn.cpp
  modified:   storage/connect/javaconn.h
  modified:   storage/connect/odbconn.cpp
  modified:   storage/connect/odbconn.h
  modified:   storage/connect/plgdbsem.h
  modified:   storage/connect/plgdbutl.cpp

- Fix INCOL bug when inserting several lines to MONGO tables
  modified:   storage/connect/cmgoconn.cpp
  modified:   storage/connect/tabcmg.cpp
  modified:   storage/connect/tabjmg.cpp

- MONGO_SUPPORT is set for both MongoDB drivers
  CMGO_SUPPORT  is set only when the C driver is available
  modified:   storage/connect/CMakeLists.txt
  modified:   storage/connect/filter.h
  modified:   storage/connect/filter.h
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/mongo.cpp
  modified:   storage/connect/mycat.cc
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h

- Separate enums JCATINFO and
  modified:   storage/connect/javaconn.h
  modified:   storage/connect/jdbconn.cpp

- Fix crash when executing JDBC catfunc=driver
  modified:   storage/connect/jdbconn.cpp

- Report an error when the Mongo driver if explicitly
specified and not supported.
  modified:   storage/connect/mongo.cpp

- Fix bug causing catalog JSON tables to fail
  modified:   storage/connect/tabjson.cpp

- Protect by mutex the Ready variable of the TBLTBM table type
  modified:   storage/connect/tabtbl.cpp

- Put testing of Thread TBL tables in a separate test
With added case and modified old case that could avoid the test to fail.
  modified:   storage/connect/mysql-test/connect/r/tbl.result
  modified:   storage/connect/mysql-test/connect/t/tbl.test
  new file:   storage/connect/mysql-test/connect/r/tbl_thread.result
  new file:   storage/connect/mysql-test/connect/t/tbl_thread.test

- jmongo3.test no more exists
  deleted:    storage/connect/mysql-test/connect/t/jmongo3.test

- Add new tests for the MONGO feature (disabled)
  modified:   storage/connect/mysql-test/connect/disabled.def
  new file:   storage/connect/mysql-test/connect/r/json_java_2.result
  new file:   storage/connect/mysql-test/connect/r/json_java_3.result
  new file:   storage/connect/mysql-test/connect/r/json_mongo_c.result
  new file:   storage/connect/mysql-test/connect/r/mongo_c.result
  new file:   storage/connect/mysql-test/connect/r/mongo_java_2.result
  new file:   storage/connect/mysql-test/connect/r/mongo_java_3.result
  new file:   storage/connect/mysql-test/connect/std_data/Mongo2.jar
  new file:   storage/connect/mysql-test/connect/std_data/cities.json
  new file:   storage/connect/mysql-test/connect/t/json_java_2.test
  new file:   storage/connect/mysql-test/connect/t/json_java_3.test
  new file:   storage/connect/mysql-test/connect/t/json_mongo_c.test
  new file:   storage/connect/mysql-test/connect/t/mongo.inc
  new file:   storage/connect/mysql-test/connect/t/mongo_c.test
  new file:   storage/connect/mysql-test/connect/t/mongo_java_2.test
  new file:   storage/connect/mysql-test/connect/t/mongo_java_3.test
  new file:   storage/connect/mysql-test/connect/t/mongo_test.inc
2017-08-05 18:08:51 +02:00
Olivier Bertrand
510e67c6fd Fix discovery crash for Java Mongo tables. Modified: storage/connect/tabjmg.cpp 2017-07-23 12:41:44 +02:00
Olivier Bertrand
38c9c0d22b Fix compile warnings and errors for nullptr. Modified /storage/connect/tabjmg (cpp and h) 2017-07-18 16:12:32 +02:00
Olivier Bertrand
f590296c28 -- Finalize work on MongoDB access
Implement discovery for the MongoDB Java Driver
   Create classes to minimize code and avoid dupicates
   Rearrange and rename implied files
  modified:   storage/connect/CMakeLists.txt
  renamed:    storage/connect/mongofam.cpp -> storage/connect/cmgfam.cpp
  renamed:    storage/connect/mongofam.h -> storage/connect/cmgfam.h
  modified:   storage/connect/cmgoconn.h
  modified:   storage/connect/javaconn.h
  modified:   storage/connect/jdbconn.cpp
  modified:   storage/connect/jmgoconn.cpp
  modified:   storage/connect/jmgoconn.h
  modified:   storage/connect/mongo.cpp
  modified:   storage/connect/mongo.h
  renamed:    storage/connect/tabmgo.cpp -> storage/connect/tabcmg.cpp
  renamed:    storage/connect/tabmgo.h -> storage/connect/tabcmg.h
  modified:   storage/connect/tabjmg.cpp
  modified:   storage/connect/tabjmg.h
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h

-- Trace Sarea allocation and freeing
  modified:   storage/connect/connect.cc
  modified:   storage/connect/plugutil.cpp
  modified:   storage/connect/user_connect.cc

-- Null Json values where not mark as null in JSNX::SetJsonValue
   This was added in TYPE_NULL (declared as TYPE_VOID)
  modified:   storage/connect/json.cpp
  modified:   storage/connect/json.h
  modified:   storage/connect/jsonudf.cpp

-- Null JValues are ignored in JSNX::CalculateArray
   Also done in tabjson.cpp for JSONCOL::CalculateArray
  modified:   storage/connect/json.h
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/tabjson.cpp

-- Null JSON values now represented by connect_json_null session variable
  modified:   storage/connect/json.cpp
  modified:   storage/connect/json.h
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/tabjson.cpp

-- JVALUE has size = 1
  modified:   storage/connect/json.h

-- Fix by vuvova because Debian compilation failure.
  modified:   storage/connect/value.cpp
2017-07-18 13:16:55 +02:00
Olivier Bertrand
b63f847743 - Suppress warning when creating mongo JSON tables
modified:   storage/connect/ha_connect.cc

- Add catalog table for Mongo java driver tables
  modified:   storage/connect/mongo.cpp
  modified:   storage/connect/mongo.h
  modified:   storage/connect/tabjmg.cpp
  modified:   storage/connect/tabjmg.h
2017-07-03 22:28:55 +02:00
Olivier Bertrand
94e5d7de85 - Add Support of the MongoDB Java Driver.
modified:   storage/connect/CMakeLists.txt
  modified:   storage/connect/JavaWrappers.jar
  modified:   storage/connect/colblk.h
  modified:   storage/connect/filter.cpp
  modified:   storage/connect/filter.h
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h
  modified:   storage/connect/jdbccat.h
  modified:   storage/connect/jdbconn.cpp
  modified:   storage/connect/jdbconn.h
  modified:   storage/connect/mongofam.cpp
  modified:   storage/connect/mongofam.h
  modified:   storage/connect/mycat.cc
  modified:   storage/connect/mycat.h
  modified:   storage/connect/tabext.h
  modified:   storage/connect/tabjdbc.cpp
  modified:   storage/connect/tabjdbc.h
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabjson.h
  modified:   storage/connect/tabmgo.cpp
  modified:   storage/connect/tabmgo.h
  created:    storage/connect/Mongo2Interface.java
  created:    storage/connect/Mongo3Interface.java
  created:    storage/connect/cmgoconn.cpp
  created:    storage/connect/cmgoconn.h
  created:    storage/connect/javaconn.cpp
  created:    storage/connect/javaconn.h
  created:    storage/connect/jmgfam.cpp
  created:    storage/connect/jmgfam.h
  created:    storage/connect/jmgoconn.cpp
  created:    storage/connect/jmgoconn.h
  created:    storage/connect/mongo.cpp
  created:    storage/connect/mongo.h
  created:    storage/connect/tabjmg.cpp
  created:    storage/connect/tabjmg.h

- tdbp not initialized when catched exception
in CntGetTDB (connect.cc line 188)
  modified:   storage/connect/connect.h

- CheckCleanup should sometimes doing cleanup on pure info
Sometimes MariaDB loops on info to get the size of all tables in a database.
This can sometimes fail by exhausted memory.
CheckCleanup now have a force boolean parameter (defaulting to false)
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/user_connect.cc
  modified:   storage/connect/user_connect.h

Change the copyright of some source files
  modified:   storage/connect/connect.cc
  modified:   storage/connect/connect.h
  modified:   storage/connect/engmsg.h
  modified:   storage/connect/global.h
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h
  modified:   storage/connect/msgid.h
  modified:   storage/connect/mycat.cc
  modified:   storage/connect/mycat.h
  modified:   storage/connect/os.h
  modified:   storage/connect/osutil.c
  modified:   storage/connect/osutil.h
  modified:   storage/connect/user_connect.cc
  modified:   storage/connect/user_connect.h
2017-07-02 22:41:11 +02:00