Old style C functions `strcpy()`, `strcat()` and `sprintf()` are vulnerable to
security issues due to lacking memory boundary checks. Replace these in the
Connect storage engine with safe new and/or custom functions such as
`snprintf()` `safe_strcpy()` and `safe_strcat()`.
With this change FlawFinder and other static security analyzers report 287
fewer findings.
All new code of the whole pull request, including one or several files that are
either new files or modified ones, are contributed under the BSD-new license. I
am contributing on behalf of my employer Amazon Web Services, Inc.
This commit replaces sprintf(buf, ...) with
snprintf(buf, sizeof(buf), ...),
specifically in the "easy" cases where buf is allocated with a size
known at compile time.
The changes make sure we are not write outside array/string bounds which
will lead to undefined behaviour. In case the code is trying to write
outside bounds - safe version of functions simply cut the string
messages so we process this gracefully.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the BSD-new
license. I am contributing on behalf of my employer Amazon Web Services,
Inc.
bsonudf.cpp warnings cleanup by Daniel Black
Reviewer: Daniel Black
accessing freed memory.
Before XMLCOL::WriteColumn() Tdbp->Clist gets assigned
a nodelist in
Clist = RowNode->SelectNodes(g, Colname, Clist);
which is RowNode->Doc->Xop->nodesetval.
In XMLCOL::WriteColumn()
ValNode = ColNode->SelectSingleNode(g, Xname, Vxnp);
calls LIBXMLDOC::GetNodeList() again, which frees the previous
XPath object Xop and replaces it with a new one.
In this case RowNode->Doc == ColNode->Doc, so Clist->Listp
points to a freed memory now.
- Replace strncpy() with strmake() to ensure things are \0 terminated
- Initialized some variables that caused compiler warnings
- Increased buffer that caused warnings from strncpy
One warning from InnoDB, others from Connect.
This when the HTTP contains & characters
modified: storage/connect/tabbson.cpp
modified: storage/connect/tabjson.cpp
- Make stringfy option work on only one Json item
modified: storage/connect/tabbson.cpp
modified: storage/connect/tabbson.h
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabjson.h
- Make Json/Bson DATE columns accept JSON date syntax
modified: storage/connect/tabbson.cpp
modified: storage/connect/tabjson.cpp
- Fix bug making REST table default file not being
erased when dropping the table
modified: storage/connect/tabbson.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabrest.cpp
modified: storage/connect/tabxml.cpp
- Suppress CHAR(36) --> VARCHAR(36) when DEVELOPMENT
This was fixed in MyClient
modified: storage/connect/ha_connect.cc
Also add a new member Saved_Size in the Global structure.
modified: storage/connect/global.h
modified: storage/connect/plugutil.cpp
modified: storage/connect/user_connect.cc
modified: storage/connect/jsonudf.cpp
- Add session variables json_all_path and default_depth
modified: storage/connect/ha_connect.cc
modified: storage/connect/mongo.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabxml.cpp
- ADD column options JPATH and XPATH
Work as FIELD_FORMAT but are more readable
modified: storage/connect/ha_connect.cc
modified: storage/connect/ha_connect.h
modified: storage/connect/mysql-test/connect/r/json_java_2.result
modified: storage/connect/mysql-test/connect/r/json_java_3.result
modified: storage/connect/mysql-test/connect/r/json_mongo_c.result
- Handle negative numbes in the option list
modified: storage/connect/ha_connect.cc
- Fix Json parse that could crash the server.
Was because it could use THROW out of the TRY block.
Also handle all error by THROW.
It is now done by a new class JSON.
modified: storage/connect/json.cpp
modified: storage/connect/json.h
- Add a new UDF function jfile_translate.
It translate a Json file to pretty = 0.
Fast because it does not a real parse of the file.
modified: storage/connect/jsonudf.cpp
modified: storage/connect/jsonudf.h
- Add a now options JSIZE and STRINGIFY to Json tables.
STRINGIFY makes Objects or Arrays to be returned by their
json representation instead of by their concatenated values.
JSIZE allows to specify the LRECL (was 256) defaults to 1024.
Also fix a bug about locating the sub-table by its path.
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabjson.h
All variables handling sizes that were uint are now size_t.
The variable connect_work_size is now ulong (was uint);
Also make Json functiosn to allocate a larger memory (M=9 was 7)
modified: storage/connect/global.h
modified: storage/connect/ha_connect.cc
modified: storage/connect/json.cpp
modified: storage/connect/jsonudf.cpp
modified: storage/connect/plgdbutl.cpp
modified: storage/connect/plugutil.cpp
modified: storage/connect/user_connect.cc
- Fix uninitialised variable (pretty) in Json_File.
Make Jbin_file accept the same arguments as Json_File ones.
modified: storage/connect/jsonudf.cpp
- Change the Level option to Depth (the word currently used)
(Level being still accepted)
modified: storage/connect/mongo.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabxml.cpp
- Suppress 2nd argument default value for MYSQLtoPLG function
modified: storage/connect/myutil.h
- Allow REST tables to be create not specifying a file_name
modified: storage/connect/tabrest.cpp
Import complex XML from multiple files in MariaDB
Some row results are missing and replaced by the last file one.
Thats because Nx and Sx column members are not reset when changing file.
modified: storage/connect/tabxml.cpp
modified: storage/connect/tabxml.h
Import complex XML from multiple files in MariaDB
Some row results are missing and replaced by the last file one.
Thats because Nx and Sx column members are not reset when changing file.
modified: storage/connect/tabxml.cpp
modified: storage/connect/tabxml.h
Problem with NOT LIKE queries.
modified: storage/connect/ha_connect.cc
modified: sql/item_cmpfunc.h
- Fix MDEV-21084
Misusage of strncat could cause buffer overflow.
modified: storage/connect/reldef.cpp
modified: storage/connect/tabcmg.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabrest.cpp
modified: storage/connect/tabxml.cpp
In addition to files and Mongo collections, JSON as well as XML and CSV data can be retrieved
from the net as answers from REST queries. Because it uses and external package (cpprestsdk)
this is currently available only to MariaDB servers compiled from source.
-- Add the REST support when Microsoft Casablanca package (cpprestsdk) is installed.
-- Add compile flags needed on Windows /MD or /MDd (debug)
-- Also include some changes specific to MariaDB 10.3.
modified: storage/connect/CMakeLists.txt
-- Add conditional REST support
-- Added string options HTTP and URI.
-- Added added internal table type TAB_REST.
modified: storage/connect/ha_connect.cc
modified: storage/connect/mycat.cc
modified: storage/connect/mycat.h
modified: storage/connect/plgdbsem.h
-- Add conditional code based on the preprocessor definition MARIADB
-- This to be able to use the same code in CONNECT and EOM modules
modified: storage/connect/osutil.h
modified: storage/connect/tabrest.cpp
-- Add files for the REST OEM module
added: storage/connect/mini-global.h
added: storage/connect/rest.def
-- Fix MDEV-19648 Variable connect_conv_size doesn't change
-- Change Variable wrong block parameter from 8169 to 1.
-- Also change connect_conv_size default value to 1024.
modified: storage/connect/ha_connect.cc
-- Fix compilation error when ZIP is not supported
modified: storage/connect/ha_connect.cc
modified: storage/connect/tabfmt.cpp
-- Replace PlugSetPath by some concat (crashed on Fedora) + typo
modified: storage/connect/reldef.cpp
-- Avoid possible buffer overflow
-- In particular by the function ShowValue.
modified: storage/connect/tabdos.cpp
modified: storage/connect/tabfmt.cpp
modified: storage/connect/value.cpp
modified: storage/connect/value.h
-- Add some cast to avoid some compiler warnings
modified: storage/connect/filamdbf.cpp
-- Fix some C++ error
modified: storage/connect/javaconn.cpp
modified: storage/connect/jmgoconn.cpp
modified: storage/connect/plugutil.cpp
-- Add some tracing + typo
modified: storage/connect/mycat.cc
modified: storage/connect/tabjson.cpp
-- Add the xtrc tracing function
modified: storage/connect/global.h
modified: storage/connect/plugutil.cpp
-- Modify tracing to use xtrc and some typo
modified: storage/connect/array.cpp
modified: storage/connect/block.h
-- Miscellaneous Typo and warning suppressing changes
modified: storage/connect/connect.cpp
modified: storage/connect/connect.h
modified: storage/connect/filamvct.cpp
modified: storage/connect/inihandl.cpp
modified: storage/connect/jsonudf.cpp
modified: storage/connect/libdoc.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabtbl.cpp
modified: storage/connect/tabxml.cpp
modified: storage/connect/user_connect.cc
modified: storage/connect/user_connect.h
-- Update failing test results and disbling
modified: storage/connect/mysql-test/connect/disabled.def
modified: storage/connect/mysql-test/connect/r/dir.result
modified: storage/connect/mysql-test/connect/r/grant.result
modified: storage/connect/mysql-test/connect/r/jdbc.result
modified: storage/connect/mysql-test/connect/r/jdbc_postgresql.result
modified: storage/connect/mysql-test/connect/r/xml.result
modified: storage/connect/mysql-test/connect/r/xml2.result
modified: storage/connect/mysql-test/connect/r/xml2_mult.result
modified: storage/connect/mysql-test/connect/r/xml_mult.result
-- Add an option
modified: storage/connect/mysql-test/connect/t/grant.test
In addition to files and Mongo collections, JSON as well as XML and CSV data can be retrieved
from the net as answers from REST queries. Because it uses and external package (cpprestsdk)
this is currently available only to MariaDB servers compiled from source.
-- Add the REST support when Microsoft Casablanca package (cpprestsdk) is installed.
-- Also include some changes specific to MariaDB 10.3.
modified: storage/connect/CMakeLists.txt
-- Add conditional REST support
-- Added string options HTTP and URI.
-- Added added internal table type TAB_REST.
modified: storage/connect/ha_connect.cc
modified: storage/connect/mycat.cc
modified: storage/connect/mycat.h
modified: storage/connect/plgdbsem.h
-- Fix MDEV-19648 Variable connect_conv_size doesn't change
-- Change Variable wrong block parameter from 8169 to 1.
-- Also change connect_conv_size default value to 1024.
modified: storage/connect/ha_connect.cc
-- Avoid possible buffer overflow
-- In particular by the function ShowValue.
modified: storage/connect/tabdos.cpp
modified: storage/connect/tabfmt.cpp
modified: storage/connect/value.cpp
modified: storage/connect/value.h
-- Add some cast to avoid some compiler warnings
modified: storage/connect/filamdbf.cpp
-- Fix some C++ error
modified: storage/connect/javaconn.cpp
modified: storage/connect/jmgoconn.cpp
modified: storage/connect/plugutil.cpp
-- Miscellaneous Typo and warning suppressing changes
modified: storage/connect/connect.cpp
modified: storage/connect/connect.h
modified: storage/connect/filamvct.cpp
modified: storage/connect/inihandl.cpp
modified: storage/connect/jsonudf.cpp
modified: storage/connect/libdoc.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabtbl.cpp
modified: storage/connect/tabxml.cpp
modified: storage/connect/user_connect.cc
modified: storage/connect/user_connect.h
-- Update failing test results and disbling
modified: storage/connect/mysql-test/connect/disabled.def
modified: storage/connect/mysql-test/connect/r/dir.result
modified: storage/connect/mysql-test/connect/r/grant.result
modified: storage/connect/mysql-test/connect/r/jdbc.result
modified: storage/connect/mysql-test/connect/r/jdbc_postgresql.result
modified: storage/connect/mysql-test/connect/r/xml.result
modified: storage/connect/mysql-test/connect/r/xml2.result
modified: storage/connect/mysql-test/connect/r/xml2_mult.result
modified: storage/connect/mysql-test/connect/r/xml_mult.result
-- Add an option
modified: storage/connect/mysql-test/connect/t/grant.test
commit 6a6a1f37798
Author: Olivier Bertrand <bertrandop@gmail.com>
Date: Fri Jan 4 12:31:52 2019 +0100
- Fix a few bug mainly concerning discovery and call from OEM
(and prepare new table types)
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabjson.h
modified: storage/connect/tabxml.cpp
modified: storage/connect/tabxml.h
- Fix wrong line estimate
modified: storage/connect/mysql-test/connect/r/part_table.result
modified: storage/connect/mysql-test/connect/t/part_table.test
commit bd7d2e912d9
Author: Olivier Bertrand <bertrandop@gmail.com>
Date: Tue Dec 4 23:35:09 2018 +0100
Fix wrong version number
commit 4933680e7ab
Author: Olivier Bertrand <bertrandop@gmail.com>
Date: Sun Dec 2 00:25:05 2018 +0100
- Make PlugSubAlloc to be exportable
Suppress unused parameter from PlugSubSet
modified: storage/connect/global.h
modified: storage/connect/plugutil.cpp
modified: storage/connect/jsonudf.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/user_connect.cc
- Fix a bug making column catalog XML tables fail
modified: storage/connect/tabxml.cpp
- Comment out wrong message
modified: storage/connect/ha_connect.cc
- Update error message when sorting an ODBC table fails
modified: storage/connect/tabodbc.cpp
- Add error message when gettting an address
from an OEM fails.
modified: storage/connect/reldef.cpp
- Make some modifications useful for OEM module writting
Export discovery functions for CSV, JDBC and XML
Remove unuseful include from tabjson.h
Move TDBXML::data_charset function from header file to source
modified: storage/connect/tabfmt.h
modified: storage/connect/tabjson.h
modified: storage/connect/tabxml.cpp
modified: storage/connect/tabxml.h
- Update test result
modified: storage/connect/mysql-test/connect/r/jdbc_oracle.result
Suppress unused parameter from PlugSubSet
modified: storage/connect/global.h
modified: storage/connect/plugutil.cpp
modified: storage/connect/jsonudf.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/user_connect.cc
- Fix a bug making column catalog XML tables fail
modified: storage/connect/tabxml.cpp
- Comment out wrong message
modified: storage/connect/ha_connect.cc
- Update error message when sorting an ODBC table fails
modified: storage/connect/tabodbc.cpp
- Add error message when gettting an address
from an OEM fails.
modified: storage/connect/reldef.cpp
- Make some modifications useful for OEM module writting
Export discovery functions for CSV, JDBC and XML
Remove unuseful include from tabjson.h
Move TDBXML::data_charset function from header file to source
modified: storage/connect/tabfmt.h
modified: storage/connect/tabjson.h
modified: storage/connect/tabxml.cpp
modified: storage/connect/tabxml.h
- Update test result
modified: storage/connect/mysql-test/connect/r/jdbc_oracle.result
Suppress unused parameter from PlugSubSet
modified: storage/connect/global.h
modified: storage/connect/plugutil.cpp
modified: storage/connect/jsonudf.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/user_connect.cc
- Fix a bug making column catalog XML tables fail
modified: storage/connect/tabxml.cpp
- Comment out wrong message
modified: storage/connect/ha_connect.cc
- Update error message when sorting an ODBC table fails
modified: storage/connect/tabodbc.cpp
- Add error message when gettting an address
from an OEM fails.
modified: storage/connect/reldef.cpp
- Make some modifications useful for OEM module writting
Export discovery functions for CSV, JDBC and XML
Remove unuseful include from tabjson.h
Move TDBXML::data_charset function from header file to source
modified: storage/connect/tabfmt.h
modified: storage/connect/tabjson.h
modified: storage/connect/tabxml.cpp
modified: storage/connect/tabxml.h
- Update test result
modified: storage/connect/mysql-test/connect/r/jdbc_oracle.result
