Old style C functions `strcpy()`, `strcat()` and `sprintf()` are vulnerable to
security issues due to lacking memory boundary checks. Replace these in the
Connect storage engine with safe new and/or custom functions such as
`snprintf()` `safe_strcpy()` and `safe_strcat()`.
With this change FlawFinder and other static security analyzers report 287
fewer findings.
All new code of the whole pull request, including one or several files that are
either new files or modified ones, are contributed under the BSD-new license. I
am contributing on behalf of my employer Amazon Web Services, Inc.
This commit replaces sprintf(buf, ...) with
snprintf(buf, sizeof(buf), ...),
specifically in the "easy" cases where buf is allocated with a size
known at compile time.
The changes make sure we are not write outside array/string bounds which
will lead to undefined behaviour. In case the code is trying to write
outside bounds - safe version of functions simply cut the string
messages so we process this gracefully.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the BSD-new
license. I am contributing on behalf of my employer Amazon Web Services,
Inc.
bsonudf.cpp warnings cleanup by Daniel Black
Reviewer: Daniel Black
Test prior to this change:
CURRENT_TEST: connect.mysql
mysqltest: At line 485: query 'INSERT IGNORE INTO t3 VALUES (5),(10),(30)' failed: ER_GET_ERRMSG (1296): Got error 122 '(1062) Duplicate entry '10' for key 'PRIMARY' [INSERT INTO `t1` (`a`) VALUES (10)]' from CONNECT
So the ignore table option wasn't getting passed to the remove server.
Closes#2008
If the connecting user doesn't have alter table privilege this isn't
allowed.
This patch removes enable / disable key commands that should never have been here
Closes#2002
This new CONNECT version 1.07 fully implements NOSQL support.
It allows working on JSON or XML data retrieved as REST query results
from all binary distributions of MariaDB when cpprestsdk is installed
and the GetRest library is available.
=====================================================================
- Make Rest available for MariaDB binary distributed versions.
Change RestGet function so it can be called from a library.
modified: storage/connect/CMakeLists.txt
modified: storage/connect/restget.cpp
modified: storage/connect/tabrest.cpp
- Make column FLAG option available to discovery functions.
modified: storage/connect/ha_connect.cc
modified: storage/connect/plgdbsem.h
- Update CONNECT version number and date.
modified: storage/connect/ha_connect.cc
- Move OEMColumns function from mycat.cc to reldef.cpp.
modified: storage/connect/mycat.cc
modified: storage/connect/reldef.cpp
- Allocate tables as TABREF (was RELDEF)
modified: storage/connect/mycat.cc
modified: storage/connect/mycat.h
- Fix MDEV-20845 by commenting out TIMEOUT setting.
modified: storage/connect/myconn.cpp
- Call DefineAM before calling GetColCatInfo. Column offset
is now based on record format instead of table type.
The RECFM_VCT format was added.
This enables tables to specify the record format and is
useful in particular for OEM tables.
modified: storage/connect/plgdbsem.h
modified: storage/connect/reldef.cpp
modified: storage/connect/reldef.h
modified: storage/connect/tabdos.cpp
modified: storage/connect/tabdos.h
modified: storage/connect/tabfix.cpp
modified: storage/connect/tabfmt.cpp
modified: storage/connect/tabmysql.cpp
modified: storage/connect/tabutil.cpp
modified: storage/connect/tabutil.h
modified: storage/connect/tabvct.cpp
modified: storage/connect/xindex.cpp
- clang complains about register in C++
- Removed not used variables
- Fixed bug when printing date in filamdbf.cpp
- Added {} to fix warning about dangling else
- Changed connect_done_func() to be global to remove conflict with header
files
- Added extra () around assignment in if
when joining a table to a MYSQL indexed table.
modified: storage/connect/myconn.cpp
modified: storage/connect/myconn.h
modified: storage/connect/tabmysql.cpp
- Add more tests to the mysql_index.test file
modified: storage/connect/mysql-test/connect/r/mysql_index.result
modified: storage/connect/mysql-test/connect/t/mysql_index.test
- Fix and error causing remote indexing to fail when for not unique index.
Was experienced with MYSQL, ODBC and JDBC tables.
modified: storage/connect/connect.cc
- Fix MDEV-9966 (zero lines returned)
modified: storage/connect/ha_connect.cc
modified: storage/federatedx/ha_federatedx.cc
The table type is MYSQL
The query where clause includes an indexed column
The where clause contains < or <= operator on this column
Change version date
modified: storage/connect/ha_connect.cc
modified: storage/connect/tabmysql.cpp
Add visual studio 2013 files to ignore
modified: .gitignore
- Added JSON OBJECT specification for pretty != 2.
- Fix NULL values not recognized for nullable JSON columns
- Issue an error message when a JSON table is created without specifying LRECL if PRETTY != 2.
- Make JSONColumns use a TDBJSON class.
- Make JSON table using MAPFAM
modified:
filamap.h
filamtxt.h
ha_connect.cc
json.result
tabjson.cpp
tabjson.h
table.cpp
- Implementing Discovery for the XML table type.
modified:
domdoc.cpp
domdoc.h
ha_connect.cc
libdoc.cpp
plgxml.cpp
plgxml.h
reldef.cpp
reldef.h
tabxml.cpp
tabxml.h
- Providing an error message when creating an ODBC table via discovery returns
columns of more than one table.
modified:
ha_connect.cc
- TableOptionStruct declaration moved from ha_connect.h to mycat.h
To make it easier to use by other classes.
modified:
ha_connect.cc
ha_connect.h
mycat.cc
mycat.h
reldef.cpp
tabmysql.cpp
taboccur.cpp
tabpivot.cpp
tabtbl.cpp
tabutil.cpp
tabxcl.cpp
tables. This should fix bug MDEV-7343.
modified:
storage/connect/ha_connect.cc
storage/connect/myconn.cpp
storage/connect/myconn.h
storage/connect/reldef.cpp
storage/connect/reldef.h
storage/connect/table.cpp
storage/connect/tabmysql.cpp
storage/connect/xtable.h
- Prevent double column evaluation when CONNECT does filtering
modified:
storage/connect/connect.cc
- Export CreateFileMap and CloseMemMap (for OEM tables)
modified:
storage/connect/maputil.h
- Add the compute function to be used on VALUE types.
Preserve precision for DOUBLE values.
modified:
storage/connect/value.cpp
storage/connect/value.h
- Typo (in preparation to the future JSON table type)
modified:
storage/connect/ha_connect.cc
storage/connect/mycat.cc
storage/connect/plgdbsem.h
They are still experimental and should not be used in production.
added:
storage/connect/encas.h
storage/connect/english.msg
storage/connect/enids.h
storage/connect/frcas.h
storage/connect/french.msg
storage/connect/frids.h
storage/connect/frmsg.h
modified:
storage/connect/connect.cc
storage/connect/engmsg.h
storage/connect/filamdbf.cpp
storage/connect/global.h
storage/connect/ha_connect.cc
storage/connect/msgid.h
storage/connect/plgdbutl.cpp
storage/connect/plugutil.c
storage/connect/rcmsg.c
storage/connect/resource.h
storage/connect/tabfmt.h
- Fix global variable not being properly initialized (MDEV-6690, MDEV-7094)
modified:
storage/connect/ha_connect.cc
storage/connect/plugutil.c
storage/connect/user_connect.cc
storage/connect/xindex.cpp
- Implement Rewind for ODBC tables (MDEV-7097)
modified:
storage/connect/odbconn.cpp
storage/connect/odbconn.h
storage/connect/tabmysql.cpp
storage/connect/tabodbc.cpp
- Reset N when reopening MYSQL tables
modified:
storage/connect/tabmysql.cpp
the beginning. Defining the STRING class and begining to use it (MYSQL)
2) Change the xtrace, use_tempfile and exact_info connect variables from
GLOBAL to SESSION. Remaining GLOBAL variables have been made readonly.
3) Take care of LEX_STRING variables. The .str should not be regarded as
allways being 0 terminated. This is handled by the Strz functions that
make sure to return 0 terminated strings.
Bug fix:
- When inserting in MYSQL table with special column(s) a query such as:
insert into t2 values(0,4,'new04'),(0,5,'new05');
failed saying: column id (the special column) not found in t2.
It is now accepted but must be counted in values (these 0 are ignored)
- ROWID was returning row numbers based 0. Now it is from base 1.
modified:
storage/connect/array.cpp
storage/connect/blkfil.cpp
storage/connect/colblk.cpp
storage/connect/connect.cc
storage/connect/filamap.cpp
storage/connect/filamdbf.cpp
storage/connect/filamfix.cpp
storage/connect/filamtxt.cpp
storage/connect/filamvct.cpp
storage/connect/filamzip.cpp
storage/connect/filamzip.h
storage/connect/filter.cpp
storage/connect/global.h
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/libdoc.cpp
storage/connect/mycat.cc
storage/connect/myconn.cpp
storage/connect/odbconn.cpp
storage/connect/plgdbutl.cpp
storage/connect/plugutil.c
storage/connect/reldef.cpp
storage/connect/tabcol.cpp
storage/connect/tabdos.cpp
storage/connect/tabfix.cpp
storage/connect/tabfmt.cpp
storage/connect/table.cpp
storage/connect/tabmul.cpp
storage/connect/tabmysql.cpp
storage/connect/tabmysql.h
storage/connect/taboccur.cpp
storage/connect/tabodbc.cpp
storage/connect/tabpivot.cpp
storage/connect/tabsys.cpp
storage/connect/tabtbl.cpp
storage/connect/tabutil.cpp
storage/connect/tabvct.cpp
storage/connect/tabwmi.cpp
storage/connect/tabwmi.h
storage/connect/tabxcl.cpp
storage/connect/tabxml.cpp
storage/connect/user_connect.cc
storage/connect/valblk.cpp
storage/connect/value.cpp
storage/connect/value.h
storage/connect/xindex.cpp
storage/connect/xobject.cpp
storage/connect/xobject.h
storage/connect/xtable.h
Add an union in the PARM structure to contain int values
Use a cast to ptrdiff_t in MAPFAM/MXPFAM::InitDelete required by some compilers
modified:
storage/connect/array.cpp
storage/connect/filamap.cpp
storage/connect/filamtxt.cpp
storage/connect/global.h
- Suppress some GCC warnings
modified:
storage/connect/array.cpp
storage/connect/filter.cpp
storage/connect/tabdos.cpp
storage/connect/tabmysql.cpp
storage/connect/value.cpp
a temporary file should be used for UPDATE/DELETE of file tables.
Also use the "sorted" argument of index_init to help decide if
sorting of positions must be done.
modified:
storage/connect/checklvl.h
storage/connect/connect.cc
storage/connect/connect.h
storage/connect/filamdbf.cpp
storage/connect/filamfix.cpp
storage/connect/filamfix.h
storage/connect/filamtxt.cpp
storage/connect/ha_connect.cc
storage/connect/mysql-test/connect/r/part_table.result
storage/connect/plgdbsem.h
storage/connect/plgdbutl.cpp
storage/connect/reldef.cpp
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/tabfix.cpp
storage/connect/tabfmt.cpp
storage/connect/tabvct.cpp
storage/connect/tabvct.h
storage/connect/xindex.cpp
- Fix a bug in TDBASE::ColDB that caused some special columns not to
be found in the column list and reallocated without their Value
causing a crash of some queries.
modified:
storage/connect/table.cpp
- Fix a bug causing RestoreNrec to be called before closing a table
causing a wrong value given to Spos
modified:
storage/connect/tabdos.cpp
storage/connect/xindex.cpp
- Add a new CONNECT global variable connect_exact_info. Set to ON, it
tells CONNECT to return exact record numbers on info queries. If OFF
it just gives an estimate. In version 10.0.13 this was unconditionally
ON and caused info queries on remote tables to be extremely long and
was the subject of MDEV-6612.
modified:
storage/connect/ha_connect.cc
storage/connect/tabdos.cpp
storage/connect/tabmysql.cpp
storage/connect/tabodbc.cpp
have been created by the last addition of new CONNECT features.
The version previous to this one is a preliminary test version and
should not be distributed.
- Handle indexed UPDATE/DELETE. Previously this was just tested and
an error message send when it could not be done. Now CONNECT can
do it in all the cases. It is done by a MRR like tchnique by making
a list of all update or delete to do, sort them, then execute them.
modified:
storage/connect/array.cpp
storage/connect/array.h
storage/connect/filamap.cpp
storage/connect/filamap.h
storage/connect/filamdbf.cpp
storage/connect/filamfix.cpp
storage/connect/filamfix.h
storage/connect/filamtxt.cpp
storage/connect/filamtxt.h
storage/connect/filamvct.cpp
storage/connect/filamvct.h
storage/connect/filamzip.cpp
storage/connect/filamzip.h
storage/connect/global.h
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
- Differenciate Cardinality that returns a true or estimated table size
and GetMaxSize that return a value equal or greater than the table
row number. This fixes the errors of non matching opt files.
modified:
storage/connect/connect.cc
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/tabfix.cpp
storage/connect/table.cpp
storage/connect/tabmac.h
storage/connect/tabmysql.cpp
storage/connect/tabmysql.h
storage/connect/tabodbc.cpp
storage/connect/tabodbc.h
storage/connect/tabpivot.h
storage/connect/tabtbl.cpp
storage/connect/tabtbl.h
storage/connect/tabutil.cpp
storage/connect/tabutil.h
storage/connect/tabwmi.h
storage/connect/xtable.h
- Fix some errors and issues when making index and opt files.
Erase opt and index files for void tables.
Fix wrong calculation of Block and Last in MakeBlockValues.
Invalidate indexes before making opt file.
Fully handle blocked variable tables. Make opt file for blocked
variable tables even when they have no optimised colums.
modified:
storage/connect/tabdos.cpp
storage/connect/xindex.h
- Fix some errors making index
Return an error when the allocation is too small (should not
really occur now that GetMaxSize is sure)
Don't use XXROW index for DBF tables because of soft deleted lines.
modified:
storage/connect/xindex.cpp
- Typo
modified:
storage/connect/macutil.cpp
storage/connect/tabdos.h
storage/connect/tabsys.cpp
storage/connect/tabsys.h
