Commit graph

9 commits

Author SHA1 Message Date
Sergei Golubchik
897b51db43 make S3 tests to run when S3 is statically linked
* use the environment variable HA_S3_SO, not a literal ha_s3 in cnf files
* make ConfigFactory to support empty option values
* update no_s3.result after MDEV-11412
2020-09-10 13:32:27 +02:00
Daniel Black
f98b8d36bd MDEV-23662: Make S3 Storage Engine tests better
To ensure that S3 is using its access and secret
key in the right way, make these different in the test
suite.
2020-09-08 13:42:55 +03:00
Daniel Black
64c10b03a0 MDEV-23662: Make S3 Storage Engine tests usable
added plugin-maturity = alpha to s3 mtr cnf files
and a missing  plugin-load-add=ha_s3 in the slave
configuration.
2020-09-08 13:42:55 +03:00
Sergei Petrunia
e38ac39104 MDEV-23662: Make S3 Storage Engine usable with MinIO
Add @@s3_port and @@s3_use_http. One can set these to access the
local MinIO, for example.
2020-09-08 13:42:55 +03:00
Sergei Golubchik
35034d819c S3 is pluggable now 2020-06-19 19:43:07 +02:00
Monty
eca5c2c67f Added support for more functions when using partitioned S3 tables
MDEV-22088 S3 partitioning support

All ALTER PARTITION commands should now work on S3 tables except

REBUILD PARTITION
TRUNCATE PARTITION
REORGANIZE PARTITION

In addition, PARTIONED S3 TABLES can also be replicated.
This is achived by storing the partition tables .frm and .par file on S3
for partitioned shared (S3) tables.

The discovery methods are enchanced by allowing engines that supports
discovery to also support of the partitioned tables .frm and .par file

Things in more detail

- The .frm and .par files of partitioned tables are stored in S3 and kept
  in sync.
- Added hton callback create_partitioning_metadata to inform handler
  that metadata for a partitoned file has changed
- Added back handler::discover_check_version() to be able to check if
  a table's or a part table's definition has changed.
- Added handler::check_if_updates_are_ignored(). Needed for partitioning.
- Renamed rebind() -> rebind_psi(), as it was before.
- Changed CHF_xxx hadnler flags to an enum
- Changed some checks from using table->file->ht to use
  table->file->partition_ht() to get discovery to work with partitioning.
- If TABLE_SHARE::init_from_binary_frm_image() fails, ensure that we
  don't leave any .frm or .par files around.
- Fixed that writefrm() doesn't leave unusable .frm files around
- Appended extension to path for writefrm() to be able to reuse to function
  for creating .par files.
- Added DBUG_PUSH("") to a a few functions that caused a lot of not
  critical tracing.
2020-04-19 17:33:51 +03:00
Monty
6a9e24d046 Added support for replication for S3
MDEV-19964 S3 replication support

Added new configure options:
s3_slave_ignore_updates
"If the slave has shares same S3 storage as the master"

s3_replicate_alter_as_create_select
"When converting S3 table to local table, log all rows in binary log"

This allows on to configure slaves to have the S3 storage shared or
independent from the master.

Other thing:
Added new session variable '@@sql_if_exists' to force IF_EXIST to DDL's.
2020-03-24 21:00:02 +02:00
Monty
c51f615bf5 Added options --s3-protocol-version and --s3-host-name 2019-06-01 15:21:50 +03:00
Monty
ab38b7511b MDEV-17841 S3 storage engine
A read-only storage engine that stores it's data in (aws) S3

To store data in S3 one could use ALTER TABLE:
ALTER TABLE table_name ENGINE=S3

libmarias3 integration done by Sergei Golubchik
libmarias3 created by Andrew Hutchings
2019-05-23 02:28:23 +03:00