> After May 31, 2024, CentOS Stream 8 will be archived and no further
updates will be provided. [1]
CentOS Stream 8 is now EOL and should be updated to using CentOS Stream
9 for compatibility testing in GitLab CI.
[1] https://blog.centos.org/2023/04/end-dates-are-coming-for-centos-stream-8-and-centos-linux-7/https://www.centos.org/centos-linux-eol/
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services.
Line numbers had to be removed from the ignorelists in order to be
diffed against since locations of the same findings can differ
across runs. Therefore preprocessing has to be done on the CI findings
so that it can be compared to the ignorelist and new findings can be
outputted. However, since line numbers have to be removed, a situation
occurs where it is difficult to reference the location of findings
in code given the output of the CI job.
To lessen this pain, change the cppcheck template to include
code snippets which make it easier to reference where in the code
the finding is referring to, even in the absence of line numbers.
Ignorelisting works as before since locations of the finding may
change but not the code it is referring to.
Furthermore, due to the innate difficulty in maintaining ignorelists
across branches and triaging new findings, allow failure as to not
have constantly failing pipelines as a result of a new findings that
have not been addressed yet.
Lastly, update SAST ignorelists to match the newly refactored cppcheck
job and the current state of the codebase.
All new code of the whole pull request, including one or several
files that are either new files or modified ones, are contributed
under the BSD-new license. I am contributing on behalf of my
employer Amazon Web Services, Inc.
Add .gitlab-ci.yml file to earliest supported branch to enable
automated building and testing for all MariaDB major branches.
Note to mergers:
GitLab CI is available for branches >= 10.6. This commit includes a
GitLab CI file identical to that in branches >= 10.6, except for the
MARIADB_MAJOR_VERSION variable which should reflect the branch version.
A modified CI will be included in branches 10.4 with PR !2418.
Also changed is the `allow_failure: true` for the MSAN build,
which should be merged up to later branches.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
Note to mergers: Do not merge this commit to 10.5+. An additional PR
will be created for the 10.5 branch which is compatible with later
branches.
Include cppcheck and FlawFinder for SAST scanning.
From 10.6, cherry-picked 12bf5c46 (Remove unused French translations in
Connect engine) and c6072ed9 (Ensure that source files contain only
valid UTF8 encodings). Necessary for FlawFinder to execute and useful
anyway.
Removing MSAN build and test as it was not introduced until 10.5 and
does not successfully build.
Remove failing upgrade test since Fedora installs MariaDB 10.5 and the
10.5->10.4 upgrade rightfully complains
Add to skiplist failing test: main.func_math (MDEV-20966)
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
Add .gitlab-ci.yml file to earliest supported branch to enable
automated building and testing for all MariaDB major branches.
For 10.4 we use the bundled SSL to build MariaDB when the platform
does not have OpenSSL 1.1 available. This requires the installation of
gnutls-devel as a dependency of MariaDB Connector/C.
OpenSSL 3.0 support was backported to MariaDB 10.5
(see https://github.com/MariaDB/server/pull/2036, f0fa40ef, 8a9c1e9c)
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
Sanitizer tests were introduced in 617f45b for GitLab CI, but started
failing on latest Fedora version with error:
$ yum install -y /usr/lib64/libasan.so.6.0.0 /usr/lib64/libtsan.so.0.0.0 /usr/lib64/libubsan.so.1.0.0
Last metadata expiration check: 0:00:51 ago on Fri Dec 9 20:05:01 2022.
No match for argument: /usr/lib64/libasan.so.6.0.0
No match for argument: /usr/lib64/libtsan.so.0.0.0
Error: Unable to find a match: /usr/lib64/libasan.so.6.0.0 /usr/lib64/libtsan.so.0.0.0
The reason for using specific library versions is unknown. Switch to
simply using latest package versions, as is works and is likely to work
best in the long run.
Also, enclose "../rpmlist-$CI_JOB_NAME-$CI_COMMIT_REF_SLUG.log" in
quotes to avoid `ambiguous redirect` error when $CI_JOB_NAME has spaces.
Additionally use "needs" statements to allow tests to run immediately
after dependent jobs passed instead of waiting for the full stage to
complete.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the BSD-new
license. I am contributing on behalf of my employer Amazon Web Services
The version was fixed to be Fedora 36 due to previous issues on
Gitlab-CI, but those seem to be solved now.
Use 'mariadb' name in scripts and server binary as Fedora switched name in
df76620f9e
Switch to using the `default:` section supported by newer Gitlab-CI,
see https://docs.gitlab.com/ee/ci/yaml/#default.
Also define an explicit timeout of 3 hours to ensure builds don't time
out if the default timeout is too short.
NOTE TO MERGERS: These changes are version independent and should be
merged up on all MariaDB branches 10.6 -> 10.11.
Add a build and test job for each of ASAN, MSAN, TSAN, and UBSAN to the
GitLab pipeline such that current vulnerabilities will be more easily
visible and on each new commit, we can ensure that there are no
additional errors introduced. Furthermore, sanitizer test runs are run
separate from the existing mysql-test-run to isolate sanitizer error
from functional errors.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
In previous versions it was stated that MDEV-25968 was causing other
jobs in the pipeline to fail if not run with "-j 2" but this bug was not
affecting fedora-ninja. This is still true for the public gitlab runners.
However, running the fedora-ninja job on custom runners with more processors
without the "-j 2" flag will cause the compiler to crash.
When running the build with 2,4,8,16,32 threads, build times were
consistent indicating that the typical bottleneck is I/O and not CPU
cores. Therefore, "-j 2" is not a big drawback and it was chosen in
order to remain consistent with the other builds affected by MDEV-25968.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
The commits a73acf6c06 and
4d74bac8bc updated the PCRE library to a new
version, which in turn requires CMake 3.0. That does not exist in CentOS 7
nor 8, so builds started failing.
Actually the build should not be downloading anything at all. The root
cause was that pcre2-devel was missing from the dependencies. This was
originally not detected, as the download fallback had masked the issue.
- Add new Ninja and Clang build jobs. This helps to ensure those
toolchains also work in addition to default CMake/gcc.
- Generate dependencies.dot/png to illustrate the CMake/Make/Ninja
build dependencies. Viewing this image and identifying bottle necks
in parallelism can help make the build run faster.
- Enable CUnit tests now as they are fixed on 10.6 (MDEV-25820).
- Limit parallel builds to 2 CPUs (full parallelism needs MDEV-25968) on
CMake/Make. Now only the Ninja builds run full parallel builds as only
Ninja is smart enough to prevent builds failing on resource
over-consumption.
- Enable Gitlab-CI cache for job 'centos8' for ccache so that it builds
faster. Don't use Gitlab-CI cache for other jobs, as it would too easily
use up all free tier storage on Gitlab.com and force users to get a paid
account just for MariaDB builds.
- On other jobs clean away ccache, as it only had a 5% hit rate on single
builds with no downloaded cache.
- Dump full database contents during the test install so that one can
use diff to compare the database contents at different stages and thus
track/debug potential bugs in mariadb-install-db and mariadb-upgrade
code.
Bugfixes:
- Zero out ccache stats before each run so that 'ccache -s' would actually
show the stats for the latest run.
As Travis-CI has stopped offering free testing for open source projects,
and they don't seem to have any plans to revert their new restrictions,
MariaDB no longer has a good CI system outside contributors could run
independently for basic validation before submitting Pull Requests.
Implement a simple Gitlab-CI pipeline that runs basic RPM builds on
one old, one less old and one very new distro release and then do some
basic tests on the RPM packages to validate they installed and the
server actually runs.
