GitLab

The project name isn't used that often, but when trying to decide what
the component should be called in exports, it's very useful.

This refactors the VersionNum reading into another function, so that it
can be used in the same way.

The RISC OS fork of the ci-vars tool fixed a few problems, which I'm
now seeing. The ci-vars checks when run in CI wasn't checking the
origin version of the branch names, so it never found the names that
we were using.

The --null option does not do what I thought it did, so we will have
to handle the operations in a line oriented way, not match oriented.

Trying to work out why the perl code doesn't work, I've added some
extra logging which might help on the docker CI runs.

The requirements file needs to be sorted in order to be compared
sensibly.

The less than and greater than symbols were not being escaped so produced
invalid XML. update the xmlattribute escaping.

Removing the pkg-resources causes many problems. For some reason it
only appears in some installations - on my CI it needs to be present
but on the local machine it does not.

Apparently the 1.9.4 version isn't working on modern ubuntu, so it
has been upgraded.

For some reason the zip file was appearing at the front of the list
for the paths - we can omit anything that's not a directory.

tput seems to now be reporting an error, either through a change to tput
itself, or through the TERM variable not being set inside the CI.

This change makes the failure of the tput command non-fatal, but if we
are in CI, we'll instead force ANSI as we then get nice colours.

The success has already been reported, so we don't need to do so here.

The 'read' operations were missing the '-r' switch to read the '\'
as a literal character rather than an escape. This meant that the
things that were recorded might be missing the \ character.

The report_success was missing, so it wouldn't have reported that the
file was linted.

Annoying files on MacOS.

When in CI, it is common to checkout the sha in a clean repo, not checking
out a branch at all. This results in the branch not being shown in the version
strings that are generated.

Also the pydoctor invocation was reporting a different version because it
calculates the version differently. This should now have been harmonised,
but it might be good to latter use the value from the former tool.

Installation of the CPAN modules was causing a failure for the test
runs without reporting any reason. We now report the cases that failed
to install to the output, as well as the logs.

Installation of the CPAN modules was causing a failure for the test
runs without reporting any reason. We now report the cases that failed
to install to the output, as well as the logs.

The deprecations were appearing because they were given with the
ANSI colouring.

Using the YAML format seems to be a bit harder if you don't know what
you're doing wrong. Some new dedicated classes have been added for
exceptions related to the format of the YAML files and the templates.
This should hopefully give the opportunity to fix errors more easily
than without.

The output wasn't being processed because the 'Using config file' message
was confusing the output parser. In the future we probably want to be a
little more generous.

Instead of importing python libraries, which ends up with more complexity
in the paths to use, etc, we can now import the configuration as YAML
files, using a simplified YAML parser that is just able to handle the
basics of our structures - it's good enough to give us a descriptive
language that we can reuse for custom parsing.

Pylint is able to now use 2.2.2 (or 1.9.4 on python 2). This means
that we should now be able to catch more problems which are specific
to Python 3.

However, the design decision to force the 'rating' on all the reports
is not very nice for a lint report, so I'm filtering the message out
entirely.

Since the code was updated to use the new output parser, the results
of the tests were always coming out as passes. This was caused by
the lint_failed variable never being set to anything, and thus never
saying there was a failure.

The shell and xml lints now report the failure when the tests are
failing.

Oops; I'd left a debug line in that copied to ci-logs.

The ShellCheck parser now uses the state machine build in python,
which is faster and a lot move configurable. It also means that the
individual line reports are turned into testcase reports, rather than
each file being grouped as a testcase report.

We now have sufficient support in the JUnit processor to handle the
processing of the ShellCheck output. This requires a special formatting
of the ShellCheck output, but even the bare formatting should be
sufficient.

Additional rules are available which allow the parsing of other types
of information than the XML Lint output. Rules are able to give the
regular expression replacements, and resetting the parameters when
we have new formatted output to process.

This has been experimented on by the ShellCheck parsing, which will
come in a later change.

JUnitXML will now encode the newlines in attributes so that we get
well-formed XML.

The XML Lint parser is now a completely separate configuration which
should allow other parsers to be dropped in there soon.

New python processor. Works faster.

The Bash parser is dog slow for the xmllint JUnit XML generation. This
appears to be the many file manipulations and messing around parsing
the content. On the server it is taking about 15 minutes to process the
1000 errors (down from 50 minutes to process them, after the code was
optimised and 1000 were fixed).

Python setup reports some deprecation warnings that we can safely ignore
for the purposes of the environment setup.

Invocation tests would not produce the right output if the input for
those tests contained spaces at the start of the lines or had escaped
characters in them. This was caused by misused 'read' commands, without
the -r switch, and not clearing IFS beforehand.

Additionally, add the 'dim' colouring option to our 'say' function.

On entering the python environment, the errors are reported more
obviously now when we are already in a virtualenv.

Rather than doing the CDATA inline, this is now handled by a function,
which is a lot tidier.