Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chad #19

Merged
merged 153 commits into from
Mar 26, 2024
Merged

Chad #19

merged 153 commits into from
Mar 26, 2024

Conversation

blackboxsw
Copy link
Owner

Proposed Commit Message

summary: no more than 70 characters

A description of what the change being made is and why it is being
made, if the summary line is insufficient.  The blank line above is
required. This should be wrapped at 72 characters, but otherwise has
no particular length requirements.

If you need to write multiple paragraphs, feel free.

Fixes: #NNNNN (GitHub Issue number. Remove line if irrelevant)
LP: #NNNNNN (Launchpad bug number. Remove line if irrelevant)

Additional Context

Test Steps

Checklist:

  • My code follows the process laid out in the documentation
  • I have updated or added any unit tests accordingly
  • I have updated or added any documentation accordingly

aciba90 and others added 30 commits February 16, 2024 13:19
verify_clean_log assumed shadowed warning messages to be contained in a
single line entry with a `[WARNING]`, but warnings can be multiline as:

```
2024-02-16 15:01:49,354 - activators.py[WARNING]: Running ['netplan', 'apply'] resulted in stderr output: Cannot call Open vSwitch: ovsdb-server.service is not running.
Failed to connect system bus: No such file or directory
Falling back to a hard restart of systemd-networkd.service
```

wrongly asserting the number of expected_warnings.

Make this check more reliable by asserting that the number of lines
containing `WARNINGS` is less or equal than the number of appearances of
the shadowed warning messages.
Create generalized ubuntu_pro.properties definition to be shared
in the ubuntu_pro and ubuntu_advantage property schemas as they are
identical schema definitions.

Given that ubuntu_advantage is now deprecated, add the schema keys
deprecated, deprecated_version and deprecated description to provide
descriptive warning logs when deprecated keys are present in user-data.
Update cc_ubuntu_advantage module, comments, logs and examples
to prefer ubuntu_pro key over ubuntu_advantage to align with the updated
product naming.

- Deprecate ubuntu_advantage key and represent ubuntu_pro is prioritized
- Update product URL to ubuntu.com/pro instead of /advantage
- Update log messaging and examples to reference Ubuntu Pro instead of
  Ubuntu Advantage
- Add logic to error when both ubuntu_advantage and ubuntu-advantage keys
  are present in userdata.
Move the cc_ubuntu_advantage module to cc_ubuntu_pro, updating
schema definitions to align with new module name and default
/etc/cloud/cloud.cfg to represent ubuntu_pro instead of ubuntu_advantage
module.
Also update stale /etc/cloud/cloud.cfg example adding new
config modules present in tip of tree.
Reverts 4dbb08f to not change behavior
on stable releases.
Avoids LP: #1946003 on upgraded systems.

References:

[0] canonical#4799
[1] canonical@b519d86

Co-authored-by: Chad Smith <[email protected]>
…#4906)

The files in the sysvinit/gentoo directory are OpenRC files not
specific to Gentoo and are used by other distro (i.e. Alpine).
On focal we detect NoCloud rather than LXD, so ensure the test reflects
this.
Saw "cat: /before-local.start-nostatusjson: No such file or directory"
failures. After this change, I no longer see these failures.
…onical#4876)

Unlike the majority of other Linux distros /dev/mapper entries in
Alpine Linux are NOT symlinks to /dev/dm-X block devices but rather are
actual block devices themselves. This is a side-effect of Alpine's use
of mdev, rather than udev, in its initramfs.

Therefore this change works out the corresponding /dev/dm-X for any
/dev/mapper/XYZ device on Alpine.
isc-dhcp-client is no longer supported by upstream

BREAKING CHANGE: dhcpcd has been promoted to the default dhcp client.
Previous behavior can be configured in cloud.cfg via the dhcp_client_priority
key.
cc_apk_configure uses temp_utils.py, which has special logic to return
if the user is root that can't be retargeted using our current fixtures.
Fix it by setting that tmpdir in the test setup.
On Ubuntu Noble dhcpcd-base is a hard package Depends and
also not included by default in ubuntu-minimal images until
cloud-init the new cloud-init images contain latest cloud-init.

When provided with a DEB_PATH source, use apt instead of dpkg -i
to any missing install package dependencies.

Fixes: canonicalGH-4908
A new datasource is implemented for the WSL platform.
It relies on being able to access the Windows filesystem and execute cmd.exe
to determine where the user data configuration files are.
Reference documentation for that datasource is also added, explaining where
the user should place the config files, the requirements for the datasource to work
and some details that are expected to work differently or not at all.
I'm not sure that list is exhaustive, so we should expect revisions to it in the short future.
For now at least the implementation is focused on locating and ingesting user-data,
without being too smart.

Closes UDENG-1989
verify_clean_log assumed shadowed warning messages to be contained in a
single line entry with a `[WARNING]`, but warnings can be multiline as:

```
2024-02-16 15:01:49,354 - activators.py[WARNING]: Running ['netplan', 'apply'] resulted in stderr output: Cannot call Open vSwitch: ovsdb-server.service is not running.
Failed to connect system bus: No such file or directory
Falling back to a hard restart of systemd-networkd.service
```

wrongly asserting the number of expected_warnings.

Make this check more reliable by asserting that the number of lines
containing `WARNINGS` is less or equal than the number of appearances of
the shadowed warning messages.
Create generalized ubuntu_pro.properties definition to be shared
in the ubuntu_pro and ubuntu_advantage property schemas as they are
identical schema definitions.

Given that ubuntu_advantage is now deprecated, add the schema keys
deprecated, deprecated_version and deprecated description to provide
descriptive warning logs when deprecated keys are present in user-data.
Update cc_ubuntu_advantage module, comments, logs and examples
to prefer ubuntu_pro key over ubuntu_advantage to align with the updated
product naming.

- Deprecate ubuntu_advantage key and represent ubuntu_pro is prioritized
- Update product URL to ubuntu.com/pro instead of /advantage
- Update log messaging and examples to reference Ubuntu Pro instead of
  Ubuntu Advantage
- Add logic to error when both ubuntu_advantage and ubuntu-advantage keys
  are present in userdata.
Move the cc_ubuntu_advantage module to cc_ubuntu_pro, updating
schema definitions to align with new module name and default
/etc/cloud/cloud.cfg to represent ubuntu_pro instead of ubuntu_advantage
module.
Also update stale /etc/cloud/cloud.cfg example adding new
config modules present in tip of tree.
…#4906)

The files in the sysvinit/gentoo directory are OpenRC files not
specific to Gentoo and are used by other distro (i.e. Alpine).
On focal we detect NoCloud rather than LXD, so ensure the test reflects
this.
Saw "cat: /before-local.start-nostatusjson: No such file or directory"
failures. After this change, I no longer see these failures.
…onical#4876)

Unlike the majority of other Linux distros /dev/mapper entries in
Alpine Linux are NOT symlinks to /dev/dm-X block devices but rather are
actual block devices themselves. This is a side-effect of Alpine's use
of mdev, rather than udev, in its initramfs.

Therefore this change works out the corresponding /dev/dm-X for any
/dev/mapper/XYZ device on Alpine.
isc-dhcp-client is no longer supported by upstream

BREAKING CHANGE: dhcpcd has been promoted to the default dhcp client.
Previous behavior can be configured in cloud.cfg via the dhcp_client_priority
key.
TheRealFalcon and others added 29 commits March 14, 2024 08:23
patches:
debian/patches/keep-dhclient-as-priority-client.patch
debian/patches/retain-old-groups.patch
debian/patches/revert-551f560d-cloud-config-after-snap-seeding.patch
We were ignoring orig.tar.gz, but sbuild creates a .tar.gz
Support the explicit rejection of router advertisements in cases where
the network may be configured to send them, but they are not desireable,
for example, in a case where static IPv4 addressing is used.

Before this change, IPv6AcceptRA is only included in the rendered
networkd files if DHCP is used.

Now, if a boolean value for accept-ra is present, it will be included in
the rendered output, regardless of the configured addressing.

Fixes: canonical#4927
reportedly, passing `homedir` to a user struct doesn't actually change
the homedir to the desired value. This is because we hard-code the path.
Respect the value we are passed, and only make up a default when we
don't get a `homedir`.

Simplify style to remove cerimonious verbosity around strings, and amend
tests to verify what we did.

Sponsored by: The FreeBSD Foundation
- DI_MAIN: if DI_MAIN isn't a builtin function, exec() it
- /usr/libexec/ds-identify-env may be sourced to set variables

The purpose of these changes is to improve ds-identify testing capabilities and to allow testing alternative ds-identify implementations. Use of these capabilities is not supported in cloud-init deployments and therefore generates warnings in the ds-identify log.

Systemd gives generators no ability to run with custom environment variables. To set ds-identify variables at runtime, sourcing an environment file is required.

Example no-op side-loading with env var:

    $ DI_MAIN='echo' ./tools/ds-identify "hello world"
    WARN: side-loading alternate implementation: [echo]
    hello world

Example no-op side-loading with environment variable config:

    $ PATH_ROOT=. ./tools/ds-identify "test"
    WARN: loading environment file [./usr/libexec/ds-identify-env]
    WARN: side-loading alternate implementation: [echo]
    test
    $ cat ./usr/libexec/ds-identify-env
    export DI_MAIN=echo

Example side-loader that can only identify one cloud:

    $ cat nocloud-identifier.sh
    #!/bin/sh
    echo "creating nocloud config"
    echo "datasource_list: [ NoCloud ]" > cloud.cfg
    $ DI_MAIN=./nocloud-identifier.sh ./tools/ds-identify
    WARN: side-loading alternate implementation: [./nocloud-identifier.sh]
    creating nocloud config
    $ cat cloud.cfg
    datasource_list: [ NoCloud ]
…nonical#5068)

Incorrectly registering DataSourceMAAS for init-local DEP_FILESYSTEM
inadvertently short-circuits the _get_data checks performed by
DataSourceMAASLocal for presence of network config from initramfs.
This results in a 2 minute timeout on physical system
reboots because the network stage DataSourceMAAS expects all system
networking is already configured and the IMDS to be accessible.

Correct this registration to do the proper initramfs checks and avoid
trying to perform GETs against the IMDS in init-local timeframe when
we know network is not yet available.

Fixes canonicalGH-5064
LP: #2057763
Correct `subnet-ipv6-cidr-blocks` instead of `subnet-ipv6-cidr-block`.
`routes` and `routing-policy` not overwritten in dual-stacks.
It is passed through to our v1 schema from OpenStack network_data.json

Fixes canonicalGH-5051
fix: Fix breaking changes in package install

Ensure:
* Apt can still install packages that contain `-`, `/`, or `=`
* If a specified package manager fails we don't retry as a generic
  package
* We do not attempt to invoke a package manager that doesn't exist
* Packages that are unable to be installed are tracked and
  reported appropriately

Fixes canonicalGH-5066
Mocking doesn't work when we import functions directly.
These are the failures I could find when running tests in random order.
Changes that aren't self-explanatory should include a comment in the
test itself.

test_vultr.py includes a refactor to remove global state and remove
a test that didn't work and wasn't needed after the changes.
When subp function is imported directly, it cannot be easily mocked in
unit tests. Update imports and tests accordingly.
The fixture parametrization ability added in 9baf31c doesn't work
as expected. When you have a session-wide fixture, the setup is run
once, then further invocations of the fixture (including autouse) uses a
cached version of the fixture.  Teardown for the session fixture happens
at the end of all test runs. This also applies to mock patching. Since
the mock patching happens only once, parametrizing the fixture to yield
without patching doesn't undo the initial mock setup; the
parametrization of `disable_sys_net` effectively does nothing.

The good news is that patches stack, so current tests that patch
`get_sys_class_path` differently will still work fine. If we need to
disable the patching entirely, that is also possible by saving the
original `get_sys_class_path` before applying the global disable mock,
then having a separate mock that has a side effect of calling
the original function.
Bump the version in cloudinit/version.py to 24.1.2 and
update ChangeLog.
Build time dependency was already removed and changelog already updated
patches:
debian/patches/expire-on-hashed-users.patch
debian/patches/revert-551f560d-cloud-config-after-snap-seeding.patch
@blackboxsw blackboxsw merged commit fecf21e into ubuntu/focal Mar 26, 2024
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.