dpkg-reconfigure cloud-init-base always reports: Removing cloud-init/datasources in favor of cloud-init-base/datasources

[blackboxsw]

Bug report

debian/cloud-init-base.postinst has a migrate_debconf_to_cloud_init_base that attempts to migrate old package debconf values from cloud-init/datasources to cloud-init-base/datasources. It attempts to perform a db_unregister of the old debconf setting once the migration and db_set of cloud-init-base/datsources is performed. Yet, no matter how many times you run dpkg-reconfigure, that message is printed, leading me to believe the db_unregister cloud-init/datasources wasn't enough or persisted.

Steps to reproduce the problem

git checkout upstream/ubuntu/devel
change d/changelog from UNRELEASED to oracular
git commit -am 'pkg build test'
build-package; sbuild ....oracular ... ./out/dsc
lxc launch ubuntu-daily:oracular test-o
lxc file push cloud-initdeb test-o/
lxc exec test-o -- dpkg -i /cloud-init_*deb /cloud-init-base_*deb
dpkg-reconfigure cloud-init-base

Environment details

• Cloud-init version: unreleased
• Operating System Distribution: Ubuntu devel P-series
• Cloud provider, platform or installer type: lxd

cloud-init logs

root@aciba-o-new-deb:~# dpkg-reconfigure cloud-init-base -f noninteractive
Removing cloud-init/datasources in favor of cloud-init-base/datasources
root@aciba-o-new-deb:~# dpkg-reconfigure cloud-init-base -f noninteractive
Removing cloud-init/datasources in favor of cloud-init-base/datasources

[aciba90]

Tracking in SC-1967.

[aciba90]

Thanks, @blackboxsw, for reporting this issue.

I am able to reproduce it by:

root@ovm3:/home/ubuntu# export DEBCONF_DEBUG=developer

root@ovm3:/home/ubuntu# apt policy cloud-init cloud-init-base                                             
cloud-init:
  Installed: 24.4~3+really24.3.1-0ubuntu4
  Candidate: 24.4~3+really24.3.1-0ubuntu4
  Version table:
 *** 24.4~3+really24.3.1-0ubuntu4 500
        500 http://archive.ubuntu.com/ubuntu oracular/main amd64 Packages
        100 /var/lib/dpkg/status
Notice: Unable to locate package cloud-init-base

root@ovm3:/home/ubuntu# debconf-show cloud-init                                                           
  cloud-init/datasources: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, E
xoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, None

root@ovm3:/home/ubuntu# debconf-show cloud-init-base

root@ovm3:/home/ubuntu# apt install ./cloud-init-base_24.4~4gc9dce94d-0ubuntu2~ppa2_all.deb                                                                                                                        Note, selecting 'cloud-init-base' instead of './cloud-init-base_24.4~4gc9dce94d-0ubuntu2~ppa2_all.deb'
Installing:                         
  cloud-init-base                                                                                        
                                                    
REMOVING:                   
  cloud-init                 
                                                                                                         
Summary:                            
  Upgrading: 0, Installing: 1, Removing: 1, Not Upgrading: 0
  Download size: 0 B / 613 kB                                                                            
  Freed space: 11.3 kB                                                                                   
                                                                                                         
Continue? [Y/n]                                                                                          
Get:1 /home/ubuntu/cloud-init-base_24.4~4gc9dce94d-0ubuntu2~ppa2_all.deb cloud-init-base all 24.4~4gc9dce94d-0ubuntu2~ppa2 [613 kB]
Preconfiguring packages ...                                                                              
debconf (developer): starting /var/cache/debconf/tmp.ci/cloud-init-base.config.AwMdAU configure 
debconf (developer): <-- GET cloud-init-base/datasources                      
debconf (developer): --> 0 NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, 
Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, CloudCIX, None
debconf (developer): <-- GET cloud-init/datasources                                                      
debconf (developer): --> 0 NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, 
Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, None
Removing cloud-init/datasources in favor of cloud-init-base/datasources                                                                                                                                            debconf (developer): <-- UNREGISTER cloud-init/datasources              
debconf (developer): --> 0                                                                               
debconf (developer): <-- SET cloud-init-base/datasources NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStac
k, Hetzner, IBMCloud, Oracle, Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, None
debconf (developer): --> 0 value set                                                                     
debconf (developer): <-- INPUT low cloud-init-base/datasources 
debconf (developer): --> 30 question skipped                                                             
debconf (developer): <-- GO                                                                              
debconf (developer): --> 0 ok                                                                            
(Reading database ... 76019 files and directories currently installed.)
Removing cloud-init (24.4~3+really24.3.1-0ubuntu4) ... 
Selecting previously unselected package cloud-init-base.
(Reading database ... 75636 files and directories currently installed.)
Preparing to unpack .../cloud-init-base_24.4~4gc9dce94d-0ubuntu2~ppa2_all.deb ...
Unpacking cloud-init-base (24.4~4gc9dce94d-0ubuntu2~ppa2) ...
Setting up cloud-init-base (24.4~4gc9dce94d-0ubuntu2~ppa2) ...
debconf (developer): frontend started                                                                                                                                                                              
debconf (developer): frontend running, package name is cloud-init-base                                                                                                                                             
debconf (developer): starting /var/lib/dpkg/info/cloud-init-base.config configure                                                                                                                                  
debconf (developer): <-- GET cloud-init-base/datasources                                                                                                                                                           
debconf (developer): --> 0 NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, 
Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, None                                                                                                                                           
debconf (developer): <-- GET cloud-init/datasources                                                                                                                                                                
debconf (developer): --> 0 NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, 
Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, None                                                                                                                                           
Removing cloud-init/datasources in favor of cloud-init-base/datasources                                                                                                                                            
debconf (developer): <-- UNREGISTER cloud-init/datasources                                                                                                                                                         
debconf (developer): --> 0                                                                                                                                                                                         
debconf (developer): <-- SET cloud-init-base/datasources NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStac
k, Hetzner, IBMCloud, Oracle, Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, None                                                                                                             
debconf (developer): --> 0 value set                                                                                                                                                                               
debconf (developer): <-- INPUT low cloud-init-base/datasources                                                                                                                                                     
debconf (developer): --> 30 question skipped                                                             
debconf (developer): <-- GO                                                                              
debconf (developer): --> 0 ok                                                                                                                                                                                      
debconf (developer): starting /var/lib/dpkg/info/cloud-init-base.postinst configure                      
debconf (developer): <-- CAPB escape                                                                                                                                                                               
debconf (developer): --> 0 multiselect escape backup                    
debconf (developer): <-- GET cloud-init-base/local-cloud-config                                          
debconf (developer): --> 10 cloud-init-base/local-cloud-config doesn't exist                                                                                                                                       
debconf (developer): <-- GET cloud-init/local-cloud-config                                               
debconf (developer): --> 10 cloud-init/local-cloud-config doesn't exist                                  
debconf (developer): <-- GET cloud-init-base/maas-metadata-credentials                                                                                                                                             
debconf (developer): --> 10 cloud-init-base/maas-metadata-credentials doesn't exist                      
debconf (developer): <-- GET cloud-init/maas-metadata-credentials                                        
debconf (developer): --> 10 cloud-init/maas-metadata-credentials doesn't exist                           
debconf (developer): <-- GET cloud-init-base/maas-metadata-url                                                                                                                                                     
debconf (developer): --> 10 cloud-init-base/maas-metadata-url doesn't exist       
debconf (developer): <-- GET cloud-init/maas-metadata-url                                                
debconf (developer): --> 10 cloud-init/maas-metadata-url doesn't exist                                                                                                                                             
debconf (developer): <-- GET cloud-init-base/datasources                         
debconf (developer): --> 1 NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, 
Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, None
debconf (developer): <-- GET cloud-init-base/maas-metadata-url                                           
debconf (developer): --> 10 cloud-init-base/maas-metadata-url doesn't exist                                                                                                                                        
debconf (developer): <-- GET cloud-init-base/maas-metadata-credentials                                
debconf (developer): --> 10 cloud-init-base/maas-metadata-credentials doesn't exist                      
debconf (developer): <-- GET cloud-init-base/local-cloud-config                                                                                                                                                    
debconf (developer): --> 10 cloud-init-base/local-cloud-config doesn't exist                             
debconf (developer): <-- UNREGISTER cloud-init-base/local-cloud-config                                   
debconf (developer): --> 10 cloud-init-base/local-cloud-config doesn't exist                                                                                                                                       
Processing triggers for man-db (2.12.1-3) ...                                                            
Processing triggers for rsyslog (8.2406.0-1ubuntu2) ...                
Scanning processes...
Scanning linux images...                                                                                                                                                                                           

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.
Notice: Download is performed unsandboxed as root as file '/home/ubuntu/cloud-init-base_24.4~4gc9dce94d-0ubuntu2~ppa2_all.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)

root@ovm3:/home/ubuntu# debconf-show cloud-init
  cloud-init/datasources: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, None

root@ovm3:/home/ubuntu# debconf-show cloud-init-base
  cloud-init-base/datasources: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, None

The issue happens because on a package upgrade to cloud-init-base, cloud-init-base correctly adopts the cloud-init debconf values, but the db_unregister doesn't produce the effect that we want, let me explain myself.

What we would like to happen is to call db_unregister cloud-init/datasources and have cloud-init/datasources gone from the debconf db. But, the calls to db_* are localized to the package's maintainer scripts owner, that means that the db_unregister calls from cloud-init-base's maintainer scripts tell debconf that cloud-init-base is not anymore an owner of cloud-init/datasources, but cloud-init is still an owner of it[1], and then it is not removed from the db.

[1] This happens because, by default, a call to db_purge only happens when its package is purged. We can see that if we purge cloud-init, then the debconf values are gone:

root@ovm3:/home/ubuntu# export DEBCONF_DEBUG=developer

root@ovm3:/home/ubuntu# debconf-show cloud-init
* cloud-init/datasources: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, None
root@ovm3:/home/ubuntu# debconf-show cloud-init-base
root@ovm3:/home/ubuntu# apt remove cloud-init
REMOVING:                       
  cloud-init

Summary:
  Upgrading: 0, Installing: 0, Removing: 1, Not Upgrading: 0
  Freed space: 3122 kB

Continue? [Y/n] y
(Reading database ... 76019 files and directories currently installed.)
Removing cloud-init (24.4~3+really24.3.1-0ubuntu4) ...
Processing triggers for man-db (2.12.1-3) ...

root@ovm3:/home/ubuntu# debconf-show cloud-init-base

root@ovm3:/home/ubuntu# apt remove cloud-init
Package 'cloud-init' is not installed, so not removed
Summary:
  Upgrading: 0, Installing: 0, Removing: 0, Not Upgrading: 0
root@ovm3:/home/ubuntu# debconf-show cloud-init
* cloud-init/datasources: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, Exoscale, RbxCloud, UpCloud, VMware, Vultr, LXD, NWCS, Akamai, WSL, None

root@ovm3:/home/ubuntu# apt purge cloud-init
REMOVING:                       
  cloud-init*

Summary:
  Upgrading: 0, Installing: 0, Removing: 1, Not Upgrading: 0
  Space needed: 0 B / 7465 MB available

Continue? [Y/n] Y
(Reading database ... 75636 files and directories currently installed.)
Purging configuration files for cloud-init (24.4~3+really24.3.1-0ubuntu4) ...
debconf (developer): frontend started
debconf (developer): frontend running, package name is cloud-init
debconf (developer): starting /var/lib/dpkg/info/cloud-init.postrm purge
debconf (developer): <-- PURGE 
debconf (developer): --> 0
dpkg: warning: while removing cloud-init, directory '/etc/cloud' not empty so not removed
Processing triggers for rsyslog (8.2406.0-1ubuntu2) ...

root@ovm3:/home/ubuntu# debconf-show cloud-init

To fix this issue, we would wish to be able to unregister cloud-init as an onwer of cloud-init/datasources from a cloud-init-base maintainer script, because we cannot know if a user will install both cloud-init-base and cloud-init, we cannot rely on cloud-init's maintainer scritps fully unregister the values.

I would propose to just remove the db_unregister debugging message and live with it, as a purge of cloud-init will remove the debconf values and it doesn't harm to have them there.