Dataverse v4.20 on K8s

This release contains references to the upstream release 4.20 and will deploy it when used.

📖 The Cloud & Cointainer guide v4.20

🔍 Pre-Deploy Actions

No special action seems to be necessary regarding to the upstream release docs.

📢 Post-Deploy Actions

• You will need to adapt your storage configurations due to the new upstream feature of multiple storage locations.
  See upstream release notes for details and change your options accordingly.
• The core bundled citation metadata block has changed. Simply follow the guide about "Metadata Blocks Job".

As always: Dataverse has release new functionality with this release. Be sure to read the upstream release notes, too.

🎁 New and changed functionality

See also v4.20 milestone

• Dataverse: support new multi storage location feature. See #196 and #195
• Dataverse: update to v4.20. See #183
• Guide: refactor to have a more streamlined usage experience for demo, development and production use. #160
• Guide: add hints how to use with cloud providers. #108, #95
• Personas: add a proof of concept docker-compose based demo option. #118
• Personas: add an example how to use this project with an S3 compatible storage. #195, #51
• Add Heap dump handler on OOM as a sidecar and provide loads of docs on memory usage. #204

🐛 Bug fixes, 🧠 workarounds and 🦎 tweaks

• Stop empty ConfigMaps when using kubectl apply. #194
• Update tini in container image, making builds more stable. #187

🤖 General Notes

Since this release, this project lives in the bosom of the big Global Dataverse Community Consortium family. (#199)
And we moved from the pattern of unstable master to a stable branch release and unstable branch develop, respecting inclusive language and streamlining with IQSS workflow for Dataverse. (#198)

Please be aware that future enhancements to scripting and more used to deploy and configure Dataverse will not be added to the release. Thus docker images for this will not be updated.

Please be aware, that tagged images are not automatically updated by Kubernetes when updated on Docker Hub (image pull policy). See the guide for ideas.

Dataverse v4.19 on K8s

This release contains references to the upstream release 4.19 and will deploy it when used.

📖 The Cloud & Cointainer guide v4.19

🔍 Pre-Deploy Actions

No special action seems to be necessary regarding to the upstream release docs.

📢 Post-Deploy Actions

• The core bundled citation metadata block has changed. Simply follow the guide about "Metadata Blocks Job".

As always: Dataverse has release new functionality with this release. Be sure to read the upstream release notes, too.

🎁 New and changed functionality

See also v4.19 milestone

• Dataverse: start supporting appserver Payara 5 based container images. See also #181
  (Please note: upstream considers this as experimental use! This is mostly intended for development usage.)
• Dataverse: update to v4.19. See #175
• Dataverse: adding auth provider persistance, introducing "Day 3" about integrations to the guide. See #174
• Solr: update to 7.7.2. Officially supported by upstream as of 4.20. See #169
• Bootstrap: trigger initial configure run, too. See #167
• Demo persona: switch to FAKE DOI provider by default. See #167
• Guide: add more information about how to handle secrets. See #158
• Guide: add example script to deploy Secrets from a KeePassX file. See #158

🐛 Bug fixes, 🧠 workarounds and 🦎 tweaks

• Store uploads in emptyDir to avoid container base filesystem bashing. See #179, not this is a workaround.
• Make Sphinx docs generation on Jenkins running again. See #172
• Update JDBC Postgres driver to 42.2.10. See #168
• Remove selectors from all PersistantVolumeClaims to be as agnostic as possible for all cloud environments. See #154
• Fix a regression when running with a sidecar for SSL termination where DV API could not be reached anymore. See #184
• Make manual image build triggering in Jenkins possible (Jenkinsfile tweak). See 44ecabb
• Guide: add note about podman build options to successfully build images without Docker. See c36d50d

🤖 General Notes

Please be aware that future enhancements to scripting and more used to deploy and configure Dataverse will not be added to the release. Thus docker images for this will not be updated.

Please be aware, that tagged images are not automatically updated by Kubernetes when updated on Docker Hub (image pull policy). See the guide for ideas.

Dataverse v4.16 on K8s

This release contains references to the upstream release 4.16 and will deploy it when used.

Hotfix Release

BEWARE: this is a hotfixed 4.16 release to fix an issue in Jenkins, see #128 You should use the latest release, currently being 4.18.1

Pre-Deploy Actions

No special action seems to be necessary regarding to the upstream release docs.

• When using kubectl to upgrade Dataverse after upgrading Solr, you need to remove the Deployment first, as the selector labels have changed (and they are immutable):
  1. kubectl delete deployment dataverse
  2. kubectl apply -k k8s/dataverse
• Depending on your deployment mechanisms, you might need to add the /docroot PVC yourself. See below for context. You might use kubectl apply -k k8s/dataverse, but see above.

Post-Deploy Actions

• The core bundled citation metadata block has changed. If you are upgrading
  from an older release, you will need to upload the new schema and reexport.
  Please see metadata docs on how to do that.
• Remember when you upgrade from pre-v4.15, you will have to perform a re-index.
• None the less, please take a look at the rest of upstream release notes, too.

New and changed functionality

Use more Kustomize.io!

As Kubernetes 1.16 is out, people should really adopt using Kustomize.io bundled with kubectl since 1.14.
This project supports the usage and welcomes the relief of easily modifiable K8s objects. See that easy call in the README... 😉 In future releases, we will take this even further, see #91.

Default admin password changed, can provision from Secret

As of this release (4.16) you should be aware that the default
dataverseAdmin password is no longer admin, but admin1. As
IQSS/dataverse-ansible does the same, let's be consistent. See #75 for details.

Persistance for dataverse logos, sitemap, etc

Since this release, Dataverse will save logos for dataverses, its sitemap
and other statefull information at /docroot of the container. See #65.

A PersistentVolumeClaim and volume mount has been added. For now this is
in ReadWriteOnce mode. For using it with a scaled, multi-instance deployment,
that should be changed to ReadWriteMany. See also #10.

Support for running any branch in K8s

This release adds initial support for running any branch in Kubernetes by using
local Kind clusters and Skaffold for building and deploying images.

For more information, see the development-usage docs.

General Notes

Please be aware that future enhancements to scripting and more used to deploy and configure Dataverse will not be added to the release. Thus docker images for this will not be updated.

This is especially important in terms of security patches for images. This might change in the future with scheduled builds from CI.

Please be aware, that tagged images are not automatically updated by Kubernetes when updated on Docker Hub (image pull policy).

Dataverse v4.18.1 on K8s

This release contains references to the upstream release 4.18.1 and will deploy it when used.

Pre-Deploy Actions

No special action seems to be necessary regarding to the upstream release docs.

Post-Deploy Actions

On upstream, this has been a maintenance / bug fix release, so there are no special things to do when you are already running v4.18. If you're not on 4.18, please look at the earlier releases, too.

As always: Dataverse has release new functionality with this release. Be sure to read the upstream release notes, too.

New and changed functionality in dataverse-k8s

• None

General Notes

Please be aware that future enhancements to scripting and more used to deploy and configure Dataverse will not be added to the release. Thus docker images for this will not be updated.

Please be aware, that tagged images are not automatically updated by Kubernetes when updated on Docker Hub (image pull policy). See the guide for ideas.

Dataverse v4.18 on K8s

This release contains references to the upstream release 4.18 and will deploy it when used.

Pre-Deploy Actions

No special action seems to be necessary regarding to the upstream release docs.

You might need to change your Kustomize patches to reflect the new SECRETS_DIR location for dataverse-k8s.

Post-Deploy Actions

• The core bundled citation metadata block has changed. Simply follow the guide about "Metadata Blocks Job".

As always: Dataverse has release new functionality with this release. Be sure to read the upstream release notes, too.

New and changed functionality

See also v4.18 milestone

• Add support for provisioning BuiltinUsers.KEY
• Move SECRETS_DIR of dataverse-k8s to /secrets
• Support Prometheus JMX exporter agent in dataverse-k8s (see #82)

General Notes

Please be aware that future enhancements to scripting and more used to deploy and configure Dataverse will not be added to the release. Thus docker images for this will not be updated.

Please be aware, that tagged images are not automatically updated by Kubernetes when updated on Docker Hub (image pull policy). See the guide for ideas.

Dataverse v4.17 on K8s

This release contains references to the upstream release 4.17 and will deploy it when used.

Pre-Deploy Actions

No special action seems to be necessary regarding to the upstream release docs.

Post-Deploy Actions

• The core bundled citation metadata block has changed. Simply follow the guide about "Metadata Blocks Job".

New and changed functionality

Docs

The documentation has been completly refactored, rewritten in Sphinx and massively extended. It contains almost everything from the old docs, some minor things still missing (like AWS deployment, detailed reuse tips, proper roadmap, ...)

The new guide lives at https://dataverse-k8s.rtfd.io

Solr deployment

We now replace rather than rollout in parallel due to locking issues. Those are completely avoidable now.

Metadata Schema Config in Solr

We inherit the upstream flexibility in configuring the Solr schema for indexing metadata fields. See the guide for how to use it.

New dev build cache

As Maven is downloading the internet when building Dataverse with our multistage build, a build cache has been split out. That way we can avoid the slow downloads of many packages and instead just download the small changes. The cache is updated every night to latest upstream develop branch.

Image documentation

The container images have their own doc section in the guide now. Those are also pushed to Docker Hub automatically on release with Jenkins.

Many things more.

See also milestone v4.17

General Notes

Please be aware that future enhancements to scripting and more used to deploy and configure Dataverse will not be added to the release. Thus docker images for this will not be updated.

Please be aware, that tagged images are not automatically updated by Kubernetes when updated on Docker Hub (image pull policy). See the guide for ideas.

Dataverse v4.15.1 on K8s

This release contains references to the upstream release 4.15.1 and will deploy it when used.

Pre-Deploy Actions

When upgrading from pre-v4.15, you might need to adapt a few things. See release notes for v4.15. Otherwise you should be good to go.

Post-Deploy Actions

When upgrading from pre-v4.15, you might need to adapt a few things. See release notes for v4.15. Otherwise you should be good to go.

None the less, please take a look at the rest of upstream release notes, too.

General Notes

Please be aware that future enhancements to scripting and more used to deploy and configure Dataverse will not be added to the release. Thus docker images for this will not be updated.

This is especially important in terms of security patches for images. This might change in the future with scheduled builds from CI.

Please be aware, that tagged images are not automatically updated by Kubernetes when updated on Docker Hub (image pull policy).

Dataverse v4.15 on K8s

This release contains references to the upstream release 4.15 and will deploy it when used.

Pre-Deploy Actions

You might need to fix your users. Quoting from upstream release docs:

• In an effort to prevent accidental duplicate accounts, user spoofing, or other username-based confusion, this release introduces a database constraint that no longer allows usernames that are exactly the same but use different capitalization, e.g. Bob11 vs. bob11. You may need to do some cleanup before upgrading to deal with existing usernames like this.
• To check whether you have any usernames like this that need cleaning up, run the case insensitive duplicate queries from our Useful Queries doc.
• Once you identify the usernames that need cleaning up, you should use either Merge User Accounts (if it’s the same person) or Change User Identifier (if they are different people). After the cleanup you can safely upgrade without issue.

Post-Deploy Actions

• The Solr Schema changed, so you should do an inplace reindex.
  Have a look at the docs to learn how to do this within your K8s cluster easily.

None the less, please take a look at the rest of upstream release notes, too.

Notes:

• You don't need to deal with jhove.conf, this is done at container build time. Just deploy.
• Don't worry about the optional replacing of solrconfig.xml. Not necessary and would have been done at container build time.

General Notes

Please be aware that future enhancements to scripting and more used to deploy and configure Dataverse will not be added to the release. Thus docker images for this will not be updated.

This is especially important in terms of security patches for images. This might change in the future with scheduled builds from CI.

Please be aware, that tagged images are not automatically updated by Kubernetes when updated on Docker Hub (image pull policy).

Dataverse v4.14 on K8s

This release contains references to the upstream release 4.14 and will deploy it when used.

Post-Deploy Actions

None has been recommended or advised by upstream.
Not the less, please take a look at the rest of upstream release notes, too.

General Notes

Please be aware that future enhancements to scripting and more used to deploy and configure Dataverse will not be added to the release. Thus docker images for this will not be updated.

This is especially important in terms of security patches for images. This might change in the future with scheduled builds from CI.

Please be aware, that tagged images are not automatically updated by Kubernetes when updated on Docker Hub (image pull policy).

Dataverse v4.13 on K8s

This release contains references to the upstream release 4.13 and will deploy it when used.

Post-Deploy Actions

None has been recommended or advised by upstream.
Not the less, please take a look at the rest of upstream release notes, too.

General Notes

Please be aware that future enhancements to scripting and more used to deploy and configure Dataverse will not be added to the release. Thus docker images for this will not be updated.

This is especially important in terms of security patches for images. This might change in the future with scheduled builds from CI.

Please be aware, that tagged images are not automatically updated by Kubernetes when updated on Docker Hub (image pull policy).