Update Kubeflow Installation with Standalone Mode #3724

andreyvelich · 2024-04-28T19:29:17Z

As we discussed multiple times before, we want to explain our users that Kubeflow components can be deployed:

in Standalone mode.
in Kubeflow Platform for end-to-end AI/ML experience.

In this PR I made changes to the installation section:
I removed What is Kubeflow? section from this page since we already have this section here: https://www.kubeflow.org/docs/started/introduction/#what-is-kubeflow, we can update it if you think it is out-of-date.

I made these sections:

Install Kubeflow Components Standalone
Install Kubeflow Platform from Raw Manifests
Install Kubeflow Platform from Packaged Distributions

From my point of view this is logically correct, for users to learn about Kubeflow ecosystem. Firstly, users will explore the value of individual Kubeflow components. Secondly, if users need end-to-end AI/ML experience with all Kubeflow components, they will explore the Raw Manifests and Package Distributions installation.

What do you think about it ?

I am happy to work on the language to make it less confusing for our users if you have other ideas on how we can improve it.

/hold for review

/assign @kubeflow/wg-notebooks-leads @kubeflow/wg-pipeline-leads @kubeflow/wg-training-leads @kubeflow/wg-data-leads @kubeflow/release-team @kubeflow/kubeflow-steering-committee @kubeflow/wg-deployment-leads

tarilabs

I like this a lot, I believe it is important to make evident that you can opt-in to a subset of specific components and not "necessarily be force to the whole" (which was never the case).

Hence, I believe it's good the section "Install Kubeflow Components Standalone" comes first.

Maybe we can add some/additional brief paragraph when to chose either, for example:

for Components-Standalone: helpful when you want to try out locally or in your Kubernetes installation a specific component
for Raw-Manifests, in addition to being the basis for distribution (which is what is currently mentioned): helpful when you want to try out the end-to-end Kubeflow platform capabilities, or see how a specific component integrates in the Kubeflow ecosystem
for Packages Distributions: nothing to add here

for considerations, hope this helps

tarilabs · 2024-04-29T16:37:12Z

content/en/docs/started/installing-kubeflow.md

@@ -5,30 +5,65 @@ weight = 20

 +++

-## What is Kubeflow?
+Kubeflow is the ecosystem of various applications created to address each stage in the AI/ML lifecycle,


Suggested change

Kubeflow is the ecosystem of various applications created to address each stage in the AI/ML lifecycle,

Kubeflow is an ecosystem of various applications created to address each stage in the AI/ML lifecycle,

thesuperzapper · 2024-04-29T18:13:39Z

I think this update is much more confusing to new users and significantly confuses the message of "What is Kubeflow?".

To be clear, I am not against telling users that some components of Kubeflow can be installed on their own, but we need to be careful not to confuse people or make representations of support that are not true.

Alternate Proposal

The current page is actually pretty good at explaining the options users have to install Kubeflow.

I propose we simply add a 3rd option under "How to install Kubeflow?" so we have the following (in this order):

Packaged Distributions
Raw Manifests ^{(advanced users)}
Standalone Components ^{(advanced users)}

We also must get firm commitments from the maintainers of components listed under "Standalone Components". Users will expect deployment support, and don't want to impose that on maintainers without their agreement.

hbelmiro

I agree with removing "What is Kubeflow?" from here.
Regarding the order of the types of installations, I don't see much difference in which one goes first. As long as we mention three types of installations in the beginning, the order doesn't seem to matter to me.

hbelmiro · 2024-04-29T19:45:41Z

content/en/docs/started/installing-kubeflow.md

+- Install [Kubeflow Pipelines](/docs/components/pipelines/v2/installation/quickstart/)
+- Install [Kubeflow Training Operator](/docs/components/training/installation/#installing-training-operator)
+- Install [Kubeflow MPI Operator](/docs/components/training/user-guides/mpi/#installation)
+- Install [Kubeflow Katib] (TODO: Add link after #3723)


I'm not sure about adding "TODO" here. Maybe "Not yet available", or something like that would be better. Also, adding to kubeflow/katib#2314 a task to update the link here.

I want to merge this PR after Katib and Model registry docs will be updated: #3723, #3698.
So I can add the appropriate links here.

hbelmiro · 2024-04-29T19:47:12Z

content/en/docs/started/installing-kubeflow.md

+- Install [Kubeflow MPI Operator](/docs/components/training/user-guides/mpi/#installation)
+- Install [Kubeflow Katib] (TODO: Add link after #3723)
+- Install [KServe](https://kserve.github.io/website/0.10/admin/serverless/serverless/)
+- Install Kubeflow Model Registry (TODO: Add link after #3698)


Same as above, but using the respective Model Registry issue.

andreyvelich · 2024-04-29T20:12:40Z

helpful when you want to try out locally or in your Kubernetes installation a specific component

@tarilabs I added your suggestion to the manifests installation, thanks! For the standalone installation, do you think these messages are not sufficient to explain value of Standalone Components ?

Kubeflow components can be deployed as standalone applications.
You can integrate those components to your existing AI/ML platform.
For example, for Model Training you can install Training Operator or for Model Serving you can install KServe.

andreyvelich · 2024-04-29T20:19:53Z

I think this update is much more confusing to new users and significantly confuses the message of "What is Kubeflow?".

@thesuperzapper Please can you explain why this update confuses users about Kubeflow Ecosystem value ?
From my perspective we make it very clear that Kubeflow project can be used in various ways.
The main motivation is to give new Kubeflow components (Model Registry, Spark Operator) visibility and grow the community around it (outside of Kubeflow Platform scope).

To be clear, I am not against telling users that some components of Kubeflow can be installed on their own, but we need to be careful not to confuse people or make representations of support that are not true.

Which message shows incorrect representation of support ? If users deploy Kubeflow components standalone and get some potential bugs/integrations issues, they should raise it to the components owners to get support.

Raw Manifests (advanced users)

What is the end-user value to tell that Raw Manifest and Standalone Installation is for advanced users ?
Some Kubeflow components are very light-weight and easy to install.

Users will expect deployment support, and don't want to impose that on maintainers without their agreement.

I am happy to remove some of the Kubeflow components from this page if WG leads don't want their components to be presented here.

andreyvelich · 2024-04-29T20:23:59Z

cc @jeremyeder @bigsur0 Would love to get your feedback for this change as well.

thesuperzapper · 2024-04-29T23:47:38Z

@andreyvelich

I have made an alternative PR which tries to be less confusing:

Add "Standalone Components" section to "Installing Kubeflow" #3726

It is a much smaller change to the structure, and uses tables to format the information about if each component can be deployed standalone.

If you agree that its better, lets move the discussion to that PR and close this.

juliusvonkohout · 2024-04-30T08:06:16Z

@andreyvelich I have made an alternative PR which tries to be less confusing:
* [Add "Standalone Components" section to "Installing Kubeflow" #3726](https://github.com/kubeflow/website/pull/3726)
It is a much smaller change to the structure, and uses tables to format the information about if each component can be deployed standalone.

If you agree that its better, lets move the discussion to that PR and close this.

Doesnt it change this order here significantly?

- [**Install Kubeflow Components Standalone**](#install-kubeflow-components-standalone)
- [**Install Kubeflow Platform from Raw Manifests**](#install-kubeflow-platform-from-raw-manifests)
- [**Install Kubeflow Platform from Packaged Distributions**](#install-kubeflow-platform-from-packages-distributions)

seems to become

- [**Install Kubeflow Platform from Packaged Distributions**](#install-kubeflow-platform-from-packages-distributions)
- [**Install Kubeflow Components Standalone**](#install-kubeflow-components-standalone)
- [**Install Kubeflow Platform from Raw Manifests**](#install-kubeflow-platform-from-raw-manifests)

tarilabs · 2024-04-30T08:40:18Z

@tarilabs I added your suggestion to the manifests installation, thanks! For the standalone installation, do you think these messages are not sufficient to explain value of Standalone Components ?

glad that helped, about the standalone very-imho is worthy to call out testing a component "locally" or in isolation, in addition to the existing AI/ML/K8s one might have.
for example, as a relatively newcomer to Kubeflow, I thought I needed it all if I wanted to try one single part :)
my2c, of course I can understand if this comment is not incorporated! 🚀 👍

andreyvelich · 2024-04-30T11:13:38Z

@andreyvelich I have made an alternative PR which tries to be less confusing:

Add "Standalone Components" section to "Installing Kubeflow" #3726

It is a much smaller change to the structure, and uses tables to format the information about if each component can be deployed standalone.

If you agree that its better, lets move the discussion to that PR and close this.

@thesuperzapper As we discussed, I am happy to incorporate your feedback in this PR based on our discussion during today's community call. A few thoughts from my side:

If you don't like the ordering of installation instructions we can make these 2 sections:

Install Kubeflow Components Standalone
Install Kubeflow Platform

In that case, we can move Distributions + Raw Manifests to the Install Kubeflow Platform section.

From my point of view, it doesn't give a lot of value for users to add Supports Standalone column to the table. We can always update the table with other Kubeflow Component, if it supports it. For example, we can add Kubeflow Notebooks in the future.

I am not sure if we should add the Note about limited or unavailable feature section to the Standalone Components. We can add the section to the Kubeflow Introduction:

What is Kubeflow ?
What is Kubeflow Standalone Components ?
What is Kubeflow Platform ?

And explain the differences there.

andreyvelich · 2024-04-30T21:03:52Z

Thanks everyone who participated in our today's discussion to show the real power of Kubeflow ecosystem.
I strongly believe that this change will help us to grow the community especially with the new project adoption: Spark Operator and Model Registry.

I want to summarize our discussion today about this PR:

We agree that it makes sense to explain that Kubeflow Components can be deployed standalone.
We want to collect feedback from distribution owners for this change.
We should re-use this ML Lifecycle diagram that @franciscojavierarceo and I created for Training Operator and re-use it in the ML Workflow guide
We are discussing to add the following statement to the installation page:
What is Kubeflow ?
Example text:

Kubeflow is an ecosystem of various applications created to address each stage in the AI/ML lifecycle,
from exploration to training and serving. You can deploy Kubeflow components as standalone applications
or deploy Kubeflow as an end-to-end AI/ML platform. Anywhere you are running Kubernetes,
you should be able to run Kubeflow.

What is Kubeflow Standalone Components ?

TODO: Add text

What is Kubeflow Platform ?

TODO: Add text

If we agree with that, we can add these subsections to the installation page:
Installing Kubeflow Standalone Components
Installing Kubeflow Platform

In the future PRs we can split them between 2 separate pages if we want.

If we agree, I can keep (Advanced User) and existing warnings for manifests installation in this PR. We can make followup discussion if we should change it.
I am happy to take the table from PR: Add "Standalone Components" section to "Installing Kubeflow" #3726, with these columns: Kubeflow Component, Source Code.
As I said before: Update Kubeflow Installation with Standalone Mode #3724 (comment) I am not sure if we should have "Supported Standalone" section.

Please let me know if I missed something.

@thesuperzapper Please can you share recordings from our today's call.

andreyvelich · 2024-04-30T21:04:41Z

Distribution owners please give your feedback for this PR.
/cc @ca-scribner @gkcalat @zijianjoy @Linchin @Tomcli @yhwang @johnugeorge @nagar-ajay @rimolive @thesuperzapper @liuqi @xujinheng @alexeadem

google-oss-prow · 2024-04-30T21:04:53Z

@andreyvelich: GitHub didn't allow me to request PR reviews from the following users: Linchin, alexeadem.

Note that only kubeflow members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

Distribution owners please give your feedback for this PR.
/cc @ca-scribner @gkcalat @zijianjoy @Linchin @Tomcli @yhwang @johnugeorge @nagar-ajay @rimolive @thesuperzapper @liuqi @xujinheng @alexeadem

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Signed-off-by: Andrey Velichkevich <[email protected]>

andreyvelich · 2024-05-23T11:05:47Z

@kubeflow/wg-manifests-leads @jbottum @terrytangyuan I renamed Raw Manifests to Kubeflow Manifests.

Signed-off-by: Andrey Velichkevich <[email protected]>

terrytangyuan · 2024-05-24T02:33:07Z

/lgtm

Signed-off-by: Andrey Velichkevich <[email protected]>

thesuperzapper

@andreyvelich I am happy with the page now, and will LGTM once these small typos are fixed.

thesuperzapper · 2024-05-27T21:28:15Z

content/en/docs/started/installing-kubeflow.md

+You can use one of the following methods to install [Kubeflow Platform](/docs/started/introduction/#what-is-kubeflow-platform)
+to get full suite of standalone Kubeflow components bundled together with additional tools.


The sentence had a few typos, and I dont think we need to say "standalone" as "kubeflow components" is more accurate in this case:

Suggested change

You can use one of the following methods to install [Kubeflow Platform](/docs/started/introduction/#what-is-kubeflow-platform)

to get full suite of standalone Kubeflow components bundled together with additional tools.

You can use one of the following methods to install the [Kubeflow Platform](/docs/started/introduction/#what-is-kubeflow-platform)

and get the full suite of Kubeflow components bundled together with additional tools.

thesuperzapper · 2024-05-27T21:35:53Z

content/en/docs/started/installing-kubeflow.md


-Advanced users may choose to install the manifests for a specific Kubeflow version by following the
+Kubeflow Manifests contain all Kubeflow Components, Kubeflow Central Dashboard, and other Kubeflow
+applications which makes **Kubeflow Platform**. This installation is helpful when you want to try


Its not valid to use "makes" in this context:

Suggested change

applications which makes **Kubeflow Platform**. This installation is helpful when you want to try

applications that comprise the **Kubeflow Platform**. This installation is helpful when you want to try

thesuperzapper · 2024-05-27T21:36:21Z

content/en/docs/started/installing-kubeflow.md


-When using the raw manifests, the Kubeflow community is not able to provide support for environment-specific issues or custom configurations.
+When using the Kubeflow manifests, the community is not able to provide support for environment-specific issues or custom configurations.
 If you need support, please consider using a [packaged distribution](#packaged-distributions-of-kubeflow).


We renamed this heading, so the link needs to be updated:

Suggested change

If you need support, please consider using a [packaged distribution](#packaged-distributions-of-kubeflow).

If you need support, please consider using a [packaged distribution](#packaged-distributions).

thesuperzapper

@andreyvelich I am happy with the page now, and will LGTM once these small typos are fixed.

Signed-off-by: Andrey Velichkevich <[email protected]>

andreyvelich · 2024-05-28T13:03:10Z

I believe, I addressed all of your comments.

terrytangyuan · 2024-05-28T13:05:18Z

/lgtm

james-jwu · 2024-05-28T17:23:34Z

/lgtm

thesuperzapper · 2024-05-28T17:50:14Z

/lgtm

johnugeorge · 2024-05-28T18:14:13Z

/lgtm

juliusvonkohout · 2024-05-29T08:18:05Z

@andreyvelich If we can address #3724 (comment)

In a follow up PR

I can also
/lgtm

thesuperzapper · 2024-05-29T15:58:32Z

@andreyvelich If we can address #3724 (comment)

@juliusvonkohout that warning is important to make sure that people understand that using the manifests requires some advanced knowledge. It's very important that we keep it to prevent novice users trying to deploy the manifests (without knowing what they're getting into).

If you think the wording could be improved, let's discuss that in a follow up PR, because that warning already exists on the current page, so it's not really related to this PR and I really want to get this merged.

andreyvelich

Thanks everyone for your review, I appreciate your time and effort for this!
/hold cancel

google-oss-prow bot added the do-not-merge/hold label Apr 28, 2024

google-oss-prow bot requested review from alfsuse and knkski April 28, 2024 19:29

google-oss-prow bot added approved size/M labels Apr 28, 2024

tarilabs reviewed Apr 29, 2024

View reviewed changes

hbelmiro reviewed Apr 29, 2024

View reviewed changes

thesuperzapper mentioned this pull request Apr 29, 2024

Add "Standalone Components" section to "Installing Kubeflow" #3726

Closed

google-oss-prow bot requested review from zijianjoy, yhwang, Tomcli, rimolive, liuqi, gkcalat, johnugeorge, thesuperzapper, ca-scribner, nagar-ajay and xujinheng April 30, 2024 21:04

andreyvelich added 2 commits May 22, 2024 23:14

Add link to Kubeflow Platform

fb14fb1

Signed-off-by: Andrey Velichkevich <[email protected]>

Rename Raw Manifests to Kubeflow Manifests

e18ac08

Signed-off-by: Andrey Velichkevich <[email protected]>

Change H3

7df289c

Signed-off-by: Andrey Velichkevich <[email protected]>

google-oss-prow bot added the lgtm label May 24, 2024

This was referenced May 24, 2024

Kubeflow v1.8 distributions kubeflow/kubeflow#7411

Closed

Optionally require auth token/header for all dashboard requests kubeflow/kubeflow#7498

Open

Rename to quick and easy

90df53d

Signed-off-by: Andrey Velichkevich <[email protected]>

google-oss-prow bot removed the lgtm label May 25, 2024

andreyvelich added 2 commits May 25, 2024 22:34

Remove new line in Introduction

63c3a61

Signed-off-by: Andrey Velichkevich <[email protected]>

Modify the install standalone section

b0de894

Signed-off-by: Andrey Velichkevich <[email protected]>

thesuperzapper requested changes May 27, 2024

View reviewed changes

Fix install guide

7e7b91c

Signed-off-by: Andrey Velichkevich <[email protected]>

google-oss-prow bot added the lgtm label May 28, 2024

andreyvelich commented May 29, 2024

View reviewed changes

google-oss-prow bot removed the do-not-merge/hold label May 29, 2024

google-oss-prow bot merged commit 44fbaf9 into kubeflow:master May 29, 2024
6 checks passed

andreyvelich deleted the update-kubeflow-install branch May 29, 2024 16:06

thesuperzapper mentioned this pull request May 29, 2024

Update deployKF version to 1.8 #3746

Merged

juliusvonkohout mentioned this pull request Jun 3, 2024

Update manifests warning #3748

Merged

thesuperzapper mentioned this pull request Jun 11, 2024

Fix links to packaged distributions section #3758

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Kubeflow Installation with Standalone Mode #3724

Update Kubeflow Installation with Standalone Mode #3724

andreyvelich commented Apr 28, 2024

tarilabs left a comment

tarilabs Apr 29, 2024

thesuperzapper commented Apr 29, 2024

hbelmiro left a comment

hbelmiro Apr 29, 2024 •

edited

Loading

andreyvelich Apr 29, 2024

hbelmiro Apr 29, 2024

andreyvelich commented Apr 29, 2024

andreyvelich commented Apr 29, 2024 •

edited

Loading

andreyvelich commented Apr 29, 2024

thesuperzapper commented Apr 29, 2024 •

edited

Loading

juliusvonkohout commented Apr 30, 2024

tarilabs commented Apr 30, 2024

andreyvelich commented Apr 30, 2024 •

edited

Loading

andreyvelich commented Apr 30, 2024

andreyvelich commented Apr 30, 2024

google-oss-prow bot commented Apr 30, 2024

andreyvelich commented May 23, 2024

terrytangyuan commented May 24, 2024

thesuperzapper left a comment

thesuperzapper May 27, 2024

thesuperzapper May 27, 2024

thesuperzapper May 27, 2024

thesuperzapper left a comment

andreyvelich commented May 28, 2024

terrytangyuan commented May 28, 2024

james-jwu commented May 28, 2024

thesuperzapper commented May 28, 2024

johnugeorge commented May 28, 2024

juliusvonkohout commented May 29, 2024

thesuperzapper commented May 29, 2024 •

edited

Loading

andreyvelich left a comment

	Kubeflow is the ecosystem of various applications created to address each stage in the AI/ML lifecycle,
	Kubeflow is an ecosystem of various applications created to address each stage in the AI/ML lifecycle,

		You can use one of the following methods to install [Kubeflow Platform](/docs/started/introduction/#what-is-kubeflow-platform)
		to get full suite of standalone Kubeflow components bundled together with additional tools.

	applications which makes Kubeflow Platform. This installation is helpful when you want to try
	applications that comprise the Kubeflow Platform. This installation is helpful when you want to try

	If you need support, please consider using a [packaged distribution](#packaged-distributions-of-kubeflow).
	If you need support, please consider using a [packaged distribution](#packaged-distributions).

Update Kubeflow Installation with Standalone Mode #3724

Update Kubeflow Installation with Standalone Mode #3724

Conversation

andreyvelich commented Apr 28, 2024

tarilabs left a comment

Choose a reason for hiding this comment

tarilabs Apr 29, 2024

Choose a reason for hiding this comment

thesuperzapper commented Apr 29, 2024

Alternate Proposal

hbelmiro left a comment

Choose a reason for hiding this comment

hbelmiro Apr 29, 2024 • edited Loading

Choose a reason for hiding this comment

andreyvelich Apr 29, 2024

Choose a reason for hiding this comment

hbelmiro Apr 29, 2024

Choose a reason for hiding this comment

andreyvelich commented Apr 29, 2024

andreyvelich commented Apr 29, 2024 • edited Loading

andreyvelich commented Apr 29, 2024

thesuperzapper commented Apr 29, 2024 • edited Loading

I have made an alternative PR which tries to be less confusing:

juliusvonkohout commented Apr 30, 2024

tarilabs commented Apr 30, 2024

andreyvelich commented Apr 30, 2024 • edited Loading

andreyvelich commented Apr 30, 2024

andreyvelich commented Apr 30, 2024

google-oss-prow bot commented Apr 30, 2024

andreyvelich commented May 23, 2024

terrytangyuan commented May 24, 2024

thesuperzapper left a comment

Choose a reason for hiding this comment

thesuperzapper May 27, 2024

Choose a reason for hiding this comment

thesuperzapper May 27, 2024

Choose a reason for hiding this comment

thesuperzapper May 27, 2024

Choose a reason for hiding this comment

thesuperzapper left a comment

Choose a reason for hiding this comment

andreyvelich commented May 28, 2024

terrytangyuan commented May 28, 2024

james-jwu commented May 28, 2024

thesuperzapper commented May 28, 2024

johnugeorge commented May 28, 2024

juliusvonkohout commented May 29, 2024

thesuperzapper commented May 29, 2024 • edited Loading

andreyvelich left a comment

Choose a reason for hiding this comment

hbelmiro Apr 29, 2024 •

edited

Loading

andreyvelich commented Apr 29, 2024 •

edited

Loading

thesuperzapper commented Apr 29, 2024 •

edited

Loading

andreyvelich commented Apr 30, 2024 •

edited

Loading

thesuperzapper commented May 29, 2024 •

edited

Loading