New Fields architecture proposal

[wmertens]

Introduction

Work in Progress!

The subject of this design document is the architecture of the Admin UI and all related server parts.

I'm trying to find a minimal architecture that enables maximum flexibility, and writing things down as I go along, so this is half exploration, half manifesto. I'm hoping to highlight architectural problems in KS and find ways to fix those.

Please comment, I will be adjusting this over time.

Current Situation

• Fields can only be defined in Keystone source
• Admin UI code is compiled at runtime
• Field UI code is not straight React, so tooling doesn't work
• Each UI component has to maintain and mutate own state, error-prone and not DRY

Design goals

• minimal API surface for defining UI
• easy extensibility
• composable
• allow changing as much of the keystone forms UI as possible
• make UI components as pure as possible (only state for presentation)
• make use of Redux

Secondary goals

• make the keystone repo as lean as possible, move most field types to plugins
• all client javascript pre-built, lean deps for keystone
• field code pre-built or inline with server code
• Loosen hard dependency on Mongoose

Not in scope: look/feel of UI

Example use cases

	How edit UI changes	How list view changes	How filter changes	How db schema changes
section	section header	-	-	-
name mapping	form title	1 extra column	standard value filter	1 extra field
zipcode input	custom input	1 extra column	standard value filter	1 extra field
currency field	custom input	1 extra column	numeric filter, enum filter (used currencies)	1 nested object
input grouper	show child inputs on one line	child columns	child filters	adds child fields non-nested
action button, custom code on click	maybe button, perhaps in header or footer	maybe button	-	-
summary view	shows sums etc	-	if corresponding virtual, then value filter	maybe added virtual
i18n wrapper	edit same field for multiple languages	extra column per configured language	value filter for the field for each language	nested object with languages as keys
array wrapper	list view that allows adding/removing/ordering child field	column with first few items inline, column with item count	value filter, count filter	array with child field
upload field	hybrid field with upload button and current file details (* footnote 2)	columns: size, name, thumbnail	filters for size, name, mime type	opaque data as provided by the KS Storage instance
createdBy	read-only field in footer of form	column with name	name filter	as stored by track plugin

Glossary

Let's name the various parts of the puzzle.

• First of all, there are multiple presentations of the same data.
  • UI model: what the Admin UI sees and sends (* see footnote 1)
  • Service model: the data in memory in the server
  • DB model: the stored version of the data
  • In an ideal world, the UI model and Service model are the same. For example, a date might be an actual Date object, and it survives communication between server and UI as such.
• Keystone Model: defines a List by specifying form fields
  • automatically translated into all the above models by the server
  • includes hooks code
• List: Data entity, described by a KS Model
  • List UI: Everything in the Admin UI to manage the List
  • schema: DB description of the List, currently a Mongoose schema
  • form: UI description of the EditForm
  • columns: List of available columns for the ItemTable
  • filters: List of available filters for the ItemTable
  • item: Instance of a List item, according to the data model of the layer
• Field: The group name for:
  • EditField: React component that shows content in the EditForm
  • ColumnField: React component that shows content in the ItemTable
  • FilterForm: React component that shows UI to edit filters in the List view
  • reducer: Redux reducer with data for the field type
  • actions: Redux actions (e.g. button click handling)
  • FieldType: Server side code
    • registers the db schema and converts between server and db models
    • adds entry in the form
    • Transforms DB and UI data presentations

Analysis

From the examples, We conclude that specifiying a Field in a Model file produces:

• 0 or moreform entries, possibly in title, header or footer
• 0 or more columns entries
• 0 or more filters entries
• 0 or more flat or nested db schema fields
• optional reducer and actions for async state management

Client loading

The client UI needs

• form, columns, filters for each List (form includes field configuration)
• javascript code for client UI
• javascript code for used Field React Components, including Redux code
• UI model data for lists and items

form

• Title: {path: Array, editable: Boolean, ...?TODO} where the Title value can be found, and whether it can be edited, and if the editor should also show in the form
• For header, main and footer: Array of {type: String, props: Object, children: Array} objects naming the EditField and its properties
• Nesting is done with an array in the children property
• Note that actually referring to a path in the item is entirely optional
• JSON serializable

Type is a string because the React components need to be referenced by name, to allow loading Field UI code separately.

columns

TODO. What about nesting?

filters

TODO. What about nesting?

ListView: React Component

Provided by Keystone

EditForm: React Component

Provided by Keystone

Inputs:

• form
• UI model data

Output:
Element composing EditFields showing a form as configured by the KS Model.

EditField: React Component (interface)

Interface definition. Implementation provided by Field

Inputs:

• private:
  • properties derived from the Model field definition by the FieldType
  • path to data in item (array of keys)
  • optional label for input
• shared between EditFields:
  • item: current value of the entire edited object (UI model)
  • onChange, onMutate: functions to call to mutate the value, via Redux
  • original object before changes (UI model)

Output:

• React element that allows editing the data. Changes are done via the provided functions and applied with Redux

TODO not entirely sure if this should run connect() by itself.

By using react-redux's connect(), the EditField can get at the shared data in the Redux store and can bind actions to dispatch. The private inputs are provided as properties.

These extra derived properties are provided directly for ease of use and embedding:

• the value in the object at its path is provided directly, or undefined if the path does not exist.
• onChange(value) will dispatch the new value for the UI data model to the correct path in the object

A helper component, EditFieldWrapper, is made available that takes a field type and a path and returns the appropriate React Element with all the properties set up as above, for nesting. This means that you can e.g. embed a GeoPoint input that sets data at any point in the schema, or if you get access to the non-connect()-ed version you can simply show it as a bound input element and convert the output to whatever you like in your own data structure.

ColumnField: React Component (interface)

Interface definition. Implementation provided by Field

Inputs:

• private:
  • properties derived from the Model field definition by the FieldType
  • path to data in item (array of keys)
• shared between ColumnFields:
  • item: value of the entire object in the current row (UI model)

Output: contents for a cell in the ListView

FilterField: React Component (interface)

TODO

Footnotes

1. Note that in the current keystone, the UI model gets data in JSON but sends the data as read by FormData from the input fields and then passed as-is to the FieldType instances. This means that the UI model abstraction has to be known down to the individual field level (Number fields need to accept strings, Date fields get multiple strings that together make up a date etc). I am not happy with this, I would prefer the UI abstraction to stay in the UI, or at most be translated when it arrives at the server.
2. File fields are special because in general the storage data is only metadata, and an upload field is completely different from the UI model. Therefore they are a hybrid field that shows either a file as the browser will upload it, or the storage metadata.

[emagnier]

+1 to move all the fields in a separate repo(s) / module(s)

I don't use Keystone on my project but I need a few fields that are all in the Keystone project.
Instead of reinventing the wheel (or do an ugly integration), have them as plugins would also help to integrate them on any React projects.

And this would help contributions (even if we don't use the Keystone framework).

[gautamsi]

Keystone 4 is going in maintenance mode. Expect no major change. see #4913 for details.
Keystone v5 has decoupled architecture and have support for custom fields and field views.