Feeding data into universal rendering components

[ericelliott]

I've been looking at how to compile data from the server and make it available for the new universal React routing & rendering capability that was recently added to 0.4.x.

That data should obviously include user data, relevant lists, etc... but we don't want to pass it explicitly through the component view hierarchy. We haven't discussed the topic in-depth, but the community seems to be converging around Redux and it's rapidly expanding ecosystem of developer tools. There's a really great tutorial course by @gaearon.

Currently, it appears we take a bit of a wild west approach to bootstrapping the client page with data from the server, with no consistent user-extensible mechanism that I can see to determine what data is intended to be delivered to the client.

It also seems that there are bits and pieces of it being written by different page views (e.g., via viewLocals and arbitrary data being shoved directly into render() calls). Additionally, our current React admin session store doesn't appear to be designed for universal rendering and the Node request/response cycle. This stuff hasn't been a huge issue so far because users define their own views and can essentially stuff any data they want anywhere they want, but that approach won't work with baked in support for universal rendering.

Thankfully, both Express and React provide mechanisms that may be an effective path forward.

First, Express has a res.locals environment that is intended to be a safe way to gather data for the current request/response cycle, only. This is a good place to store things like authentication data, current user preferences, CSRF, "view locals", and so on. We should use it for its intended purpose, and create a special key that users of Keystone can easily extend with whatever custom data they need to generate on the server for delivery to the client.

Second, React has a context mechanism that seems to be a good candidate for this stuff. It's a way of implicitly passing data down through the entire React component hierarchy as opposed to explicitly passing everything from parent to children in props.

Receiving context is opt-in, and a React component must opt-in to the particular context keys that they're interested in. Additionally, Redux & React-Redux offer a simple way to pass store context into the app.

I'm thinking that we should create a special res.locals.context key that Keystone users can add to in their own middleware, and then we should ensure that it gets added to a Redux store and made available in the client bootstrap (currently we set a bunch of variables directly on a Keystone object on the client page... we can do something similar, or keep doing that).

I need a working mechanism to pass state into universal routes for both server and client rendering right now. I'm using it for language preferences and user data in the short-term. I'd love to hear your feedback.

[adamscybot]

This touches on the way the server currently communicates with React -- as you mention, by bootstrapping variables in jade templates.

I was surprised when I saw this. This is preventing keystone from being a true SPA with no page reloads. Errors are handled by reloading the page with a Keystone.errors variable bootstrapped in. Why does it not grab the error from the response body of an ajax request and modify the DOM with the error message appropriately? Isn't this the point in React? The project seems to have half-adopted React as it is now.

Also...all for redux. At the moment, the state store is bootstrapped variables in a script tag. We would need some other state store and redux is the way to go.

Very keen to hear what @JedWatson thinks as I can work on this.

[ericelliott]

This is preventing keystone from being a true SPA with no page reloads.

Isn't that more to do with the incomplete API? (Also high on my priority list). I see the data bootstrapping as a separate issue from SPA support -- it just provides initial data, not all the data all the time.

[adamscybot]

Yeh that's true for the List data. The exception would be transient data like Keystone.messages which is bootstrapped when there is an error. Error handling needs to be moved to parse errors from asynchronous API responses instead. At the moment, there are areas where this is the case and areas where it is not. Saving a model causes errors/messages to be inserted into Keystone.messages. However, deleting one does an ajax call but theres a TODO where the error handling code should be.

[ericelliott]

Wow. Seems like that needs a fix. Is there an open issue tracking that?

[adamscybot]

Not that I know of. Once I'm done with #1206 I will open one and hopefully get started on it.

[ericelliott]

👍

[w01fgang]

About feeding data components: I have tried to use the universal rendering. There is one problem: when component start to render the keystone database isn't started yet and I get response 502 from the server. The data fetched successfully after render on the client.
So for the start I use sample response form the servers api which I was saved and include for the store initial state.

I don't know what's happened, but now starts normal. I use store-prototype to pass data.

[ericelliott]

Data Injection RDD

Here is my proposal to add Redux & data injection capabilities to the Universal Routing and Rendering support.

Universal Route & Render with React & Redux

For Universal JavaScript projects, simply pass a few options into your Keystone.init(). For example, if you have the following smart component you'd like to use as a route:

import list from 'components/ui/list';
import { connect } from 'react-redux';

export default React => {
  const List = list(React);

  const component = (props) => <List { ...props }/>;

  const mapStateToProps = ({ teams, teamListClass, teamClass }) => ({
    list: teams,
    listClass: teamListClass,
    itemClass: teamClass
  });

  return connect(mapStateToProps)(component);
};

You'll need reducers defined for those props (reducers/index.js):

import teams from './teams';

export default {
  teams,
  teamClass: (state = '') => state, // just use the initial state
  teamListClass: (state = '') => state
};

Add the component to your routes in routes/react-routes:

import React from 'react';
import { Router, Route } from 'react-router';

import createTeams from 'components/teams';

const createRoutes = (React) => {
    // A route for a teams page
    const Teams = createTeams(React);

    return (
        <Router>
            <Route path='/teams' component={ Teams } />
        </Router>
    );
};

export default createRoutes;

You could can use it by importing it into your Keystone.js file, along with your Redux reducers:

import routes from './path/to/react-routes';
import reducers from './path/to/reducers';

// And in your Keystone.init() block:

Keystone.init({
    'react routes': routes, // expects a factory that takes React and returns a React Router element.
    'redux reducers': reducers, // expects an object { key1: reducer, key2: reducer }
    'redux middleware', // optional
    'react root', // optional DOM node ID to treat as render root. Default: keystone-wrapper
    // ...

Unit Testing Smart Components

If you'd like to unit test your smart components, you'll need to create a store. To facilitate that, there's a new keystone.createStore() utility that takes initialState, reducers, and reduxMiddleware parameters. You'll need to pass the store into your component props to get the rendered values:

const createStore = keystone.createStore;

test('Children', assert => {
    const msg = 'should render children';
    const teamClass = 'team';

    const initialState = {
      teams: [
        {
          name: 'A Team',
          id: '1'
        }, {
          name: `Charlie's Angels`,
          id: 2
        }, {
          name: 'Team CoCo',
          id: 3
        }
      ],
      teamClass,
      teamListClass: 'team-list'
    };

    const store = createStore({ initialState, reducers });

    const Teams = createTeams(React);
    const el = (
      <Teams store={ store } />
    );
    const $ = dom.load(render(el));

    const expected = 3;
    const actual = $(`.${ teamClass }`).length;

    assert.equal(actual, expected, msg);
    assert.end();
  });

Building Your Client App

In general, you're free to do whatever you want with your client app. Here's how to bootstrap it with the server-rendered data:

import React from 'react';
import universal from 'keystone/universal/client';

import routes from './path/to/react-routes';
import reducers from './path/to/reducers';

// returns a function that must be invoked to trigger render
const app = universal({ React, routes, reducers });

// The app function will return your store so you can dispatch actions.
const store = app();

// Do stuff in your client app to trigger re-renders.
// e.g., subscribe to server updates, etc...
store.dispatch({
  type: 'SET_TITLE',
  title: 'Client render'
});

Redux Middleware

You may want to pass Redux middleware into the store. Here's how to do it on the server:

Keystone.init({
    'redux middleware': reduxMiddleware,
    // ...

And in the client app, simply add it to the call to universal():

import routes from './path/to/react-routes';
import reducers from './path/to/reducers';

import reduxMiddleware from `./path/to/redux-middleware';

// Add middleware here:
const app = universal({ React, routes, reducers, reduxMiddleware });

Injecting Data from the Server

Data is made available to React components via the react-redux context. To add to it on the server, simply add data keys to res.locals.context. Data in res.locals.context will be assigned to initialState, overriding whatever defaults exist in your reducers.

Important: Adding data to res.locals.context will make data available to both the server view and the client view. Be careful not to expose sensitive information to the client.

---

Note: This bit needs discussion and fleshing out. I won't implement it in the first pass.

Automatic Context Data

Some data will be made available to your app's stores automatically:

• language
• user
• ...

[gautamsi]

Keystone 4 is going in maintenance mode. Expect no major change. see #4913 for details.
Keystone v5 is using GraphQL extensively and one of the few examples are based on SSR with Next.js for React.